Large language models present challenges and opportunities in the industry.
Transition from task-specific to more general models requires human evaluation for user preferences.
Deep dives
Discussing Large Language Models in Production
Large language models offer challenges in industry, but also tremendous opportunities. Companies like Replied are navigating decisions on using pre-trained models or building bespoke models for specific use cases.
Experience with Training and Deploying Bespoke Models
Replied has ventured into training bespoke models for chatbots, such as their Ghostwriter model. Evaluations such as human eval have shown user preferences and adoption rates for new vs. existing models.
The Evolution of LLMs and Benchmarking Challenges
The field of large language models is evolving rapidly, transitioning from task-specific models toward more general models. Benchmarking models like human eval reveal real-world user preferences over purely metric-driven evaluations.
In this second installment of AI panels at Replit, Michele, Chip, and Amjad discuss putting Large Language Models (LLMs) into production. The challenges and opportunities offered by LLMs are considered, as well as their potential impact on the industry.