Shahul

Creators of Ragas, Shahul and Jithin, discuss challenges in building LLM applications, emphasizing the importance of evaluation, data quality, and continuous RAG evolution. Practical takeaways include starting with a solid testing strategy and embracing synthetic data to automate test data set creation.

Shahul, involved in the open-source RAGAS project, joins the discussion on metrics-driven development for LLM applications. He sheds light on the critical differences between evaluating models and their applications, emphasizing the need for tailored assessments. The conversation delves into the role of synthetic test data, and how innovative speech AI models convert voice data into actionable insights. Shahul also highlights the promise of improved evaluation standards and the future possibilities of LLM applications powered by tool use and enhanced metrics.