The Complexity of Fine Tuning AI Models and Dealing with Downtimes in External APIs

MLOps Coffee Sessions #173 with Beyang Liu, Building Cody, an Open Source AI Coding Assistant. We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O // Abstract Root about the development of Cody, an open-source AI coding assistant. Cody empowers developers to query and comprehend code within codebases through the integration of robust language model capabilities. Sourcegraph tackles the intricacies of understanding intricate codebases by creating comprehensive code maps and employing AI for advanced search functionalities. Cody harnesses the potential of AI to offer features such as code exploration, natural language queries, and AI-powered code generation, augmenting developer productivity and code comprehension. // Bio Beyang Liu is the CTO and Co-founder of Sourcegraph. Prior to Sourcegraph, Beyang was an engineer at Palantir Technologies building large-scale data analysis tools for Fortune 500 companies with large, complex codebases. Beyang studied computer science at Stanford, where he discovered his love for compilers and published some machine learning research as a member of the Stanford AI Lab. // MLOps Jobs board https://mlops.pallet.xyz/jobs // MLOps Swag/Merch https://mlops-community.myshopify.com/ // Related Links ⁠Website: https://beyang.com --------------- ✌️Connect With Us ✌️ ------------- Join our slack community: https://go.mlops.community/slack Follow us on Twitter: @mlopscommunity Sign up for the next meetup: https://go.mlops.community/register Catch all episodes, blogs, newsletters, and more: https://mlops.community/ Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/ Connect with Beyang on LinkedIn: https://www.linkedin.com/in/beyang-liu/ Timestamps: [00:00] Beyang's preferred coffee [00:19] Takeaways [01:25] Please like, share, and subscribe to our MLOps channels! [01:48] Beyang background before Sourcegraph [03:10] War stories [04:30] Technological tool solution [06:41] Landscape change in the past 10 years [09:32] Code search engine evolution [16:28] Vector databases [17:40] Actual tech breakdown [19:52] Incorporating AI into products amid organizational challenges [25:39] Breaking down Cody [28:04] Context fetching [30:44] AI replicating human code understanding? [36:22] Key for software creation [40:26] Speak the language [42:20] Leveraging LLMs [44:18] Low code, no code movement [47:54] Reliability issues amongst agents [53:12] LLMs used in code and chat generation [56:12] Dealing with rate limits and followers or failovers [57:33] Unnecessary comparison [1:00:26] Wrap up