Jason Liu, creator of Instructor, a leading LLM framework, discusses the benefits of structured output parsing with language models and the future of function calling. The podcast also explores selecting tools, organizing prompts, using Go library for reranking, and the capabilities of modal API for serverless GPU compute. The importance of fine-tuning embedding models for improved performance is emphasized.
Instructor simplifies code and improves the IDE experience by providing type hints and accurate data structures.
Fine-tuning embedding models on specific company data boosts performance and delivers personalized results.
Instructor focuses on making code easier to read, write, and understand, enhancing productivity and code quality.
Deep dives
The Power of Instructor: Simplifying Code and Providing Type Hints
Instructor is a library that aims to simplify code by providing type hints and improving the IDE experience. It allows developers to communicate with language models effectively and describe data structures more accurately. The goal of Instructor is to make programming with language models easier and more efficient, enabling better productivity and code quality.
Fine-Tuning Embedding Models and the Business Value
Fine-tuning embedding models can significantly improve their performance and enable network effects in various applications. By fine-tuning on specific company data, developers can differentiate their models, boost performance, and deliver more personalized and accurate results. The key is to focus on specific verticals and tailor the embedding models to meet specific business outcomes.
The Future of Instructor: Building a Sharp Knife and Teaching Knife Skills
The focus of Instructor is to provide a powerful tool for developers without creating a complex framework. It aims to improve the developer experience, making code easier to read, write, and understand. While Instructor itself may not become a business, its core philosophy can be applied to consulting work, improving AI systems, and sharing knowledge to enhance productivity and code quality.
Scaling Out with Modal and the Speed of Vector Databases
Modal, an API for serverless GPU compute, offers an efficient way to scale out GPU processing. This can be beneficial for tasks like vector search, where GPUs can provide faster and more efficient processing. By leveraging the power of GPUs, developers can achieve higher speeds and optimize costs, making large-scale processing more accessible.
Using Instructor for Evaluation, Moderation, and Optimization
Instructor can be used for evaluation and moderation, allowing developers to define validation rules, implement moderation features, and ensure safe and ethical AI applications. By leveraging the context-based validation offered by Instructor, developers can create powerful tools for content moderation and error handling, enhancing the safety and usability of their applications.
Jason Liu is the creator of Instructor, one of the world's leading LLM frameworks, particularly focused on structured output parsing with LLMs, or as Jason puts it "making LLMs more backwards compatible". It is hard to understand the impact of Instructor, this is truly leading us to the next era of LLM programming. It was such an honor chatting with Jason, his experience currently as an independent consultant and previously engineering at StitchFix and Meta makes him truly one of the most unique guests we have featured on the Weaviate podcast! I hope you enjoy the podcast!
Get the Snipd podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share & Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered podcast player
Listen to all your favourite podcasts with AI-powered features
Discover highlights
Listen to the best highlights from the podcasts you love and dive into the full episode