Weaviate Podcast

Hey everyone, thank you so much for watching the 51st episode of the Weaviate Podcast with Greg Kamradt and Colin Harmon! Greg and Colin are both entrepreneurs in the space of new AI tools powered by LLMs! This podcast is about keeping up with the evolution of LLM Agents from AutoGPT to connecting LLMs with Vector Databases or Wolfram Alpha, as well as the ChatGPT Marketplace, Personalized LLMs, Private LLMs, and many more! I think there are so many interesting nuggets from this podcast, thank you so much to Greg and Colin for joining, really enjoyed this one! Data Independent: https://www.youtube.com/@DataIndependent Greg Kamradt on Twitter: https://twitter.com/GregKamradt Nesh: https://hellonesh.io/ Colin Harmon on LinkedIn: https://www.linkedin.com/in/coluha/ Colin Harmon Blog: https://colinharman.substack.com/ Colin Harmon at Haystack US 2023: https://www.youtube.com/watch?v=LO3U5iqnTpk Chapters 0:00 Introduction 0:42 Backgrounds 2:43 Defining “LLM Agents” 6:12 Data-Aware LLMs 13:04 Tool Use 13:38 ChatGPT API vs. Marketplace 17:40 Personalized LLMs, LLM for Greg 19:20 PrivateGPT 25:14 AutoGPT and Chain-of-Thought Prompting 32:30 Few-Shot Examples 35:30 Early AI Signals and Open-Source 43:10 Multi-Agent LLMs 47:14 Fine-Tuning and Long Input Lengths 52:20 Greg’s LLM Wishlist Hierarchy 53:15 Keeping up with Greg and Colin!

This video explores a new paper exploring the use of summarization chains to represent long texts and use (original text, summary) pairs for optimizing text embeddings models! Here are 3 main takeaways I think everyone working with Weaviate may get value from: 1. Understanding of Summary Indexing and the Prompts (as well as Prompt Chains) used to build them. 2. Continued development of LLM-generated data for search -- creating (full text, summary) pairs gives you (1) data to build a summary index with as mentioned, (2) data to compare different embedding models with, and (3) data to train your own embedding model. 3. Tournament style evaluation with human annotators -- the top 5 retrieved texts from one model are concatenated with the top 5 from another model, these 10 are given to human annotators to pick 5 and this is how the authors are reporting the performance of their models rather than traditional benchmarks. This m ay be a more productive evaluation technique for most real world search applications. Thank you so much for watching, here are some links mentioned in the video! Retrieving Texts based on Abstract Descriptions: https://arxiv.org/abs/2305.12517 Weaviate Blog - Combining LangChain and Weaviate: https://weaviate.io/blog/combining-langchain-and-weaviate Weaviate Blog - Generative Feedback Loops: https://weaviate.io/blog/generative-feedback-loops-with-llms Jerry Liu in Llama Index Blog - A New Document Summary Index for LLM-powered QA Systems: https://medium.com/llamaindex-blog/a-new-document-summary-index-for-llm-powered-qa-systems-9a32ece2f9ec Learning to Retrieve Passages without Supervision (Spider): https://arxiv.org/pdf/2112.07708.pdf Weaviate Blog - Analysis of Spider - https://weaviate.io/blog/research-insights-spider Chapters 0:00 Introduction 0:13 Quick Overview 7:30 How to use in Weaviate! 7:50 Background 12:08 Motivation 14:20 Prompts Used 18:14 More Details of training 21:12 Human Evaluation Study 22:40 My Takeaways from the Paper

Hey everyone, thank you so much for watching the 50th (!!!) Weaviate Podcast with Emil Sorensen and Finn Bauer from Kapa AI! Are you curious about taking either your, or your company's, specific information and putting into a Vector DB + LLM system? Emil and Finn are doing this at the highest level, taking the documentation of software companies like Weaviate and building these LLM-augmetnted assistant systems for them. This podcast takes a complete tour from Data Ingestion to Cleaning, Chunking, LLM latency, and emerging trends in LLMs such as cheap fine-tuning with LoRA or Long Context Windows such as GPT-4 32K, MPT-7B 65K, or Anthropic Claude's 100k. I learned so much from speaking with Emil and Finn! Please let us know any questions you have or ideas you would like to discuss! Check out Kapa here! https://www.kapa.ai/ Chapters 0:00 Welcome Emil and Finn! 0:42 Origin Story of Kapa 2:08 Data Ingestion 5:10 Data Cleaning 6:20 Slack / Discord / Forum Ingestion 9:05 Testing Models on Support QA 11:14 Selling Kapa to Weaviate and friends 12:37 Hallucinations in LLMs 14:06 Trends in Open-Source LLMs 15:20 Long Input LLMs (32K, 65K, 100K, …) 16:54 Retrieval-Augmentation for Long Input LLMs 18:08 Fine-Tuning LLMs 23:00 As much or as refined content as possible? 24:40 Adding Docs from Integrations 26:15 Generative Feedback Loops 29:00 What in AI excites you the most?

Professor Laura Dietz discusses Neurosymbolic Search, Entity Linking, Entity Re-Ranking, Knowledge Graphs, and Large Language Models. They explore the potentials of bias in using LLMs for relevance judgments and the complexities of merging neural technologies with symbolic systems in search technology. The conversation delves into enhancing search algorithms, filtered vector search, entity linking with context-specific models, and the nuances of relevance judgments in research papers.

Hey everyone, thank you so much for watching the 48th episode of the Weaviate Podcast!! This is a SUPER exciting one, welcoming Brian Raymond the CEO / Founder of Unstructured! Unstructured is a perfect complimenting technology for Weaviate, helping people get their Unstructured data into Weaviate! The podcast dives into the nuances of this task, but it generally revolves around Unstructured's abstraction of Partitioning, Cleaning, and Staging! Unstructured is making groundbreaking innovations on using Visual Document Layout models for Partitioning, for example saying that this part of the PDF is the header, body, image caption, and so on. Cleaning then describes removing pesky details like whitespaces or odd characters. Staging then describes the transformations of say formatting a text chunk with it's metadata into the JSON for a Weaviate object upload! I really hope you find this podcast interesting! We are publishing a blog post as well showing an example of how to use Unstructured to get PDF data into Weaviate, please please check that out and let us know if it works for your data and how we can improve it! This blog post can be found on weaviate.io and we will be managing discussions around it both in the Weaviate slack, as well as Unstructured! Thank you so much for listening! Check out Unstructured here! https://www.unstructured.io/ Chapters 0:00 Welcome Brian!! 0:27 What is Unstructured? 5:42 Why now? New Advancements in Unstructured 8:02 Thoughts on Data Connectors Hub 10:55 PDFs to Weaviate with Unstructured 13:53 State-of-the-Art in OCR and Document Parsing 16:10 How to get the data from Weaviate.io? 18:06 Foundation Models from Unstructured 20:45 Evaporate-Code+ 23:15 CSV, Parquet, JSON transformations in Staging 25:08 Cleaning Bricks 28:02 Visual Document Examples 30:45 Text Chunking with Metadata 33:25 Knowledge Graphs with Goldman Sachs example 39:10 LLM Hallucinations 42:10 Announcements from Brian!

Hey everyone, thank you so much for watching the Weaviate podcast! I am so excited about this episode! ChatArena is a software framework for multi-agent chat games. There are quite a few interesting applications of this, firstly we can use this kind of system to evaluate the intelligence of an LLM based on how intelligent it sounds in conversation with another LLM! Another interesting idea is to have the LLM impersonate people such as Lex Fridman or Sam Altman and simulate conversations between these people -- retrieving from their digital content to facilitate the impersonation. I thought there was so many interesting ideas in this podcast, please let us know what you think! Links: ChatArena on GitHub (please give it a star!) - https://github.com/chatarena/chatarena Twitter thread from Yuxiang describing the launch of ChatArena - https://twitter.com/YuxiangJWu/status/1643633046208249856 Chapters 0:00 Welcome Yuxiang! 0:38 What is ChatArena? 2:38 Impersonating People with LLMs 4:58 Weaviate and ChatArena 8:14 Generative Feedback Loops 11:10 Chat Games 16:30 Scientific Peer Review Discussions 20:05 Code Repos and Multi-Agent LLMs 23:05 Scaling Multi-Agent LLMs 25:16 Role Evolution in Startups 26:00 Evolution of Multi-Agent RL Research 29:22 AlphaGo and MCTS Text Generation 36:55 Hallucination in Role Maintenance 41:15 Evaluating LLMs with ChatArena 45:40 ChatGPT Marketplace and Tool Use 50:30 Upcoming work from Yuxiang and ChatArena!

Hey everyone! Thank you so much for watching the Weaviate Podcast! This is pretty novel episode featuring both Weaviate Co-Founders Bob van Luijt and Etienne Dilocker! This is also extremely novel because we are featuring a competitor vector database, HyperDB! John Dagdelen is the founder of HyperDB which is a hyper-fast local vector database for use with LLM Agents. Now accepting SAFEs at $135M cap. HyperDB: https://github.com/jdagdelen/hyperDB More seriously, John has produced an incredible body of research - https://scholar.google.com/citations?user=TiCS5FEAAAAJ&hl=en&oi=ao. John's work on Scientific Literature Mining for Materials Science literature has played an enormous role in my personal education of this technology and what it is capable. Please also follow John on twitter @jmdagdelen. Chapters 0:00 Introduction 0:26 HyperDB! 3:58 Initial Discovery of Vector Dos 15:00 Search Engine versus Databases 18:40 Scientific Literature Mining 21:42 Structured Information Extraction 27:47 Generative Feedback Loops

Hey everyone! Thank you so much for watching the Generative Feedback Loops Podcast! We have also created a blog post and GitHub repository for more information! Chapters 0:00 Bob the Podcast Host 1:20 Retrieval-Augmented Generation 4:10 Hallucination in LLMs 6:15 Solving Hallucination with RLHF 7:44 LLM Monster - Reasoning and Knowledge 10:12 Feedback Loops 11:00 Hands-on Code Demo 26:00 Demo Analysis from Bob and Connor 30:35 Star Wars Wes Anderson Generated Video 34:12 Multimodal Vector Databases 36:00 Speculative Design Theory Links: John Schulman - Reinforcement Learning from Human Feedback: Progress and Challenges: https://www.youtube.com/watch?v=hhiLw5Q_UFg Colin Nesh (HaystackUS 2023 slide deck) - Ground is NOT all you need, Stop hallucinations & defects in generative search: https://docs.google.com/presentation/d/1uycLEUeRuF8A85Uso_A3OU6EF-qq4aYswWVxkPWHBKI/edit#slide=id.p Generative Starwars video source - https://twitter.com/CuriousRefuge/status/1652412004626497536 Speculative Design Theory - https://readings.design/PDF/speculative-everything.pdf Aggregation Theory - https://stratechery.com/aggregation-theory/

Hey everyone! Thank you so much for watching the Weaviate 1.19 release podcast! We have all sorts of cool new features, in addition to the database and module features, I really want to encourage readers to see the `groupBy` search discussed at 14:32, quite an interesting idea for improving search performance! Chapters 0:00 Welcome Etienne! 0:38 gRPC API 9:50 Generative Cohere 14:32 groupBy search 19:33 Bitmap or BM25 index tuning 22:20 Additional Tokenization Options 24:05 Tunable Consistency

Thank you so much for watching the 43rd episode of the Weaviate Podcast with Roman Grebennikov and Vesvolod Goloviznin from Metarank, as well as Erika Cardenas from Weaviate! This podcast is a masterclass on Ranking models, additionally touching on the connection between Search and Recommendation. Learning-to-rank is an exciting idea where we use models that produce more fine-grained relevance scores than the offline indexing techniques of vector search and bm25, however with the tradeoff of the speed of these inferences. Romand and Vsevolod touched on another extremely interesting part of these ranking models which is the estimation of features such as Click-through-Rates and how they use streaming technology to do this. I learned so much from this podcast about the directions in ranking, I hope you enjoy it as well! As always, we are more than happy to answer any questions or discuss any ideas with you! In reflecting on this podcast, Erika and I wrote up our latest thoughts on Ranking Models in a Weaviate blogpost, check it out here if interested: https://weaviate.io/blog/ranking-models-for-better-search. Chapters 0:00 Welcome Everyone! 0:40 Recommendation with Weaviate 4:20 Metarank - Founding Story 8:20 Ranking MLOps 9:52 User Friendliness Perspective 15:10 Retrieval vs. Ranking 17:45 Ranking Optimization 25:20 Multi-Vector Object Representations 27:55 Click-through-Rate Feature Streaming 33:06 Weaviate Properties vs. Feature Stores 40:06 Cold-Start Recommendation Problem 46:04 Ranklens Demo - RecSys Datasets 52:02 Cross Encoders