Chang She, CEO of Lens and co-creator of the Pandas library, shares insights on building LanceDB for AI data management. He discusses how LanceDB tackles data bottlenecks and speeds up machine learning experiments with unstructured data. The conversation dives into the decision to use Rust for enhanced performance, achieving up to 1,000 times faster results than Parquet. Chang also explores multimodal AI's challenges, future applications of LanceDB in recommendation systems, and the vision for more composable data infrastructures.

Mór Kapronczay, Head of ML at Superlinked, unpacks the nuances of embeddings beyond just text. He emphasizes that traditional text embeddings fall short, especially with complex data. Mór introduces multi-modal embeddings that integrate various data types, improving search relevance and user experiences. He also discusses challenges in embedding numerical data, suggesting innovative methods like logarithmic transformations. The conversation delves into balancing speed and accuracy in vector searches, highlighting the dynamic nature of real-time data prioritization.

Today we have Jessica Talisman with us, who is working as an Information Architect at Adobe. She is (in my opinion) the expert on taxonomies and ontologies.That’s what you will learn today in this episode of How AI Is Built. Taxonomies, ontologies, knowledge graphs.Everyone is talking about them no-one knows how to build them.But before we look into that, what are they good for in search?Imagine a large corpus of academic papers. When a user searches for "machine learning in healthcare", the system can:Recognize "machine learning" as a subcategory of "artificial intelligence"Identify "healthcare" as a broad field with subfields like "diagnostics" and "patient care"We can use these to expand the query or narrow it down.We can return results that include papers on "neural networks for medical imaging" or "predictive analytics in patient outcomes", even if these exact phrases weren't in the search queryWe can also filter down and remove papers not tagged with AI that might just mention it in a side not.So we are building the plumbing, the necessary infrastructure for tagging, categorization, query expansion and relexation, filtering.So how can we build them?1️⃣ Start with Industry Standards • Leverage established taxonomies (e.g., Google, GS1, IAB) • Audit them for relevance to your project • Use as a foundation, not a final solution2️⃣ Customize and Fill Gaps • Adapt industry taxonomies to your specific domain • Create a "coverage model" for your unique needs • Mine internal docs to identify domain-specific concepts3️⃣ Follow Ontology Best Practices • Use clear, unique primary labels for each concept • Include definitions to avoid ambiguity • Provide context for each taxonomy nodeJessica Talisman:LinkedInNicolay Gerold:⁠LinkedIn⁠⁠X (Twitter)00:00 Introduction to Taxonomies and Knowledge Graphs 02:03 Building the Foundation: Metadata to Knowledge Graphs 04:35 Industry Taxonomies and Coverage Models 06:32 Clustering and Labeling Techniques 11:00 Evaluating and Maintaining Taxonomies 31:41 Exploring Taxonomy Granularity 32:18 Differentiating Taxonomies for Experts and Users 33:35 Mapping and Equivalency in Taxonomies 34:02 Best Practices and Examples of Taxonomies 40:50 Building Multilingual Taxonomies 44:33 Creative Applications of Taxonomies 48:54 Overrated and Underappreciated Technologies 53:00 The Importance of Human Involvement in AI 53:57 Connecting with the Speaker 55:05 Final Thoughts and Takeaways

Jo Bergum, Chief Scientist at Vespa, dives into the game-changing technology of ColPali, which revolutionizes document processing by merging late interaction scoring and visual language models. He discusses how ColPali effectively handles messy data, allowing for seamless searches across complex formats like PDFs and HTML. By eliminating the need for extensive text extraction, ColPali enhances both efficiency and user experience. Its applications span multiple domains, promising significant advancements in information retrieval technology.

Aamir Shakir, founder of mixedbread.ai, is an expert in crafting advanced embedding and reranking models for search applications. He discusses the transformative power of rerankers in retrieval systems, emphasizing their role in enhancing search relevance and performance without complete overhauls. Aamir highlights the benefits of late interaction models like ColBERT for better interpretability and shares creative applications of rerankers beyond traditional use. He also navigates future challenges in multimodal data management and the exciting possibilities of compound models for unified search.

Join Nils Reimers, a prominent researcher in dense embeddings and the driving force behind foundational search models at Cohere. He dives into the intriguing limitations of text embeddings, such as their struggles with long documents and out-of-domain data. Reimers shares insights on the necessity of fine-tuning to adapt models effectively. He also discusses innovative approaches like re-ranking to enhance search relevance, and the bright future of embeddings as new research avenues are explored. Don't miss this deep dive into the cutting-edge of AI!

Nirant Kasliwal, an author known for his expertise in metadata extraction and evaluation strategies, shares invaluable insights on scaling Retrieval-Augmented Generation (RAG) systems. He dives into common pitfalls such as the challenges posed by naive RAG and the sensitivity of LLMs to input. Strategies for query profiling, user personalization, and effective metadata extraction are discussed. Nirant emphasizes the importance of understanding user context to deliver precise information, ultimately aiming to enhance the efficiency of RAG implementations.

Doug Turnbull, a search engineer at Reddit and author of "Relevant Search," dives into the transformation of search from keyword basics to advanced methods like semantic search and LLMs. He highlights the ongoing challenges of defining relevance based on user intent and context. Doug also discusses the importance of integrating various search techniques for better results, emphasizing the role of operational concerns in shaping search technology. With insights on the resurgence of underappreciated methods like LambdaMART, he shares how understanding user perspectives can significantly enhance search performance.

Charlie Hull, a search expert and the founder of Flax, dives into the world of data-driven search optimization. He discusses the challenges of measuring relevance in search, emphasizing its subjective nature. Common pitfalls in search assessments are highlighted, including overvaluing speed and user complaints. Hull shares effective methods for evaluating search systems, such as human evaluation and user interaction analysis. He also explores the balancing act between business goals and user needs, and the crucial role of data quality in delivering optimal search results.

Join Daniel Tunkelang, a seasoned search consultant and leader in AI-powered search, as he explores the nuances of query understanding. He emphasizes that the user's query is paramount and advocates for a proactive approach to enhancing search systems. Discover the significance of query specificity, the advantages of classifying queries, and how simpler techniques can rival complex models. Tunkelang also shares insights on optimizing query processing and the challenges of categorizing data in an ever-evolving landscape.