Neural Search Talks — Zeta Alpha

In this episode of Neural Search Talks, we have invited Louis Rosenberg, CEO of Unanimous.AI, to discuss the future of AI in decision-making, contrasting the development of artificial superintelligence (ASI) with collective human intelligence systems, such as swarm intelligence. In particular, Louis argues that the advancement of AI should focus on amplifying human intelligence rather than replacing it, drawing from the biological inspiration found in nature, where species evolve by connecting individuals into systems that function as a singular intelligent entity, exemplified by schools of fish and swarms of bees. Tune into our conversation to learn more about how AI can assist humans in disseminating knowledge and making better decisions! Check out the Zeta Alpha Neural Discovery platform: https://zeta-alpha.com Subscribe to the Zeta Alpha calendar to not miss out on any of our events: https://lu.ma/zeta-alpha Timestamps: 0:00 Intro by Jakub Zavrel 2:08 Using AI to amplify human intelligence 18:19 How AI and humans learn from each other 26:41 Scaling human collaboration with AI 40:13 Satisfying information needs with AI 45:57 How Unanimous AI connects experts to make better decisions 51:37 Predictions for AI progress in one year 53:21 Outro

In this episode of Neural Search Talks, we welcome Hyeongu Yun from LG AI Research to discuss the newest addition to the EXAONE Universe: EXAONE 3.0. The model demonstrates strong capabilities in both English and Korean, excelling not only in real-world instruction-following scenarios but also achieving impressive results in math and coding benchmarks. Hyeongu shares the team's approach to the development of this model, revealing key training factors that contributed to its success while also highlighting the challenges they faced along the way. We close this episode off with a look at EXAONE's future, as well as Hyeongu's perspective on the evolving role of AI systems. Check out the Zeta Alpha Neural Discovery platform. Subscribe to the Zeta Alpha calendar to not miss out on any of our events! Sources: - https://lgresearch.ai/blog/view?seq=460 - https://huggingface.co/LGAI-EXAONE/EXAONE-3.0-7.8B-Instruct - https://arxiv.org/abs/2408.03541 Timestamps: 0:00 Intro by Jakub Zavrel 1:37 The journey of the EXAONE project 4:34 The main challenges in the development of EXAONE 3.0 6:37 The secret to achieving great bilingual performance in English & Korean 7:51 How EXAONE 3.0 stacks against other open-source models 9:20 The trade-off between instruction-following and reasoning skills 12:32 How will retrieval and generative models evolve in the future 16:36 Open sourcing and user feedback on EXAONE 19:20 The role of synthetic data in model training 20:57 The role of LLMs as evaluators 23:16 Outro

In the 30th episode of Neural Search Talks, we have our very own Arthur Câmara, Senior Research Engineer at Zeta Alpha, presenting a 20-minute guide on how we fine-tune Large Language Models for effective text retrieval. Arthur discusses the common issues with embedding models in a general-purpose RAG pipeline, how to tackle the lack of retrieval-oriented data for fine-tuning with InPars, and how we adapted E5-Mistral to rank in the top 10 on the BEIR benchmark. ## Sources InPars https://github.com/zetaalphavector/InPars https://dl.acm.org/doi/10.1145/3477495.3531863 https://arxiv.org/abs/2301.01820 https://arxiv.org/abs/2307.04601 Zeta-Alpha-E5-Mistral https://zeta-alpha.com/post/fine-tuning-an-llm-for-state-of-the-art-retrieval-zeta-alpha-s-top-10-submission-to-the-the-mteb-be https://huggingface.co/zeta-alpha-ai/Zeta-Alpha-E5-Mistral NanoBEIR https://huggingface.co/collections/zeta-alpha-ai/nanobeir-66e1a0af21dfd93e620cd9f6

Manuel Faysse, a PhD student from CentraleSupélec & Illuin Technology and first author of a pivotal paper on document retrieval, discusses his innovative model, ColPali. He shares the "Aha!" moment that inspired its creation and outlines the challenges faced in research. ColPali simplifies traditional retrieval systems using vision-language models, enhancing efficiency and relevance in document search. Manuel also compares ColPali with classic multimodal models, showcasing its superiority and potential for future applications.

Ronak Pradeep, a PhD student from the University of Waterloo focusing on information retrieval and large language models, shares his insights on revolutionizing the field with LLMs. He discusses their role in enhancing ranking systems and the intricacies of integrating them into production pipelines. Ronak explores the challenges of model distillation and the need for refined training data. He also reflects on key takeaways from the latest SIGIR conference, highlighting the future potential of LLMs in shaping search technologies.

Omar Khattab, a prominent author of influential IR and LLM frameworks like ColBERT and DSPy, shares his insights on designing reliable AI systems. He discusses the critical importance of modularity and systematic engineering in integrating AI models into production. Omar envisions a future of Artificial Programmable Intelligence, focusing on effective AI integration that empowers developers. He emphasizes the role of human intuition in design and how open-sourcing DSPy promotes collaborative growth, ultimately enhancing AI's reliability and adaptability.

Florin Cuconasu, the first author of a groundbreaking paper presented at SIGIR 2024, dives into the fascinating world of Retrieval-Augmented Generation (RAG). He discusses how large language models and retrievers work together to enhance AI applications. Florin shares insights on the strategic arrangement of information and the intriguing role of 'noise' in improving retrieval effectiveness. He also outlines his ambitious research agenda and the future impact of LLMs on information retrieval. A must-listen for anyone in the AI field!

Nandan Thakur, the first author of the BEIR benchmark paper, dives deep into the realm of Information Retrieval. He discusses the current state of model evaluations, shedding light on the uphill battle of models trying to surpass the BM25 baseline. Nandan highlights the shortcomings of BEIR as of 2024 and addresses specific challenges like the Touché 2020 subset. He also shares insights on the future of benchmarking, including the newly announced TREC RAG track, and emphasizes the critical role of multilingual capabilities in advancing IR systems.

Aamir Shakir, an expert from Mixed Bread AI, shares his journey in revolutionizing AI-driven search technologies. He reveals the whimsical story behind the company’s name, emphasizing a unique baking analogy. Aamir discusses overcoming challenges like GPU shortages and the significance of diverse data in creating robust embedding models. He also highlights the potential of multilingual and multimodal capabilities for future information retrieval. Get ready for innovative insights into the future of AI search!

Ash shares his journey from software development to pioneering in the AI infrastructure space with Unum. He discusses Unum's focus on unleashing the full potential of modern computers for AI, search, and database applications through efficient data processing and infrastructure. Highlighting Unum's technical achievements, including SIMD instructions and just-in-time compilation, Ash also touches on the future of computing and his vision for Unum to contribute to advances in personalized medicine and extending human productivity. Timestamps: 0:00 Introduction 0:44 How did Unum start and what is it about? 6:12 Differentiating from the competition in vector search 17:45 Supporting modern features like large dimensions & binary embeddings 27:49 Upcoming model releases from Unum 30:00 The future of hardware for AI 34:56 The impact of AI in society 37:35 Outro