Machine Learning Street Talk (MLST)

Ryan Greenblatt, a researcher at Redwood Research known for his groundbreaking work on the ARC Challenge, discusses his innovative use of GPT-4 to achieve impressive accuracy. He explores the strengths and weaknesses of current AI models and the profound differences in learning and reasoning between humans and machines. The conversation touches on the risks of advancing AI autonomy, the effects of over-parameterization in deep learning, and the potential future advancements, including the promise of multimodal capabilities in forthcoming models.

Aidan Gomez, CEO of Cohere, dives into tackling AI hallucinations and reasoning improvements in this lively conversation. He discusses why Cohere avoids using GPT-4 for training and shares insights on the unique challenges enterprises face with AI, from onboarding to legislation. Aidan highlights their commitment to crafting robust, industry-specific solutions while addressing the societal implications of AI advancements and regulatory needs. Get a glimpse into Cohere’s ethos and their strategic vision for the future of AI technology.

In this engaging discussion, Jack Cole, a clinical psychologist and ARC Challenge winner, along with AI researcher Mohammed Osman and expert Michael Hodel, delve into the nuances of the ARC Challenge, which assesses AI reasoning. They present their winning approach of fine-tuning language models, emphasizing active inference and innovative data representation. The trio debates the philosophical implications of their methods on intelligence measurement while highlighting the addictive nature of ARC tasks and raising questions about the future of AI and generalization.

Nick Frosst, Co-founder of Cohere, previously at Google Brain alongside AI pioneer Geoff Hinton, shares insights on AI's future in business. He discusses Cohere's Command R models, which enhance language capabilities using retrieval augmented generation (RAG). Nick critiques the chase for AGI, emphasizing specialization in LLMs over generalization. He also touches on ethical data use in AI, the evolving role of software engineers in machine learning, and even gives a nod to his indie band, Good Kid, showcasing the intersection of creativity and technology.

Aman Bhargava, a PhD student at Caltech, and Cameron Witkowski, a graduate student at the University of Toronto, dive into their groundbreaking research on controlling language models using control theory. They discuss how language models operate as discrete systems and the surprising impact of prompt engineering on outputs. By examining the "reachable set" of outputs, they reveal that even minor tweaks in prompts can lead to significant changes in generated text. Their insights could pave the way for more reliable and capable AI systems.

Maria Santacaterina, a humanities expert and author of "Adaptive Resilience," offers a critical look at AI's limitations in contrast to human intelligence. She discusses how AI lacks essential traits like empathy and creativity, emphasizing its reliance on past data. Maria warns against overreliance on AI in sectors like healthcare and justice, advocating for ethical frameworks to guide its use. The concept of adaptive resilience is introduced as a way for individuals and organizations to thrive amidst challenges, stressing the importance of preserving human values in the tech age.

Join Dr. Thomas Parr, a postdoctoral scholar at the Wellcome Centre for Human Neuroimaging, as he delves into the fascinating world of active inference. He explores how living systems resist entropy and navigate their environments through a mathematical framework. Parr discusses the evolution of neural networks and contrasts traditional AI with active inference's potential for deeper world modeling. He also navigates complex topics like agency, consciousness, and the collaborative journey of writing his book on these intricate concepts.

In this discussion, Connor Leahy, CEO of Conjecture, dives into the complexities of AI alignment and the risks of advanced technologies, alongside technical researcher Daniel Clothiaux and AI alignment advocate Beff Jezos. They explore societal and cultural implications of AI, the importance of coherence in technology, and the potential for AI to develop agency. The conversation also addresses the widening gap between societal classes and the critical need for resilient institutions to navigate technological challenges, making a case for equitable opportunities and collaboration.

Chris Bishop, Technical Fellow at Microsoft Research AI4Science and a leading figure in machine learning, discusses his newly co-authored textbook 'Deep Learning: Foundations and Concepts.' He delves into the evolution of deep learning, emphasizing the importance of a probabilistic approach. The conversation covers the nature of large models like GPT-4, AI's role in scientific discovery, and strategies for improving model training with geometric priors. Bishop also reflects on the paradox of deep learning effectiveness and the ethical considerations in AI development.

Philip Ball, a freelance science writer and author of "How Life Works", delves into the fascinating realm of biology and agency. He explores how our understanding of living organisms has evolved, questioning traditional views linking agency to human consciousness alone. The discussion touches on the interplay between culture and cognitive development, risks of AI misinformation, and the chaotic nature of life as it navigates thermodynamic principles. Ball advocates for holistic approaches in biology, challenging mechanical metaphors and emphasizing the collaborative essence of intelligence.