What’s Next in LLM Reasoning? with Roland Memisevic - #646
Sep 11, 2023
auto_awesome
In this discussion, Roland Memisevic, Senior Director at Qualcomm AI Research, explores the future of language in AI systems. He highlights the shift from noun-centric to verb-centric datasets, enhancing AI's cognitive learning. Memisevic delves into the creation of Fitness Ally, an interactive fitness AI that integrates sensory feedback for a more human-like interaction. The conversation also covers advancements in visual grounding and reasoning in language models, noting their potential for more robust AI agents. A fascinating glimpse into the evolving landscape of AI!
Combining language and perception is crucial in building human-like AI systems.
Incorporating recurrence, memory, and advanced grounding in language models can advance AI reasoning and intelligence.
Deep dives
Using Language and Perception to Drive AI Forward
This podcast episode discusses the importance of combining language and perception in order to build more human-like intelligence systems. The speaker explains that language plays a crucial role in reasoning and is a key ingredient in human-like AI. They highlight the advancements in language models such as LLMs (large language models) and their ability to generate results that appear like reasoning. The episode explores various research works, including the concept of grounding language models with visual input, training agents to draw images through language instructions, and using language and vision to solve reasoning problems. The speaker also touches on the potential future developments involving recurrent connections and memory in AI systems.
Progress in AI Reasoning: Current Status and Prospects
This episode delves into the current status and future prospects of AI reasoning. The speaker acknowledges the difficulty in making accurate predictions, but highlights the ongoing progress. They mention the importance of incorporating recurrence and memory into AI models, as well as the need for more advanced grounding in language models. The speaker believes that by combining language and perception and building end-to-end systems with autoregressive models, significant advancements can be made in AI reasoning and the development of more human-like intelligence. They also mention the ongoing exploration of topics such as fast weights and the emergence of self-understanding in AI systems.
Exploring AI Reasoning through Situated Chats and Drawing on Canvas
This podcast episode discusses two specific research works in the field of AI reasoning. The first work focuses on the use of large language models combined with visual input to generate immersive and interactive conversations. The speaker highlights the evolution from previous approaches that utilized text-to-speech techniques and architectural rigidness to a more flexible and natural language model-driven system. The second work explores the generation of images by training an autoregressive model to draw lines on a canvas, enabling the model to learn effective drawing strategies through visual feedback. The episode emphasizes the potential of these works to enhance reasoning capabilities and create more agentic AI systems.
Today we’re joined by Roland Memisevic, a senior director at Qualcomm AI Research. In our conversation with Roland, we discuss the significance of language in humanlike AI systems and the advantages and limitations of autoregressive models like Transformers in building them. We cover the current and future role of recurrence in LLM reasoning and the significance of improving grounding in AI—including the potential of developing a sense of self in agents. Along the way, we discuss Fitness Ally, a fitness coach trained on a visually grounded large language model, which has served as a platform for Roland’s research into neural reasoning, as well as recent research that explores topics like visual grounding for large language models and state-augmented architectures for AI agents.
The complete show notes for this episode can be found at twimlai.com/go/646.
Remember Everything You Learn from Podcasts
Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
