The Importance of Prediction in Language Model Evaluation

Support the show to get full episodes, full archive, and join the Discord community.

Check out my free video series about what's missing in AI and Neuroscience

My guest is Michael C. Frank, better known as Mike Frank, who runs the Language and Cognition lab at Stanford. Mike's main interests center on how children learn language - in particular he focuses a lot on early word learning, and what that tells us about our other cognitive functions, like concept formation and social cognition.

We discuss that, his love for developing open data sets that anyone can use,

The dance he dances between bottom-up data-driven approaches in this big data era, traditional experimental approaches, and top-down theory-driven approaches

How early language learning in children differs from LLM learning

Mike's rational speech act model of language use, which considers the intentions or pragmatics of speakers and listeners in dialogue.

• Language & Cognition Lab
• Twitter: @mcxfrank.
  • I mentioned Mike's tweet thread about saying LLMs "have" cognitive functions:
• Related papers:
  • Pragmatic language interpretation as probabilistic inference.
  • Toward a “Standard Model” of Early Language Learning.
  • The pervasive role of pragmatics in early language.
  • The Structure of Developmental Variation in Early Childhood.
  • Relational reasoning and generalization using non-symbolic neural networks.
  • Unsupervised neural network models of the ventral visual stream.