Machine Learning Street Talk (MLST)

#50 Christian Szegedy - Formal Reasoning, Program Synthesis

Apr 4, 2021

Dr. Christian Szegedy, a deep learning pioneer at Google, dives into the potential of automating mathematical reasoning and program synthesis. He discusses autoformalisation, envisioning a super-human mathematician that comprehends natural language. Szegedy shares insights on the evolution of machine learning, particularly with transformers, and their impact on formal proofs and reasoning. The conversation also highlights challenges in research and the path toward human-level AGI, questioning traditional programming methods while exploring the nature of mathematical creativity.

01:33:22

Creator website

Episode guests

Christian Szegedy

Highlights

AI Chapters

Episode notes

Working with Batch Normalization and the Importance of the Technique

01:15

The Insightful Paper that Inspired the Inception Experiment

01:33

The Covariate Shift in Network Training

01:28

Intro

5min

Transformers and the Future of Mathematical Reasoning

15min

Advancements in Machine Learning and Formal Reasoning

17min

Exploring Multi-Agent Systems and Deep Learning

21min

Overcoming Challenges in Research and Experimentation

2min

Exploring Machine Learning's Limits and Evolution

28min

Exploring Inductive Biases and the Future of AI

6min

Dr. Christian Szegedy from Google Research is a deep learning heavyweight. He invented adversarial examples, one of the first object detection algorithms, the inceptionnet architecture, and co-invented batchnorm. He thinks that if you bet on computers and software in 1990 you would have been as right as if you bet on AI now. But he thinks that we have been programming computers the same way since the 1950s and there has been a huge stagnation ever since. Mathematics is the process of taking a fuzzy thought and formalising it. But could we automate that? Could we create a system which will act like a super human mathematician but you can talk to it in natural language? This is what Christian calls autoformalisation. Christian thinks that automating many of the things we do in mathematics is the first step towards software synthesis and building human-level AGI. Mathematics ability is the litmus test for general reasoning ability. Christian has a fascinating take on transformers too.

With Yannic Lightspeed Kilcher and Dr. Mathew Salvaris

Whimsical Canvas with Tim's Notes:

https://whimsical.com/mar-26th-christian-szegedy-CpgGhnEYDBrDMFoATU6XYC

YouTube version (with detailed table of contents) https://youtu.be/ehNGGYFO6ms

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.