Exploring Time Representation in Transformers for Enhanced Learning

Support the show to get full episodes, full archive, and join the Discord community.

Jovo, as you'll learn, is theoretically oriented, and enjoys the formalism of mathematics to approach questions that begin with a sense of wonder. So after I learn more about his overall approach, the first topic we discuss is the world's currently largest map of an entire brain... the connectome of an insect, the fruit fly. We talk about his role in this collaborative effort, what the heck a connectome is, why it's useful and what to do with it, and so on.

The second main topic we discuss is his theoretical work on what his team has called prospective learning. Prospective learning differs in a fundamental way from the vast majority of AI these days, which they call retrospective learning. So we discuss what prospective learning is, and how it may improve AI moving forward.

At some point there's a little audio/video sync issues crop up, so we switched to another recording method and fixed it... so just hang tight if you're viewing the podcast... it'll get better soon.

0:00 - Intro 05:25 - Jovo's approach 13:10 - Connectome of a fruit fly 26:39 - What to do with a connectome 37:04 - How important is a connectome? 51:48 - Prospective learning 1:15:20 - Efficiency 1:17:38 - AI doomerism