The Gradient: Perspectives on AI cover image

Ed Grefenstette: Language, Semantics, Cohere

The Gradient: Perspectives on AI

00:00

The Introduction of Transformers as Programmable Computers

In this talk you gave at the Alan Turing Institute you kind of expounded on how RNNs relate to the models of computation we're familiar with. You made the argument that in the hierarchy they lie closer to finite state machines I don't necessarily have access to unbounded memory than to Turing machines. How do you think about the introduction of transformers in terms of that computational hierarchy question? That's an interesting question to ask specifically because I had one of my student Minchi Jiang sent me a paper just the other day on this topic is called Looped Transformers as Programmable Computers by Angelique Genu and colleagues.

Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner