Hungry Hippos - What Are the High Level Ideas Behind This H3 Layer?

Amitai Etzioni: I just wanted to ask a little bit on kind of like maybe some of the high level idea behind this H3 layer. And I'm also curious like how did, in particular what's novel about this work? Because you do mention like other people have been using state space models to do this kind of stuff. But obviously they haven't been able to perform as well on various language benchmarks. So I'm kind of curious like what aremaybe some of the new key ingredients that you guys found and added that maybe these work so well? Yeah. The, and part of this answer will kind of explain why it's a hungry, hungry hippo and

Transcript

Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.