AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
#130 - Llama 2, Elon Musk’s xAI, WormGPT, LongLLaMA, AI apocalypse, actors on strike
Last Week in AI
Long Lama: A Language Model Capable of Handling Long Contexts of 256 Tokens
3min Snip
00:00
Summary
Transcript
Episode notes
Long llama is a large language model capable of handling long contexts of 256 tokens. The more information there is, the greater chance that the model will get almost distracted by irrelevant information in the input. This means essentially a whole bunch of irrelevant information, a prompt that like causes the model to get confused. And they specifically train the model using the technique called contrastive learning to explicitly figure out what is relevant and what's not relevant so it can generate its output.
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode