Latent Space: The AI Engineer Podcast cover image

ICLR 2024 — Best Papers & Talks (ImageGen, Vision, Transformers, State Space Models) ft. Durk Kingma, Christian Szegedy, Ilya Sutskever

Latent Space: The AI Engineer Podcast

CHAPTER

Advancements in Self-Supervised Learning and Model Fine-Tuning

This chapter explores the potential of video data for self-supervised learning through the introduction of a novel dataset called 'walking tours' and the self-supervised pre-training method DORA. It highlights the advancements in object tracking and transformer optimization, particularly focusing on techniques like LongLaura and shift sparse attention to enhance model performance. Additionally, practical applications of fine-tuned LAMA-3 models are discussed, showcasing their effectiveness in handling long contexts and providing suggestions for academic improvements.

00:00
Transcript
Play full episode

Remember Everything You Learn from Podcasts

Save insights instantly, chat with episodes, and build lasting knowledge - all powered by AI.
App store bannerPlay store banner