AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
2min snip
Coercing LLMs to Do and Reveal (Almost) Anything with Jonas Geiping - #678
The TWIML AI Podcast (formerly This Week in Machine Learning & Artificial Intelligence)
Optimizing with Hybrid Strategies
The speaker discusses the limitations of straightforward gradient searches for complex models like llama two models. They introduce the concept of introducing randomness in gradient-based optimizers which enhances the optimization process. The G2G algorithm involves collecting a list of 256 largest magnitude candidates and selecting randomly from them, creating a hybrid approach between random search and gradient-based optimization. This method helps overcome barriers in the optimization landscape by introducing structured randomness, which differs from traditional gradient-based approaches.
00:00
Transcript
Episode notes
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode