Google rejoins the open model party and gets some backlash for a frequent problem for generative AI.This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/gemma-google-ships-it00:00 Google ships it: Gemma open LLMs and Gemini backlash03:12 Getting to know Gemma07:11 Alignment details08:55 Aside: What is REINFORCE? Some history of RL11:08 Implementation details and RLHF12:18 Terms of use: RAIL Licenses history repeated14:05 Is Google back on top? Gemini's woesFigure 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_008.webpFigure 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_014.pngFigure 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_035.pngFigure 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_051.pngFigure 5: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/gemma/img_055.png Get full access to Interconnects at www.interconnects.ai/subscribe

10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and moreThis is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/sora-gemini-follow-up00:00 10 Sora and Gemini 1.5 follow-ups: code-base in context, deepfakes, pixel-peeping, inference costs, and more00:46 1. Deepfake detection of Sora01:59 2. Playing with long-context, problem settings, and prompting03:39 3. Gemini paper snooping: contamination and citation games05:42 4. Training data and token estimates of YouTube07:42 5. Unlocking model-based RL and downstream research08:52 6. Midjourney style matching, V-JEPA, replicating Sora in the open10:09 7. Architectures and academic links10:57 8. Pixel peeping from the arts11:58 9. Inference costs13:24 10. Pressure on Llama and Mistral14:03 11. Sound effects, physics, and the complete pictureFigure 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-2/img_003.pngFigure 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-2/img_007.mp4Figure 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-2/img_009.mp4Figure 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-2/img_011.mp4Figure 5: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-2/img_037.mp4Figure 6: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-2/img_044.pngFigure 7: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-2/img_047.pngFigure 8: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-2/img_049.mp4 Get full access to Interconnects at www.interconnects.ai/subscribe

Emergency blog! Three things you need to know from the ML world that arrived yesterday.This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/sora-gemini-and-mistral-next0:00 OpenAI's Sora for video, Gemini 1.5, and a secret Mistral model0:53 Sora: OpenAI's text-to-video model4:59 Gemini 1.5: Google's effectively infinite context length8:01 Mistral-next: Another funny release methodFigure 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-gemini-mistral/img_015.pngFigure 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-gemini-mistral/img_023.pngFigure 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-gemini-mistral/img_026.pngFigure 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/sora-gemini-mistral/img_036.png Get full access to Interconnects at www.interconnects.ai/subscribe

In an era dominated by direct preference optimization and LLMasajudge, why do we still need a model to output only a scalar reward?This is AI generated audio with Python and 11Labs. Music generated by Meta's MusicGen.Source code: https://github.com/natolambert/interconnects-toolsOriginal post: In an era dominated by direct preference optimization and LLM-as-a-judge, why do we still need a model to output only a scalar reward?Podcast figures:Figure 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/reward-models/img_004.pngFigure 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/reward-models/img_009.png0:00 Why reward models are still key to understanding alignment Get full access to Interconnects at www.interconnects.ai/subscribe

This podcast discusses the challenges faced by ScaleAI, a startup providing data services for reinforcement learning from human feedback (RLHF). It explores ScaleAI's revenue growth, partnership with major labs, and defense arm. The podcast also explores the concept of scaling alignment as a service through AI feedback alignment and potential business opportunities in RLHF.

A small model at the beginning of big changes.This is AI generated audio with Python and 11LabsSource code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/olmo0:00 Open Language Models (OLMos) and the LLM landscape6:24 Thought experiments7:51 The LLM landscape heading into 2024Figure 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/olmo/img_010.png Get full access to Interconnects at www.interconnects.ai/subscribe

Note: some of the audio in the second half is a little wonky, but the general voice was upgraded so hopefully it's a little less "poppy" until then!I'm trying to fix little pronunciation problems on a weekly basis. Thanks to my early fans! It'll keep improving. E.g. some of the months were wonky.When what seems like pure LLM black magic is actually supported by the literature.This is AI generated audio with Python and 11LabsSource code: https://github.com/natolambert/interconnects-toolsOriginal post: https://www.interconnects.ai/p/model-merging00:00 Model merging lessons in The Waifu Research Department02:21 How and why does model merging work?07:13 Aside: merging vs. ensembles vs. mixture of experts08:21 Why are people doing this?11:22 Tools & Links11:51 Brief (visual) literature review12:07 Full model merging and recent methods15:55 Weight averaging during pretraining17:18 LoRA merging17:53 More backgroundFigure 1: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_005.pngFigure 2: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_016.pngFigure 3: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_042.pngFigure 4: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_051.pngFigure 5: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_055.pngFigure 6: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_058.pngFigure 7: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_060.pngFigure 8: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_062.pngFigure 9: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_065.pngFigure 10: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_075.pngFigure 11: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_077.pngFigure 12: https://huggingface.co/datasets/natolambert/interconnects-figures/resolve/main/model-merging/img_084.png Get full access to Interconnects at www.interconnects.ai/subscribe

The podcast discusses the benefits of local LLMs, strategies to optimize latency, and the integration of LLMs into consumer devices. It explores the role of local models in machine learning for personalization and optimization for inference. The influence of ML labs and their larger ambitions on the future is also discussed, highlighting Alama's popularity and Meta's build-out plans and open-source strategy.

A podcast discusses multimodal blogging, AI tools for content creation, and expanding audience reach. The speaker shares their workflow in building a suite of tools for bloggers and explores the use of AI tools like Passport, audio conditioning, and voice cloning. They also discuss future advancements in text to video models and automation in research talks and video creation.

This podcast discusses recent developments in the multimodal space, including the Unified IO 2 model, collecting preference data for images, LLaVA-RLHF experiments, and challenges in multimodal RLHF. They explore the architecture and challenges of multimodal models, the potential of GPT for V in multimodal RLHF, and the use of RLHF technique in multimodal models. They also discuss the importance of clearer terminology and the adoption of synthetic data in this context.