AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
ThursdAI - The top AI news from the past week
๐ ThursdAI - Apr 18th - ๐ Happy LLama 3 day + Bigxtral instruct, WizardLM gives and takes away + Weights & Biases conference update
Apr 19, 2024
02:13:43
LLaMa 3 day celebration with new 8B and 70B models announced, Meta's collaboration with Google, model releases with improved benchmarks, AI integration in Premiere Pro, advancements in ML ops partnerships and fine-tuning libraries, discussions on Weights and Biases involvement, parameter usage in AI models, Reka's multi-modal capabilities, and weekly recap with gratitude to contributors and listeners.
Read more
AI Summary
Highlights
AI Chapters
Episode notes
Podcast summary created with Snipd AI
Quick takeaways
- Llama 3 model release at Meta conference with 8B and 70B parameters praised for performance and potential in open source community.
- Deep dive into Llama 3's benchmarks, context length, tokenizers, and implications for fine-tuning and diverse applications.
- Exploration of Llama 3's impact on open source AI community, deployment plans, user testing, and model comparison discussions.
- Discussion on real-world AI model deployment challenges, strategies, industry perspectives, user feedback importance, and evolving technology landscape.
- OpenAI's Batch API for non-real-time tasks management, resource optimization, and cost-effective processing benefits with a 50% discount option.
Deep dives
Overview of the Annual Meta Conference and Llama 3 Release
The podcast episode discusses the exciting events at the annual Meta conference, highlighting the release of the Llama 3 model. The host recounts their experience of attending the conference, hearing about and engaging with the new Llama 3 model, which comes in 8 billion and 70 billion parameter versions. The Llama 3 model is praised for its performance on benchmarks and its potential in the open source community. The episode emphasizes the significance of the release and encourages listeners to explore the details and implications of Llama 3.
Detailed Analysis of Llama 3 Model Performance
Listeners are provided with a comprehensive overview of the key features and performance metrics of the Llama 3 model. The discussion focuses on the model's benchmarks, such as MMOU and human eval scores, comparing it to other prominent models like Mistral. Details about the model's context length, tokenizers, and inference speed are highlighted, shedding light on its strengths and areas for potential improvement. The episode delves into the implications of the model's dense architecture, its ease of fine-tuning, and its compatibility with various applications.
Impact of Llama 3 on the Open Source Community
The conversation extends to the broader implications of Llama 3's release on the open source AI community. Guests share their perspectives on the model's capabilities, potential applications, and integration into various platforms. The episode explores the community's reactions to Llama 3, including plans for deployment, user testing, and model comparison. Insights are shared on the decision-making process for transitioning from existing models like Mixtroll to Llama 3, highlighting the considerations involved in model selection and deployment.
Reflections on Model Deployment and Adoption
The podcast covers discussions around real-world deployment of AI models, focusing on the challenges and strategies involved in transitioning to new models like Llama 3. The conversation touches on industry perspectives on model selection, testing, and production integration. Guests share their experiences with deploying Llama 3 in various use cases, emphasizing the factors influencing model adoption and the importance of user feedback and testing in the deployment process. The episode provides insights into the evaluation and adoption of AI models in practical applications, showcasing the evolving landscape of applied AI technology.
Batch API Offering Efficient Handling of Batch Tasks
OpenAI introduced a Batch API, enabling users to submit non-real-time tasks such as dataset creation or evaluation for processing in a batch manner, separate from real-time user interactions. This approach optimizes server resource management and provides a 50% discount for users opting for batch task processing, benefiting from reduced strain on server capacity.
Impact of Batch API on Server Load and Task Prioritization
The Batch API approach enables OpenAI to prioritize batch tasks more efficiently during lower usage periods, optimizing resource allocation and decreasing strain on servers. This strategic resource management ensures optimal task processing and cost-effective allocation of computational resources.
Wizard LM's Model Release Controversy and Swift Removal
Wizard LM introduced fine-tuned models, including an impressive 9.12 score model based on Bixtrel, but faced challenges due to toxicity checks by Microsoft prompting the sudden removal of the released models. The controversial removal highlighted issues with model release and licensing, impacting accessibility and adoption for users.
Considerations for Assistance API Integration and Use Cases
OpenAI's Assistance API Version 2 offers advanced conversational capabilities, though adoption rates among developers vary. Features like storing conversation history on servers and optimizing user interactions may enhance user experiences, but integration complexities and limited user feedback may influence overall utility and application in chatbot development.
OpenAI Assistance API Updates
OpenAI released a new version of its Assistance API that allows users to upload up to 10,000 files per assistant. This update includes a new file search tool, improved chunking processes, and a new vector store. Despite these enhancements, there seems to be low adoption rates, prompting OpenAI to further enhance their Assistance V2 API with a pre-built rag system.
Innovations in Multi-Modal Models
Reka's Core model, a multi-modal AI, supports input across images, videos, and audio, surpassing models from larger labs. With strong reasoning abilities and language support in 32 languages, Reka Core stands out for its performance in human evaluation tasks. Additionally, Adobe intricately integrated Firefly AI into Premiere, enabling content editors to generate and enhance videos with AI tools seamlessly.
4.
Exploring the Performance of the Llama Three Model in Different Language Capabilities
8min
6.
Discussions on Weights and Biases Involvement in Llama Models and Big Stroll Instruct Release
4min
Happy LLama 3 day folks! After a lot of rumors, speculations, and apparently pressure from the big Zuck himself, we finally can call April 18th, 2024, LLaMa 3 day!
I am writing this, from a lobby of the Mariott hotel in SF, where our annual conference is happening called Fully Connected, and I recorded today's episode from my hotel room. I really wanna shout out how awesome it was to meet folks who are listeners of the ThursdAI pod and newsletter subscribers, participate in the events, and give high fives.
During our conference, we had the pleasure to have Joe Spisak, the Product Director of LLaMa at Meta, to actually announce LLaMa3 on stage! It was so exhilarating, I was sitting in the front row, and then had a good chat with Joe outside of the show ๐
The first part of the show was of course, LLaMa 3 focused, we had such a great time chatting about the amazing new 8B and 70B models we got, and salivating after the announced but not yet released 400B model of LLaMa 3 ๐ฎ
We also covered a BUNCH of other news from this week, that was already packed with tons of releases, AI news and I was happy to share my experiences running a workshop a day before our conference, with focus on LLM evaluations. (If there's an interest, I can share my notebooks and maybe even record a video walkthrough, let me know in the comments)
Ok let's dive in ๐
Happy LLama 3 day ๐ฅ
The technical details
Meta has finally given us what we're all waiting for, an incredibly expensive (2 clusters of 24K H100s over 15 Trillion tokens) open weights models, the smaller 8B one and the larger 70B one.
We got both instruction fine tune and base models, which are great for finetuners, and worth mentioning that this is a dense model (not a mixture of experts, all the parameters are accessible for the model during inference)
It is REALLY good at benchmarks, with the 7B model beating the previous (LLaMa 2 70B) on pretty much all benchmarks, and the new 70B is inching on the bigger releases from the past month or two, like Claude Haiku and even Sonnet!
The only downsides are the 8K context window + non multimodality, but both are coming according to Joe Spisak who announced LLama3 on stage at our show Fully Connected ๐ฅ
I was sitting in the front row and was very excited to ask him questions later!
By the way, Joe did go into details they haven't yet talked about pulblicly (see? I told you to come to our conference! and some of you did!) and I've been live-tweeting his whole talk + the chat outside with the "extra" spicy questions and Joes winks haha, you can read that thread here
The additional info
Meta has also partnered with both Google and Bing (take that OpenAI) and inserted LLama 3 into the search boxes of Facebook, Instagram, Messenger and Whatsapp plus deployed it to a new product called meta.ai (you can try it there now) and is now serving LLama 3 to more than 4 Billion people across all of those apps, talk about compute cost!
Llama 3 also has a new Tokenizer (that Joe encouraged us to "not sleep on") and a bunch of new security tools like Purple LLama and LLama Guard. PyTorch team recently released finetuning library called TorchTune is now supporting LLama3 finetuning natively out of the box as well (and integrates Wandb as it's first party experiment tracking tool)
If you'd like more details, directly from Joe, I was live tweeting his whole talk, and am working at getting the slides from our team. We'll likely have a recording as well, will post it as soon as we have it.
Here's a TL;DR (with my notes for the first time) of everything else we talked about, but given today is LLaMa day, and I still have to do fully connected demos, I will "open source" my notes and refer you to the podcast episode to hear more detail about everything else that happened today ๐ซก
TL;DR of all topics covered:
* Meta releases LLama 3 -8B, 70B and later 400B (Announcement, Models, Try it, Run Locally)
* Open Source LLMs
* Meta LLama 3 8B, 70B and later 400B (X, Blog)
* Trained 15T tokens!
* 70B and 8B modes released + Instruction finetuning
* 8K context length , not multi modal
* 70B gets 82% on MMLU and 81.7% on HumanEval
* 128K vocab tokenizer
* Dense model not MoE
* Both instruction tuned on human annotated datasets
* Open Access
* The model already uses RoPe
* Bigxtral instruct 0.1 (Blog, Try it)
* Instruct model of the best Apache 2 model around
* Release a comparison chart that everyone started "fixing"
* ๐ค Mixtral 8x22B is Mistral AI's latest open AI model, with unmatched performance and efficiency
* ๐ฃ It is fluent in 5 languages: English, French, Italian, German, Spanish
* ๐งฎ Has strong math and coding capabilities
* ๐ง Uses only 39B parameters out of 141B total, very cost efficient
* ๐ Can recall info from large documents thanks to 64K token context window
* ๐ Released under permissive open source license for anyone to use
* ๐ Outperforms other open models on reasoning, knowledge and language benchmarks
* ๐ Has strong multilingual abilities, outperforming others in 4 languages
* ๐งช Excellent basis for customization through fine-tuning
* New Tokenizer from Mistral (Docs)
* Focusing on Tool Use with tokens ๐ฅ
* WizardLM-2 8x22B, 70B and 7B (X, HF)
* Released it and then pulled it back from HF and Github due to microsoft toxicity not passing
* Big CO LLMs + APIs
* OpenAI gives us Batch API + Assistants API v2
* Batch is 50% cost and win win win
* Assistants API V2 - new RAG
* new file search tool
* up to 10,000 files per assistant
* new vector store
* Reka gives us Reka Core (X, Try)
* Multimodal that understands video as well
* 20 people team
* Video understanding is very close to Gemini
* 128K context
* Core has strong reasoning abilities including for language, math and complex analysis.
* 32 languages support
* HuggingFace ios chat bot now
* This weeks Buzz
* Me + team led a workshop a day before the conference (Workshop Thread)
* Fully Connected in SF was an incredible success, over 1000 AI attendies + Meta AI announcement on stage ๐ฅ
* PyTorch new TorchTune finetuning library with first class WandB support (X)
* Vision & Video
* Microsoft VASA-1 animated avatars (X, Blog)
* Amazing level of animation from 1 picture + Sound
* Harry Potter portraits are here
* They likely won't release this during Election year
* Looks very good ,close to EMO but no code
* ๐บ Videos show faces speaking naturally with head movements and lip sync
* ๐ฌ Researchers are exploring applications in education, accessibility and more
* HuggingFace updates IDEFICS2 8B VLM (X, HF)
* Apache 2 license
* Competitive with 30B models
* 12 point increase in VQAv2, 30 point increase in TextVQA (compared to Idefics 1)
* > 10x fewer parameters than Idefics 1
* Supports image resolution up to 980 x 980+
* Better OCR capabilities (thanks to more than 6TB of OCR pre-training data)
* Adobe shows Firefly video + SORA support (X)
* Voice & Audio
* Rewind AI is now Limitless (X)
* New service & Brand name
* Transcription to you
* Hardware device that looks sleek
* 100hours
* Privacy support in cloud
* AI Art & Diffusion & 3D
* Stability - Stable Diffusion 3 is here
* Available via API only
* Partnered with Fireworks HQ for the release
* Needs stability AI membership to use / access $$
* Big step up in composition and notorious issues like hands, "AI faces" etc. (from
* Seems to prefer simpler prompts.
* Way more copyright-friendly. It's hard to get any kind of brands/logos.
* Text is amazing.
* Others
* New AIrChat with amazing transcription is out, come join us in our AI corner there
* Humane AI pin was almost killed by MKBHD review
* Rabbit reviews incoming
That's all for this week, next week we have an amazing guest, see you then! ๐ซก
This is a public episode. If youโd like to discuss this with other subscribers or get access to bonus episodes, visit sub.thursdai.news/subscribe
Get the Snipd
podcast app
Unlock the knowledge in podcasts with the podcast player of the future.
AI-powered
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode
Save any
moment
Hear something you like? Tap your headphones to save it with AI-generated key takeaways
Share
& Export
Send highlights to Twitter, WhatsApp or export them to Notion, Readwise & more
AI-powered
podcast player
Listen to all your favourite podcasts with AI-powered features
Discover
highlights
Listen to the best highlights from the podcasts you love and dive into the full episode