#164 - Meta AI, Phi-3, OpenELM, Bollywood Deepfakes
Apr 30, 2024
auto_awesome
Meta introduces smart assistants, Microsoft's Phi-3 model, Ray-Ban Meta smart glasses with AI, OpenAI's AI image generator closure, Baidu's Ernie chatbot success, Adobe Photoshop's new image generator. Intel's collaboration with Pentagon for advanced chips, Meta's increased AI spending, OpenAI CEO's investment in solar power, Google's consolidation of AI-focused teams, Microsoft and OpenAI's $100 billion project. Apple's OpenELM open source AI models, Snowflake's Arctic model launch. Deep dive into language model safety, AI chip advancements, Bollywood deepfake videos in India, LLM agents exploiting cyber vulnerabilities, China acquiring banned video chips, controversy over AI-generated voices in Drake's track.
Meta increases AI spending to 40 billion, emphasizing infrastructure development.
Apple releases open source AI models for on-device use under Apache 2.0 license.
VESA1 framework by Microsoft creates lifelike faces from audio inputs in real time.
Instruction hierarchy training by OpenAI enhances LLM robustness against attacks.
Intel's Hallop Point aims for brain-like computation in neuromorphic AI research.
Deep dives
Instruction Hierarchy Training for LLMs
OpenAI's paper introduces the concept of instruction hierarchy training to address issues like prompt injections and jailbreaks in models. By training models to prioritize certain instructions over others, they prevent misaligned inputs from causing harm.
Grox Breakthrough Chip for Llama-Free Inference
Grox's Chip achieves an exceptional speed of 800 tokens per second on Llama-Free model, significantly faster than typical speeds. The Tensor Streaming Processor eliminates the need for external memory and ensures fast, efficient processing.
Snowflake's New Model: Arctic
Snowflake introduces Arctic, an open-source mixture of experts LLM optimized for enterprise tasks like SQL and code generation. The model features a dense MLE hybrid architecture and performance that outshines other models in specific tasks.
Apple's Open-Source AI Models
Apple releases open source AI models ranging from 270 million to 3 billion parameters under the Apache 2.0 license. These models are designed for on-device functionalities and come with detailed documentation and performance stats.
Microsoft's Breakthrough on Real-Time Generation
Microsoft's VESA1 framework generates lifelike talking faces driven by audio inputs in real time, demonstrating impressive capabilities in generating high-quality video content from speech audio clips.
Meta Plans for Increased AI Spending
Meta announces plans to increase AI spending from 35 billion to 40 billion, focusing on AI infrastructure like data centers, chip design, and R&D. The decision to invest heavily in AI highlights Meta's commitment to advancing AI capabilities.
Research Advances in Safety Training
OpenAI's research paper introduces instruction hierarchy training for LLMs to prioritize system instructions over user instructions, preventing misaligned inputs. The model shows significant improvements in robustness against attacks and safeguards model behavior.
Grock Chip Achieves High Speed on Llama-Free
Grock's groundbreaking chip achieves an impressive speed of 800 tokens per second on Llama-Free model, showcasing its efficiency for AI workloads. The Tensor Streaming Processor enables fast inferencing with on-chip memory, eliminating the need for external memory transfers.
Advancements in AI Artifacts and Consumer Grade Hardware
AI models are becoming increasingly advanced, with demonstrations showing impressive results like Mona Lisa wrapping videos. These models can now be created using consumer-grade gaming hardware, like a single Nvidia RTX 4090, making them more accessible. The proliferation of this technology, including deepfakes, is expected to increase significantly in the near future, raising important questions about its impact on society.
Intel's Neuromorphic System for Sustainable AI
Intel's Hallop Point is the world's largest neuromorphic system, supporting high-performance operations for research purposes. This system aims to mimic brain-like computation methods for more efficient AI processing. While still a research prototype, it showcases ongoing investments in neuromorphic computing for potential energy efficiency improvements and advancements in brain-scale computing research.