#146 - ChatGPT’s 1 year anniversary, DeepMind GNoME, Extraction of Training Data from LLMs, AnyDream
Dec 12, 2023
auto_awesome
Celebrate the first anniversary of ChatGPT and its transformative impact on AI. Discover Perplexity AI’s groundbreaking real-time language models and Microsoft’s innovative Dolly 3 integration in Windows. Explore how AI is reshaping TurboTax's tax assistance and personalization tools like MasterCard's Shopping Muse. Delve into DeepMind’s advances in discovering materials and the ethical concerns surrounding AI in healthcare. Finally, investigate the implications of Meta's new rules on AI-generated political content amid rising challenges in the AI landscape.
DeepMind's Gnome project uses deep learning to predict new crystal structures, significantly expanding the catalog of known stable crystals.
Open-source language models like Llama2 and DeepSeek are showing promising results and narrowing the gap with closed-source models.
A study by DeepMind quantified the carbon footprint of AI tasks, revealing higher environmental impact in image generation compared to text generation.
Deep dives
DeepMind's Gnome project expands catalog of known stable crystals
DeepMind has developed the Gnome project, a custom model that uses deep learning to predict potential new crystal structures. By generating different atomic structures and using a graph neural network to model their properties, Gnome has significantly expanded the catalog of known stable crystals. The model has shown promising results, outperforming previous methods and increasing the number of computationally stable materials by an order of magnitude.
Open-source LLMs closing the gap with closed-source models
Open-source language models are making progress in catching up to closed-source models like ChatGPT. While there are still challenges, such as changing performance, unknown training data, and higher costs, open-source models like Llama2 and DeepSeek are showing promising results in various benchmarks, including language comprehension, coding, and math. The trend of open-source models improving and scaling suggests a narrowing divergence between closed-source and open-source models.
Google's DeepMind quantifies carbon footprint of AI image generation
A recent study by DeepMind has quantified the carbon footprint of various AI tasks, including image generation, object detection, text classification, and more. The study found that tasks like image generation and image captioning have a higher carbon footprint, requiring significant computational resources. On the other hand, text generation has a lower carbon footprint compared to image-based tasks. This research provides important insights into the environmental impact of AI and can guide efforts to develop more energy-efficient models.
Research highlights potential of deep learning for materials discovery
Researchers at DeepMind have developed a deep learning model called Gnome that aims to expand the catalog of known stable crystals. By using a combination of data sets and a graph neural network, Gnome can predict potential new crystal structures. The model has shown promising results, outperforming previous methods and significantly increasing the number of computationally stable materials. This research has significant implications for materials discovery and showcases the potential of deep learning in advancing scientific research.
Adopting Ethical Principles for Generative AI in Healthcare
A collaboration of different backgrounds has put forward a set of ethical principles for generative AI in healthcare, along with a framework for adopting and expanding these principles practically.
Low-Dimensional Gaussian Mixtures and Transformers
Researchers have developed a mathematical definition for low-dimensional Gaussian mixtures, allowing them to recreate transformers based on this ideal. This understanding can enhance algorithmic efficiency and deepen our comprehension of multi-headed self-attention.