Neal Gompa, a special guest, joins the hosts to discuss Asahi Linux on a MacBook, beating Apple to a major GPU milestone. They also explore self-hosted open-source ChatGPT alternatives and running Fedora Asahi Remix on Apple Silicon Macs, including limitations and the potential for using the M1 Mac mini as a home server.
The Asahi Linux team proposes adding telemetry to the installer to gather more accurate installation data, which is accepted for the project's ongoing development.
Fedora and Asahi collaborate to create the Fedora Asahi remix for a seamless Linux experience on Apple Silicon, with efforts to eventually merge all Asahi-related work into mainline Fedora.
Linux compatibility on Apple M hardware is improving, with the M1 Mac mini emerging as a potential option for running Linux as a home server.
Deep dives
Telemetry proposal for Asahi Linux
Hector Martin proposes adding telemetry to the Asahi installer to gather more accurate installation data, including device information, firmware versions, and selected settings. The proposal is met with acceptance as the project needs the data to support ongoing development. Meanwhile, progress is being made in running Linux on Apple M hardware, as Fedora and Asahi collaborate to create a Fedora Asahi remix for a seamless Linux experience on Apple Silicon. Challenges remain, such as video decoding support and external audio options, but overall, the M1 Mac mini is seen as a potential option for running Linux as a home server.
Remixing Fedora with Asahi: Collaboration and Challenges
Fedora and Asahi collaborate to create the Fedora Asahi remix, a new flagship distribution offering a polished Linux experience on Apple Silicon. Efforts are made to integrate Asahi support into the mainline Fedora project, but certain components like the kernel, Yubut, and Mesa remain separate and packaged in a copper for the remix. The goal is to eventually merge all Asahi-specific work into mainline Fedora. Asahi's focus on integrating user space fixes benefits any Fedora desktop on an ARM platform, while work on graphics driver development for M1 hardware is also progressing.
Considerations for Using Apple M Hardware with Linux
Linux compatibility on Apple M hardware is improving, but certain limitations persist. The Fedora Asahi remix provides a satisfactory Linux experience, but users need to be aware of missing features like video decoding and external audio support. Battery life and performance are commendable, with the M1 Mac mini emerging as a potential option for running Linux as a home server due to its low power consumption and quiet operation. Other laptop options like Lenovo or Dell's XPS series are worth considering, as they offer good Linux support and performance, while providing the desired features such as battery life and fast I/O.
Lama GPT: A Self-Hosted Chatbot Powered by Facebook's Open AI Engine
Lama GPT is a self-hosted offline chatbot powered by Facebook's open-source Lama to engine. It allows users to run the chatbot on their own devices, ensuring that no data leaves the device. The project optimizes the chatbot to run on consumer-grade hardware, including Raspberry Pi. It offers Docker Compose support, making it easy to set up and use. The project follows the developments of open-source tooling and provides an open AI-compatible API. Lama GPT is an exciting step towards open-source alternatives to AI models and enables users to experiment and understand the inner workings of language models.
Improved Accessibility to AI Models
Lama GPT opens doors to community-hosted large language models. It allows users to experience the capabilities of language models in a self-hosted, local, and private environment. This advance promotes better understanding of AI tools by enabling users to tweak and experiment with the models. It encourages education and awareness of AI technology, which is becoming increasingly integrated into various aspects of our lives. Community-hosted models like Lama GPT can serve as the foundation for future projects, such as searchable podcast archives or personalized virtual assistants.
We daily drive Asahi Linux on a MacBook, chat about how the team beat Apple to a major GPU milestone, and an easy way to self-host open-source ChatGPT alternatives.
Our new flagship distro: Fedora Asahi Remix — We’re still working out the kinks and making things even better, so we are not quite ready to call this a release yet. We aim to officially release the Fedora Asahi Remix by the end of August 2023. Look forward to many new features, machine support, and more!
Hector Martin: “Okay, I’m going to be honest…” — I apologize to all Asahi Linux users. You deserve better. When I chose Arch Linux ARM as a base I didn't realize it would have so many basic QA issues.
The first conformant M1 GPU driver — Our reverse-engineered, free and open source graphics drivers are the world’s only conformant OpenGL ES 3.1 implementation for M1- and M2-family graphics hardware. That means our driver passed tens of thousands of tests to demonstrate correctness and is now recognized by the industry.
llama.cpp — Port of Facebook’s LLaMA model in C/C++
Llama2.c — Inference Llama 2 in one file of pure C
Koboldcpp — A simple one-file way to run various GGML models with KoboldAI’s UI
lollms-webui — Lord of Large Language Models Web User Interface
LM Studio — Discover, download, and run local LLMs
text-generation-webui — A Gradio web UI for Large Language Models. Supports transformers, GPTQ, llama.cpp (ggml/gguf), Llama models.
A comprehensive guide to running Llama 2 locally — Code Llama is a state-of-the-art LLM capable of generating code, and natural language about code, from both code and natural language prompts.