We test two popular methods to run local language models on your Linux box. Then, we push the limits to see which language models will toe the line and which won't.

Sponsored By:

• Tailscale: Tailscale is a Zero config VPN. It installs on any device in minutes, manages firewall rules for you, and works from anywhere. Get 20 devices for free for a personal account.
• Linode Cloud Hosting: A special offer for all Linux Unplugged Podcast listeners and new Linode customers, visit linode.com/unplugged, and receive $100 towards your new account.
• Kolide: Kolide is a device trust solution for companies with Okta, and they ensure that if a device isn’t trusted and secure, it can’t log into your cloud apps.

Support LINUX Unplugged

Links:

• 🎉 Alby — Boost into the show, first grab Alby, top it off, and then head over to the Podcast Index.
• ⚡️ LINUX Unplugged on the Podcastindex.org — You can boost from the web. Once Alby is topped off, visit our page on the Podcast Index.
• Brent in Berlin Meetup — Brent is back in Berlin and we're having a LOW KEY meetup at the NixOS night at cbase.
• One Time Fiat Fun for SCaLE 2024
• Southern California Linux Expo 19
• Purchase a SCALE Pass 63
• First NixCon North America!
• NixCon North America | SCALE 21x
• NixCon North America Call for Proposals
• Texas Linux Fest 2024
• LFNW 2024: Ready or not
• LFNW 2024: Call for Sessions
• Jupiter Broadcasting Live Calendar
• 2023 Unplugged Tuxies
• Ollama — Get up and running with Llama 2 and other large language models locally.
• OllamaHub — Explore and Download Custom Ollama Modelfiles.
• ollama library
• Ollama Manual Install
• Ollama Docker Image
• ollama-webui
• LlamaGPT
• llama-gpt — A self-hosted, offline, ChatGPT-like chatbot. Powered by Llama 2. 100% private, with no data leaving your device. New: Code Llama support!
• Continue.dev — An open-source autopilot in your IDE
• Refact.ai — Refact is an open-source AI coding assistant with blazing-fast code completion, powerful code improvement tools, and chat.
• 32-Bit Challenge Chat room
• Speech Note — Speech Note let you take, read and translate notes in multiple languages. It uses Speech to Text, Text to Speech and Machine Translation to do so. Text and voice processing take place entirely offline, locally on your computer, without using a network connection. Your privacy is always respected. No data is sent to the Internet.
• Speech Note on GitHub
• LFNW: Call for Sponsors
• Framework 13 (Intel) NixOS Module
• GeneBean's hyprland nix config
• Razer Core X - Thunderbolt™ 3 eGPU
• CompleteNoobs: Ubuntu KVM to Windows 10 - Thinkpad T470 - Razer Core X - GTX 970 -
• LINUX Unplugged 308: The One About GPU Passthrough — Our crew walks you through their PCI Passthrough setups that let them run Windows, macOS, and distro-hop all from one Linux machine.
• Pick: Marker — Convert PDF to markdown quickly with high accuracy