Running AI on Your Terms: Open Source LLMs & European Innovation
Follow the Rabbit
Intro
This chapter delves into the emerging product DeepSeek, which facilitates local installations of large language models. The hosts discuss the importance of open source AI solutions for maintaining data privacy, alongside the need for explainability in AI systems, leading to insights from a tech expert guest.
From DeepSeek to local installations, exploring the democratization of AI and Europe's opportunity
In this episode of Follow the Rabbit, Igor and Johannes are joined by Axel Quack, who has led the development of AI-driven products and services for large-scale organizations. Drawing from his extensive experience in financial services innovation and technology transformation, Axel brings unique insights into how European businesses, particularly SMEs, can leverage open source AI while maintaining privacy and control. His hands-on experience with local LLM installations and commitment to democratizing AI technology provides a practical foundation for exploring these emerging possibilities.
The conversation examines how recent developments like DeepSeek are challenging the narrative that AI innovation requires massive computing resources and corporate infrastructure. Through Axel's perspective as both a practitioner and strategist, the discussion reveals how smaller organizations can meaningfully engage with AI technology while maintaining their independence and data sovereignty.
The discussion weaves together several key themes:
- How open source AI models are becoming increasingly accessible to individuals and smaller organizations
- The tension between convenience and control in AI deployment
- Europe's unique opportunity to differentiate itself through decentralized, privacy-focused approaches
- The importance of hands-on experimentation in developing AI literacy
- How product development might evolve as foundation models become commoditized
Through practical examples and strategic analysis, the episode reveals how the democratization of AI technology could reshape not just technical capabilities, but organizational structures and regulatory approaches. The conversation challenges both techno-optimistic and techno-skeptical positions, suggesting a more nuanced path forward that emphasizes experimentation, understanding, and thoughtful implementation.
Chapters:
00:00 - Introduction & Context
02:19 - Understanding Open Source LLMs
07:37 - Local Installation & Implementation
19:42 - European Perspectives & Opportunities
34:15 - Product Development & Future Directions
48:33 - Learning & Experimentation
54:22 - Closing Thoughts
Links & Resources:
- LM Studio (local LLM installation tool): https://lmstudio.ai/
- OpenWeb UI: https://docs.openwebui.com
- Ollama: https://ollama.com
- Ollama (supported models): https://ollama.com/library (e.g. Mistral, LlaMa, Phi, Gemma, DeepSeek, Qwen etc.)
- Exo (software to run your own AI cluster on a Mac Mini): https://github.com/exo-explore/exo
- Stable Diffusion: Automatic1111 (UI) - https://github.com/AUTOMATIC1111/stable-diffusion-webui
- Smart2Brain: AI plugin for Obsidian based on Ollama - https://github.com/your-papa/obsidian-Smart2Brain
- Zima Cube: https://zimacube.com/
This episode provides both practical insights for those interested in experimenting with local AI installations and broader strategic perspectives on how open source AI might reshape technological development. It suggests that while we should maintain healthy skepticism about AI hype, the democratization of these tools creates new opportunities for innovation and agency.
Axel Quack online:
---------------
Follow the Rabbit onSpotify
Follow the Rabbit onApple Podcasts
Follow the Rabbit onYoutube
Follow the Rabbit is a Known Unknowns production
Known Unknowns onYouTube
Known Unknowns onInstagram
Follow the Rabbit is hosted by Igor Schwarzmann & Johannes Kleske
Find out more aboutIgor Schwarzmann
Find out moire aboutJohannes Kleske
The AI-powered Podcast Player
