Azure & DevOps Podcast

Per Kops: Architecture of Ollama and Local LLMs - Episode 342

Mar 24, 2025

Per Kops, a distinguished software developer at Delegate with nearly two decades in .NET and Azure, shares insights on the evolution of software consulting. He discusses founding ATC-Net to streamline code sharing and foster trust through open-source initiatives. The conversation delves into Ollama's architecture for local LLMs, his fondness for Semantic Kernel's abstraction layers, and the role of vector stores in AI. Per also breaks down complex topics like 'text chunkers' and finding the right tools tailored to business needs.

35:44

Creator website

Episode guests

Per Kops

AI Summary

AI Chapters

Episode notes

Podcast summary created with Snipd AI

Quick takeaways

Per Kops emphasizes the importance of collaboration and transparency in software development through initiatives like ATC Net to promote reusable code.

He discusses the integration of local large language models with Semantic Kernel, highlighting its architecture and the significance of using vector stores for effective AI application development.

Deep dives

Per Kops' Journey in Software Development

Per Kops shares his extensive background in software development, which began with a fascination for computers at a young age after acquiring a Commodore 64. His career trajectory included working as a Java developer, transitioning to Microsoft’s business intelligence products, and eventually moving into the .NET ecosystem where he found his passion. A significant turning point came when he entered the consulting domain with Azure technologies, which he has embraced ever since. His narrative underscores the importance of mentorship and how pivotal experiences can redirect one's career path toward new opportunities.

Intro

3min

From Commodore to Consulting

8min

Building Bridges with Ollama and Semantic Kernel

7min

Exploring Olama and the Semantic Kernel

8min

Understanding Vector Databases in RAG Systems

8min

Navigating the AI Landscape: Tools for Business Needs

2min

Per is an experienced software architect and developer with more than 19 years of experience in designing, developing, and testing software. He is currently serving as a Distinguished software developer at Delegate, a Microsoft Partner in Denmark.

Topics of Discussion:

[3:02] More about Per’s background in nearly two decades in .NET and Azure consulting.

[6:11] The concept of a boutique consultant.

[7:00] Founding ATC-Net to share common .NET libraries and avoid duplicative coding.

[10:02] Building trust with customers by open-sourcing “generic” components.

[11:50] Creating a connector to run Olama (local LLMs) with Semantic Kernel.

[20:08] Why Per “fell in love” with Semantic Kernel’s abstraction layers and extension possibilities.

[21:44] The role of vector stores (like Azure Cognitive Search) in retrieval-augmented generation.

[24:33] What do we mean by “vector” in the AI space?

[26:39] What is a “text chunker”?

[31:43] Azure AI search.

[34:07] Finding the right tool for the right task.