In this episode of Crazy Wisdom, Stewart Alsop chats with Ian Mason, who works on architecture and delivery of AI and ML solutions, including LLMs and retrieval-augmented generation (RAG). They explore topics like the evolution of knowledge graphs, how AI models like BERT and newer foundational models function, and the challenges of integrating deterministic systems with language models. Ian explains his process of creating solutions for clients, particularly using RAG and LLMs to support automated tasks, and discusses the future potential of AI, contrasting the hype with practical use cases. You can find more about Ian on his LinkedIn profile.

Check out this GPT we trained on the conversation!

Timestamps

00:00 Introduction and Guest Welcome

00:32 Understanding Knowledge Graphs

02:03 Hybrid Systems and AI Models

03:39 Philosophical Insights on AI

05:01 RAG and Knowledge Graph Integration

07:11 Challenges in AI and Knowledge Graphs

11:40 Multimodal AI and Future Prospects

13:44 Artificial Intelligence vs. Artificial Linear Algebra

17:50 Silicon Valley and AI Hype

30:44 Defining AGI and Embodied Intelligence

32:29 Potential Risks and Mistakes of AI Agents

35:04 The Role of Human Oversight in AI

38:00 Understanding Vector Databases

43:28 Building Solutions with Modern Tools

46:52 The Future of Solution Development

47:43 Personal Journey into Coding

57:25 The Importance of Practical Learning

59:44 Conclusion and Contact Information

Key Insights

1. The evolution of AI models: Ian Mason discusses how foundational models like BERT have been overtaken by newer, more capable language models, which can perform tasks that once required multiple models. He highlights that while earlier models like BERT still have their uses, foundational models have simplified and expanded AI’s capabilities.
2. The role of knowledge graphs: Knowledge graphs provide structured, deterministic ways of handling data, which can complement language models. Ian explains that while LLMs are great for articulating responses based on large datasets, they lack the ability to handle logical and architectural connections between pieces of information, which knowledge graphs can provide.
3. RAG (Retrieval-Augmented Generation) systems: Ian delves into how RAG systems help refine AI output by feeding language models relevant data from a pre-searched database, reducing hallucinations. By narrowing down the possible answers and focusing the LLM on high-quality data, RAG ensures more accurate and contextually appropriate responses.
4. Limitations of language models: While LLMs can generate plausible-sounding responses, they lack deep architectural understanding and can easily hallucinate or provide inaccurate results without carefully curated input. Ian points out the importance of combining LLMs with structured data systems like knowledge graphs or vector databases to ground the output.
5. Vector databases and embeddings: Ian explains how vector databases, which use embeddings and cosine similarity, are crucial for narrowing down the most relevant data in a RAG system. This modern approach outperforms traditional keyword searches by considering semantic meaning rather than just text similarity.
6. AI’s impact on business solutions: The conversation highlights how AI, particularly through tools like RAG and LLMs, can streamline business processes. For instance, Ian uses AI to automate customer service email drafting, breaking down complex customer queries and retrieving the most relevant answers, significantly improving operational efficiency.
7. The future of AI in business: Ian believes AI’s real-world impact will come from its integration into larger systems rather than revolutionary standalone changes. While there is significant hype around AGI and other speculative technologies, the focus for the near future should be on practical applications like automating business workflows, where AI can create measurable value without over-promising its capabilities.