Exploration of Sequence-to-Sequence AI Models and Diffusion in AI Technology Ecosystem

Paco Nathan Most of us are learning about AI on the fly and just got started in the past year or two. Paco Nathan has been working with AI since the 1980s and has been doing digital business nearly as long. His background in both the technical and commercial sides of artificial intelligence gives him a unique perspective on the field that can help newcomers like me and you get oriented to this new landscape. We talked about: his extensive history in the AI field, including work with some of the earliest chatbots how graphs can serve as a way to ground and contextualize unstructured content how content that is structured properly can help help users and drive action the tech stack underlying the current generation of AI tools two technologies at the base level of the stack: sequence-to-sequence and diffusion the benefits of SSM, small specialized models, over LLMs his take on the impact of LLM chat agents on content and editorial practice four take-homes from his recent immersion in AI conferences and gatherings: the superiority of small, specialized learning models (SSMs) over LLMs the issue of losing domain knowledge as experts age and retire the importance of using your own data sets the need for detailed task analysis as you begin building any AI model the contrasts and interplay between AI developments at large, well-funded entities like Alphabet, Meta, and Microsoft and the smaller, more diverse ecosystem around open-source AI projects Paco's bio Paco Nathan, Managing Partner at Derwen, Inc., and author of Latent Space, along with other books, plus popular videos and tutorials about machine learning, natural language, and related topics. Known as a "player/coach", with +40 years tech industry experience, ranging from Bell Labs to early-stage start-ups. Werner Herzog is his spirit animal. Board member for Argilla.io; Advisor for KUNGFU.AI. Lead committer on PyTextRank, kglab. Formerly: Director, Community Evangelism for Apache Spark at Databricks. Long, long ago, when the world was much younger, Paco led a media collective / indie bookstore / performance art space / large online community called FringeWare. Beginning in 1992, this was one of the first online bookstores and likely the first commercial use of a chat bot on the Internet. Connect with Paco online Derwen.ai Argilla.io Video Here’s the video version of our conversation: https://youtu.be/bjU_q36cggw Podcast intro transcript This is the Content and AI podcast, episode number 2. You'd think from news stories and social media that AI is mostly about large language models like ChatGPT and big companies like Microsoft and Google. In fact, there's a large, well-established community of open-source AI projects and a variety of technologies in addition to LLM-based chat agents. With more than 40 years of experience in artificial intelligence and in the tech business world, Paco Nathan is uniquely qualified to orient us in the current AI landscape. Interview transcript Larry: Hey everyone, welcome to episode number two of the Content and AI Podcast. I'm really happy today to welcome to the show Paco Nathan. Paco, we could talk literally for 20 hours about this stuff we're going to talk about today. But what Paco and I are going to talk about today just kinda get you grounded in making sense of the AI ecosystem. Paco's been doing this stuff forever. He's studied AI back in the, what, the 80s or something like that. Anyhow, welcome, Paco. Oh, and one last quick thing. Paco is the managing director of Derwen.ai, his consulting company. So welcome, Paco. Tell the folks a little bit more about your work at Derwen.ai and some of your discoveries around AI lately. Paco: Oh, fantastic. Thank you very much, Larry, I really appreciate it. Yeah, Derwen, we're really focused on open-source integration to support machine learning in general. But we focus a lot on natural language and graph...