Software Huddle

Join Philip Kiely as he unpacks the intricacies of inference optimization for AI workloads. He discusses the hype of Compound AI and how to choose the right model and inference engine. Learn about optimization techniques like quantization and speculative decoding that maximize GPU efficiency. Explore the role of multi-model AI systems and the challenges of model routing, network latency, and performance tooling. Discover practical insights on enhancing inference in large language models while balancing latency, throughput, and cost.

In this discussion, Kevin Dubois, a Senior Principal Developer Advocate at Red Hat and a Java Champion, shares his journey with Java and its evolution into the AI landscape. He dives into how Java integrates with AI, particularly using frameworks like Quarkus and LangChain4j, highlighting the mindset shifts needed for generative AI development. The conversation also emphasizes the parallels between programming and human languages and critiques the balance between open-source and proprietary models. Kevin's insights reveal Java's unique strengths in AI applications within enterprise settings.

Today we have Glauber Costa on the show, who's the CEO and founder at Turso. They provide a managed SQLite service with some really interesting capabilities that's changing some of the application patterns you can do. He shares a lot of really good technical stuff on Twitter. He worked in the kernel, he worked on high-performance databases at ScyllaDB, and now he's working on Turso. He also has a great and interesting podcast, the Save File, which is about developers and religion.Glauber had some great thoughts on the future of databases, including what the future of NoSQL is like and whether we'll see vector databases as a separate category or as a feature of general-purpose databases. We’ve seen arguments both ways, but he was the most effective at changing our mind.

Today, we have Mike Buckbee on the show. Mike is the co-founder of Wafris, and he wrote a really insightful article last week about moving from Redis to SQLite for an aspect of their architecture. The article was nuanced in describing why it worked for their specific needs, and it has some surprising takeaways, including that SQLite was 3x faster than a local Redis instance for their workload.Mike has built a few different WAF (Web Application Firewall) products, so we covered that area as well. He's seen a lot here, so we walked through all the nefarious traffic patterns and the speed in which these bots adapt to new vulnerabilities.Finally, Mike has a wide-ranging skillset that includes marketing. Developers are notoriously tricky to market to, so we talked about his experience in effective marketing to developers without being disingenuous.LinksBlog Post: https://wafris.org/blog/rearchitecting-for-sqliteFor A Good Strftime: www.foragoodstrftime.comIP Lookup: wafris.org/ip-lookupTimestamps01:11 Start03:41 Wafris 07:22 Redis and SQLite19:09 Flatfile21:50 Knowatoa28:22 Web Application Firewalls46:21 Jumpstart Pro48:11 Marketing to Developers

Tejas Kumar, part of the Developer Relations team at Datastax and author of "Fluent React," dives into AI engineering and its intersection with web development. He addresses common misconceptions about AI education and shares insights on integrating AI with frontend tech. Tejas discusses the evolution of web frameworks, highlighting the advantages of React and newer systems. He also reflects on the importance of content creation and storytelling in tech, emphasizing how developer relations can evolve in the landscape of modern conferences.

Today on the show, we have Peter Hanssens, the CEO and founder of Cloud Shuttle and creator of the DataEngBytes Conference.Peter has helped build an incredible data engineering community in Australia. He runs meetups, user groups, luncheons, and entire conferences. And he's also super knowledgeable. He's been working in the data space for a long time. We picked his brain about the history of data tooling, trends he's seeing in the industry and the relationship between data engineers and other types of engineering. Even if you aren't in the data world, we think you will enjoy the conversation.

Today we have a special guest. We have Jeremy Daly, who’s been in the cloud space for a while.Jeremy is the co-founder of Ampt, which is building an abstraction infrastructure layer on top of AWS, just to make it simpler to sift through all the different options and develop on AWS and do best practices there. So we wanted to get his opinions on a lot of different infrastructure stuff that he's seeing and how AI is changing development.We even talk about some front end stuff at the end and HTMX and whether it's real, whether it's a troll. So lots of good stuff in this episode. Timestamps01:56 Start04:28 Jeremy's Background07:26 Hard things about building ampt11:59 Infrastructure from Code17:07 App Runner20:10 Comparing ampt and PaaS27:22 Managing a lot of AWS accounts30:46 Better than AWS35:27 Thoughts on AWS deprecating services47:11 Using AI57:20 ChatGPT Adoption - Non Programmers01:06:19 AI affecting the job market01:18:37 HTMXSoftware Huddle ⤵︎X: https://twitter.com/SoftwareHuddleSubstack: https://softwarehuddle.substack.com/

Stephen Chin, VP of developer relations at Neo4j and Java expert, dives into the revolutionary GraphRAG architecture. He discusses how knowledge graphs can enhance generative AI performance, tackling issues like hallucinations and explainability. Stephen highlights the crucial role of these graphs in elevating customer support accuracy and efficiency. He also touches on the challenges enterprises face in deploying language models and the shift toward smarter technology investments, emphasizing mentorship and community collaboration in tech.

Today's episode is with database educator, PHP enthusiast, and all-around good guy Aaron Francis.Aaron is one of the best out there at delivering high-quality educational content. Somehow, he's managed to have three different video courses sell over $100k in wildly different fields -- a college corporate accounting class aide, video screencasting, and high-performance SQLite.In this episode, we talked about a lot of things, including:- Why (and when!) to use SQLite- What courses he's looking at next- How to stay sharp when doing educational content- His origin story as a programmer- Getting kids to be high agency- How PHP became classy.Linkshttps://www.epicweb.dev/why-you-should-probably-be-using-sqlitehttps://highperformancesqlite.com/TimestampsIntro 01:41Why SQLite 03:31When to use SQLite 09:14SQLite Creators 14:20Holy smokes 17:29jsonb Indexing? 22:07SQLite Course 23:54Vendor Specific Courses? 26:17Postgres Course Timing 30:26Nights and Weekends 30:46Getting into Databases 35:38Going back to Programming 39:22In 20 years 40:48Kids 42:08Making money is a skill? 47:52Balancing Video Creation and Programming 50:43Sustainable Business 54:23Doing SaaS 56:13Working for someone else 57:38Secret Sauce for Video Content 58:34PHP 01:05:56Taylor and Laravel 01:13:31Vue 01:18:39Warp-up 01:20:41

Dax Raad, a witty tech Twitter personality known for his sharp insights, joins the conversation to discuss SST, the infrastructure as code tool he's passionate about. He shares exciting details about their new engine, Ion, and its user benefits. The duo dives into cloud provider trustworthiness, the evolution of infrastructure management, and the marketing strategies that resonate with developers. Dax also compares code-based workflows to UI tools, emphasizing coding's efficiency in handling complexity.