AI Tinkerers - "One-Shot"

AI agents are evolving fast, but what if they could seamlessly integrate with real-world tools, automate workflows, and take action—without breaking security or reliability?In this One-Shot episode, we sit down with Karan from Composio, a cutting-edge AI framework designed to connect LLMs with APIs, streamline tool orchestration, and unlock new levels of agent autonomy.With over 250+ API integrations, Composio is solving one of the biggest hurdles in AI automation: getting things done—securely, efficiently, and at scale.Three mind-blowing takeaways from this episode:⚡ LLMs Alone Won’t Cut It:"So essentially, if you think about it, agents are kind of LLMs plus tools, right?" – Karan, Composio💡 From Browsers to APIs: The Smarter Path to AI Execution:"Like you don’t want everything to be controlled by a browser, right? Like these LLMs are stochastic models. They can go wrong." – Karan, Composio🚀 The Future of Autonomous AI Assistants:From AI-powered SDRs and personal assistants to automated data management and software development, Composio is helping AI agents go from passive responders to proactive problem solvers.Karan breaks down how Composio handles authentication, authorization, and multi-tenant support, making it a must-have tool for anyone building AI-powered automation.But here’s the kicker…The future of AI isn’t just about answering questions—it’s about doing real work. Whether you’re building AI-powered personal assistants, automated sales workflows, or next-gen productivity tools, Composio is paving the way for AI agents that truly get things done.🔗 Watch this One-Shot episode and discover how Composio is reshaping AI automation.Welcome to the future of AI-driven action. Welcome to AI Tinkerers.

What if AI could talk, react, and even mirror human emotions in real-time? In this One-Shot episode, we explore how Tavus is pushing the boundaries of AI-driven avatars—building an experience so fast and fluid, it feels like a real conversation.Brian from Tavus has spent months refining the latency, speech processing, and personality layers of AI avatars, bringing us closer to seamless, interactive digital humans. Meet Brad—an avatar that doesn’t just listen, but truly engages.Here’s what makes this episode unmissable:1. AI Conversations Without the Lag – Tavus has cracked the sub-second response time, making interactions feel instant and natural.2. Building Personalized AI Personas – With Tavus’ API, developers can customize avatars, define personalities, and integrate them into real-world applications.3. The Future of Digital Assistants – From virtual customer service to education and beyond, AI avatars are redefining how we interact with machines.Brian takes us under the hood, explaining how AI speech gating, low-latency LLMs, and streaming architecture come together to make these avatars feel real.But this is just the beginning…AI avatars are evolving fast. Whether it’s sales, education, personal assistants, or even healthcare, real-time AI-driven conversations are about to become mainstream.Watch now and see how Tavus is building the future of human-AI interaction.Welcome to the new era of AI avatars. Welcome to AI Tinkerers.Highlights:00:00 - Introduction and Welcome00:43 - Meet Brian and His AI-Powered Avatar01:17 - Introducing Brad, the Conversational Video Interface02:03 - How the Avatar Processes Audio and Video03:06 - Latency Challenges in AI Conversations04:07 - Optimizing Speech Processing for Real-Time Response05:47 - The Role of ASR and Low-Latency Models06:45 - API Setup for Custom AI Conversations07:46 - Integrating Real-Time AI in Everyday Use Cases08:45 - Unexpected Delight in Fast AI Responses09:47 - The Future of Virtual Avatars and Trust10:27 - Configuring LLM and API for Custom AI Interaction11:43 - Balancing Function Calls and Response Speed12:53 - Designing AI Conversations for User Experience13:41 - Emerging AI Business Use Cases15:30 - AI in Sales and Marketing16:48 - The Role of AI in Education and Healthcare17:54 - AI as a Virtual Assistant for Professionals18:11 - Brad on Mimicking Human Conversations19:15 - Bridging the Uncanny Valley in AI Avatars20:14 - Pipeline Behind AI Conversations and Visuals21:34 - How AI Processes Audio and Generates Expressions22:08 - Critical Factors for Natural AI Conversations23:10 - The Future of AI in Professional and Social Spaces25:27 - Ensuring AI Safety and Preventing Misuse26:22 - Brian’s Drone Project for Illegal Dumping Detection27:56 - How AI-Powered Drones Help San Francisco28:47 - Automating City Clean-Ups with AI29:37 - Hacking Systems for Social Good30:10 - Future of AI and Open Source Innovations

In this explosive “One-Shot” episode, we’ll dismantle the complexities of building a flying Linux computer—GPU, 5G, real-time inference, and all—and see how voice commands can orchestrate everything from speed adjustments to target tracking.Maxwell Wang, SpaceX software engineer, created Dispatcher, an end-to-end aerial AI system that’s turning sci-fi ambitions into practical, weekend-ready tech. Imagine telling a drone, “Back up 50 meters,” and watching it respond instantly—no clunky menus, no joysticks. That’s the magic of Dispatcher: a system that blends local compute for real-time tasks with cloud-powered LLMs for semantic understanding. The result? A frictionless, voice-driven aerial experience that scouts parking lots, follows targets, and returns home—all in one fluid operation.Three jaw-dropping takeaways that’ll make you hit “Play” on the episode:1. Real-Time AI on the Edge: Maxwell’s drone runs powerful vision models locally for live object tracking—say goodbye to lag from iffy connections.2. Voice Commands, Simplified: Want the camera to tilt up or the drone to land? Just say it. Dispatcher’s function-calling infrastructure translates speech into drone actions.3. Hybrid Autonomy is Here: By splitting computation between on-board GPU and the cloud, Maxwell shows how voice-driven drones can go far beyond line-of-sight or simple waypoint flying.The technical depth in this episode will blow your mind. Maxwell lays out how he integrates hardware, IoT protocols, and language models into a seamless drone orchestration layer—all without needing an army of developers.But here’s the best part: Dispatcher isn’t just a demo—it’s a sign of things to come. Voice-driven drones are poised to revolutionize everything from disaster relief to autonomous inspections.Watch this “One-Shot” episode and discover a future where talking to machines is as easy—and thrilling—as it sounds.Welcome to the dawn of drone-based AI. Welcome to AI Tinkerers.Highlights:00:00 - Introduction and Welcome00:27 - Meet Maxwell and His AI-Powered Drone01:17 - Drone Dispatcher Demo02:47 - Breaking Down the Drone's Architecture04:50 - Hardware Components and Challenges06:48 - Software and Communication Protocols10:10 - Vision Models and Real-Time Processing12:55 - Project Challenges and Iterations20:08 - Getting Started with Drone Hacking20:35 - Maxwell's Background and Future Plans22:10 - Maxwell’s Background & Inspiration24:40 - Wrapping up25:27 - Call for a co-founder

🚀 What If Your AI Could Talk Back in Real-Time? Inside Daily’s Voice Revolution!Join Kwindla Kramer, CEO of Daily, as he unveils the next frontier of conversational AI that’s about to transform how we interact with technology. In this exclusive AI Tinkerers “One-Shot”, discover how real-time voice interfaces are breaking down barriers across industries—from medical support to customer service.Key Breakthroughs:✨ Ultra-Low-Latency Conversational Agents: Rapid voice-to-voice pipelines that feel truly human, optimizing for sub-second response times. Sub-800 millisecond AI responses that feel truly human✨ Live Orchestration and Context-Aware Tools: Dynamic pipelines that integrate speech-to-text, AI reasoning, text-to-speech, and tool calls—enabling complex tasks like scheduling appointments or handling call center workflows✨ Flexible, Open-Source Ecosystem: Leveraging Pipecat’s modular design, developers can easily build, modify, and scale voice-based experiences, tapping into open innovation across industries from telehealth to global enterprise supportKwindla's vision expands the boundaries of what’s possible with real-time AI voice interfaces—unifying natural conversation, contextual awareness, and robust tooling into a single, lightning-fast layer of communication. From MIT Media Lab to pioneering real-time interaction technologies, Quinn demonstrates how generative AI is creating intelligent, context-aware conversational experiences that adapt to human communication in real-time.Witness live demos of voice agents powered by Claude and GPT-4, and explore use cases that will make you reimagine the potential of AI-driven interactions.Watch more at: https://one-shot.aitinkerers.org00:00 Introduction and Customer Feedback00:51 Welcome to AI Tinkerers One-Shot01:19 Introducing Kwindla and Daily's Journey02:38 Daily's Innovations03:19 Impact of COVID and Telehealth04:40 Generative AI and Real-Time APIs06:28 Voice to Voice Interfaces and Pipecat09:04 Technical Deep Dive into Pipecat12:04 Live Demonstration of Pipecat16:13 Future of Multimodal AI and Pipecat26:27 Exploring Innovative Use Cases29:17 Voice Interfaces Revolutionizing Call Centers31:40 Voice-Driven Drones and Hybrid Systems32:51 The Future of Conversational AI39:31 Challenges and Opportunities in Scaling AI42:14 Voice Cloning and Security Concerns46:20 Emerging Gen AI Startups52:00 Building Fluid User Interfaces with AI53:21 Reflections and Future Directions#VoiceAI #ConversationalAI #AIDevelopers #NextGenTech

Imagine an AI companion that understands your body better than your annual check-up ever could. At the latest AI Tinkerers "One-Shot" session, Varun Pant—AWS engineering manager and robotics innovator—discusses his AI Tinkerers Hackathon Grand Prize winning project, SmartNourish, a groundbreaking human-in-the-loop health monitoring system targeting the 1 in 3 Americans at risk of diabetes.Key Breakthroughs:✨ Real-time Metabolic Intelligence: Using continuous glucose monitoring and advanced AI, the system provides personalized, predictive health insights✨ Proactive Health Companion: An AI agent that learns your unique metabolic responses, suggesting precise lifestyle and nutrition recommendations✨ Dynamic Human-in-the-Loop Design: Intelligently escalates critical health signals, potentially involving nutritionists or cliniciansPant's innovation demonstrates how AI can transform healthcare from reactive to predictive, leveraging technologies like LangChain, LangGraph, and Anthropic's Claude models to create a truly personalized health monitoring experience.🔬 Developed by an engineer with a background in space robotics and machine learning, SmartNourish represents the cutting edge of AI-powered personalized medicine.Watch more at: https://one-shot.aitinkerers.org0:00 Welcome to AI Tinkerers One-Shot0:28 Intro to Human-in-the-Loop Agents1:34 Varun's Background in Robotics2:51 Defining AI Agents and Workflows4:00 Human-in-the-Loop: Key Concepts6:26 SmartNourish Project Demo8:36 CGM Data Dashboard Walkthrough10:22 Understanding Glucose Spike Patterns12:06 Tech Stack Overview [code walkthrough]13:14 Agentic Workflow Design [code walkthrough]17:16 LLM Model Selection Insights18:14 Agent Framework Exploration25:40 Hackathon Development Challenges26:07 AI-Assisted Coding Techniques29:31 Future Vision: Voice and Proactive Health30:59 Continuing the SmartNourish Project31:48 Productization Challenges33:22 Building AI Guardrails34:12 Real-World Impact of AI in Healthcare34:45 Closing Remarks and Project Details#aitinkerers #healthtech #langchain #anthropic

What if your database could instantly spawn unlimited parallel universes for AI agents to explore? In this exclusive AI Tinkerers One-Shot, Nikita Shamgunov, founder of Neon database and former MemSQL founder, reveals how his team is building the database architecture for the AI age.Nikita deep dives into their innovative storage subsystem built in Rust, demonstrating how copy-on-write mechanics make it possible to create database branches in milliseconds, regardless of size.Learn how Neon's architecture supports "disposable software" where AI agents can simultaneously explore multiple implementation paths, and why features like instant branching and automated scaling are crucial for the emerging world of autonomous development.As the database landscape shifts to accommodate AI agents and serverless architectures, Nikita shares invaluable insights from building two successful database companies and contributing to the PostgreSQL ecosystem. His vision of infrastructure evolution for AI-first development offers a compelling glimpse into the future of software creation.#AIInfrastructure #Databases #PostgreSQL #ServerlessComputing #AIDevelopment #SoftwareEngineering #DevTools #CloudNativeReady to experience the future of databases? Visit neon.tech to start building with serverless Postgres, or join our global community of AI Tinkerers to connect with fellow innovators pushing the boundaries of what's possible with AI.Subscribe to AI Tinkerers for more exclusive deep dives with elite tech leaders revolutionizing the AI landscape! 🚀[00:00] Welcome & Intro to Neon Database[03:45] PostgreSQL's Rising Dominance in Modern Apps[08:30] Instant Database Branching Demo[15:20] GenAI & Database Architecture Evolution[21:15] Agent-Driven Software Development[27:40] Database Scaling & Serverless Architecture[32:55] Auth Integration & Developer Experience[38:10] Real-World Use Cases & Customer Stories[43:25] Future of AI Infrastructure & Closing Thoughts

🚀 What if AI could build its own tools, learn from its mistakes, and continuously improve without human intervention?Join Yohei Nakajima as he unveils Baby AGI 2 – a framework that transforms how artificial intelligence creates, stores, and leverages its own functional toolkit. In this exclusive AI Tinkerers "One-Shot" session, witness a revolutionary approach where AI doesn't just execute tasks, but dynamically generates and refines its capabilities.Key Breakthroughs:• Autonomous Function Generation: AI that can create, test, and optimize its own tools in real-time• Database-Driven Learning: A flexible system where functions are stored, shared, and continuously improved• Self-Evolving Intelligence: An agent that learns from each interaction, becoming more sophisticated with every taskYohei, known for pioneering the original Baby AGI that sparked the global AI agents conversation, demonstrates how constraints and creative thinking can unlock unprecedented AI potential. From research automation to adaptive workflow creation, this session offers a glimpse into the future of intelligent systems.#AIInnovation #AutonomousAgents #MachineLearning #TechFrontiers🔥 Calling all AI Tinkerers, entrepreneurs, and tech visionaries: Are you ready to build? Join our global community of pioneers to create and share what's next. https://one-shot.aitinkerers.orgSubscribe to AI Tinkerers and never miss a cutting-edge insight. 🌐🤖0:00 Welcome to AI Tinkerers One-Shot0:25 Baby AGI Origins & Evolution2:06 Agent Frameworks: Current Landscape7:14 Baby AGI 2: Tool-Building Approach13:08 Database-Driven Function Storage [code walkthrough]18:34 Function Dependency Graphs [code walkthrough]22:01 Dynamic Function Generation Demo [code walkthrough]30:23 Self-Building Agent Levels Explained37:39 Vision for Autonomous AI Tools40:12 Current AI Agent Use Cases41:49 Future of AI Tool Development42:05 Closing Thoughts & Agent Fund Announcement

Today, Browserbase released Stagehand, an open source standard and implementation that bridges the gap between things like Playwright, Playwright, Puppeteer or Selenium, which can control a browser but which were built primarily for testing applications, with LLMs to allow a much more expressive and fault-tolerant solution for agentic systems that control web browsers.We sit down with Paul Klein, the CEO and Founder of Browserbase, and AI Tinkerers San Francisco organizer, as he unveils Stagehand and takes us on a tour of it's natural language interface and simple command interface -- "act," "extract," and "observe."Paul explores the challenges of running headless browsers in production, highlighting Browserbase's role in providing a robust, scalable infrastructure and walks us through examples of Stagehand tackling the complexities of web automation.Join AI Tinkerers "One-Shot" for this exclusive, in-depth look at the future of AI and web automation. Like and subscribe now and don't miss out on future episodes!#AITinkerers #OneShot #Stagehand #BrowserAutomation #AI #LLM #WebAutomation #OpenSource #HeadlessBrowsers #AIIntegration #Browserbase #selenium #playwright #puppeteer [00:00] Intro: Global Stagehand Launch[00:49] Browserbase: The Problem[01:55] Stagehand: Natural Language[03:31] Live Demo: To-Do List Automation[05:45] Stagehand Code Deep Dive[07:52] Browserbase Observability[15:09] Advanced Usage Scenarios[26:25] Session Recordings & Logs[30:09] Exciting AI Use Cases[34:25] Conclusion: The Future of AI @AITinkerers ​

E2B is emerging as a leader in secure sandboxes for agentic systems. Did you know they power Perplexity's code interpretation? E2B announced their $11.5M seed round today!In this exclusive AI Tinkerers "One-Shot" episode, we dive deep into E2B, a groundbreaking open-source platform that provides a secure cloud runtime for AI agents and apps. Join Vasek Mlejnsky, Co-Founder and CEO of E2B, as he takes us under the hood and explains how agentic systems benefit from seamless and secure AI code execution.➡ Discover how E2B's innovative sandbox environment empowers developers to run AI-generated code safely in the cloud, eliminating security risks and streamlining the deployment process. ➡ Witness a live demo of E2B integrated with Perplexity AI, showcasing its power in data analysis and visualization. ➡ Explore the potential of agentic systems and learn how E2B is paving the way for their widespread adoption.Vasek, co-founder and CEO of E2B, shares his insights into the future of agentic systems and the critical role of secure code execution. Don't miss this opportunity to gain exclusive knowledge and accelerate your AI projects.#AITinkerers #OneShot #AICodeExecution #AIAgents #E2B #OpenSource #AIInnovation #SecureAICloud #PerplexityAI #AgenticSystems #DataVisualization #SoftwareEngineering #TechEntrepreneurs #AIDevelopment #AISecuritySubscribe now to join the global community of AI Tinkerers and stay ahead of the curve![00:00] Intro to E2B & Agentic Systems[00:36] What is E2B? Open Source Power[01:23] E2B Architecture & Repositories[02:29] Perplexity Integration Demo[03:31] Building with E2B: Code Examples[05:44] E2B Use Cases & Applications[07:16] Securing AI Code: E2B's Approach[09:50] E2B's Vision for the Future[11:18] Q&A with Vasek Mlejnsky[12:47] Outro: Building with AI Agentshttps://one-shot.aitinkerers.org/https://e2b.dev/On Twitter:@aitinkerers@e2b_dev

Join Div Garg, founder of MultiOn, as we dive into fully autonomous AI agent tech and learn how we can use it in our own projects. We explore MultiOn's Chrome extension, API, and the many challenges including overcoming complex website structures and bot detection for practical applications like price analysis, scraping and more - towards a world where our relationship with computers may fundamentally change. https://one-shot.aitinkerers.org https://multion.ai/ - AI agents that act on your behalf ‪@AITinkerers‬ ‪@PleasePlatforms‬ @DivGarg_ on X