GraalVM, TornadoVM and on-device LLM performance

journey from Java Champion to Principal Software Engineer at Microsoft focusing on AI, the evolution from Java EE standards to modern AI development, writing technical books with LLM assistance, langchain4j as a Java SDK for LLMs providing abstraction over different AI providers, the importance of Java standards and patterns for LLM code generation, Boundary Control Entity (BCE / ECB) pattern recognition by LLMs, quarkus integration with LangChain4J enabling dependency injection and multi-tenancy, MCP (Model Context Protocol) as a new standard potentially replacing some RAG use cases, enterprise AI adoption using Azure AI Foundry and AWS Bedrock, model routers for optimal LLM selection based on prompt complexity, the future of small specialized models versus large general models, tornadovm enabling Java execution on GPUs with 6x performance improvements, GraalVM native compilation for LLM applications, the resurgence of Java EE patterns in the age of AI, using prompts as documentation in READMEs and JavaDocs, the advantage of type-safe languages like Java for LLM understanding, Microsoft's contribution to open source AI projects including LangChain4J, teaching new developers with AI assistance and the importance of curiosity, CERN's particle accelerator and its use of Java, the comparison between old "hallucinating architects" and modern LLM hallucinations, writing books about AI using AI tools for assistance, the structure of the Understanding LangChain4j book covering models RAG tools and MCP, enterprise requirements for data privacy and model training restrictions

Antonio Goncalves on twitter: @agoncal