Using Close Source API for Application-Dependent LLM Output

5min Snip

00:00

Play full episode

Summary

Transcript

Episode notes

Building applications using LLM output can be challenging. Using a close source API can solve the headache of deploying your own LLM. However, if your business is not about deploying an LLM, it makes sense to use a service. The speed and token size of LLM can be problematic, but there are open source tools available. Hallucination of LLM models is a major concern, so exposing prompts to end users should be done carefully. When dealing with optimization and correctness, the approach depends on the programming language and existing tools. Unit tests and minimum code changes with gradual optimization are recommended. Validating recommended changes is crucial to minimize risk.

You might have heard a lot about code generation tools using AI, but could LLMs and generative AI make our existing code better? In this episode, we sit down with Mike from TurinTech to hear about practical code optimizations using AI “translation” of slow to fast code. We learn about their process for accomplishing this task along with impressive results when automated code optimization is run on existing open source projects.

Leave us a comment

Changelog++ members save 2 minutes on this episode because they made the ads disappear. Join today!

Sponsors:

Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com
Fly.io – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs.
Typesense – Lightning fast, globally distributed Search-as-a-Service that runs in memory. You literally can’t get any faster!
Changelog News – A podcast+newsletter combo that’s brief, entertaining & always on-point. Subscribe today.