MLOps Coffee Sessions #170 with Phillip Carter, All the Hard Stuff with LLMs in Product Development.
We are now accepting talk proposals for our next LLM in Production virtual conference on October 3rd. Apply to speak here: https://go.mlops.community/NSAX1O
// Abstract
Delve into challenges in implementing LLMs, such as security concerns and collaborative measures against attacks. Emphasize the role of ML engineers and product managers in successful implementation. Explore identifying leading indicators and measuring ROI for impactful AI initiatives.
// Bio
Phillip is on the product team at Honeycomb, where he works on a bunch of different developer tooling things. He's an OpenTelemetry maintainer -- chances are, if you've read the docs to learn how to use OTel, you've read his words. He's also Honeycomb's (accidental) prompt engineering expert by virtue of building and shipping products that use LLMs. In a past life, he worked on developer tools at Microsoft, helping bring the first cross-platform version of .NET into the world and grow to 5 million active developers. When not doing computer stuff, you'll find Phillip in the mountains riding a snowboard or backpacking in the Cascades.
// MLOps Jobs board
jobs.mlops.community
// MLOps Swag/Merch
https://mlops-community.myshopify.com/
// Related Links
Website: https://phillipcarter.dev/
https://www.honeycomb.io/blog/improving-llms-production-observability
https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llmhttps://phillipcarter.dev/posts/how-to-make-an-fsharp-code-fixer/
The "hard stuff" post: https://www.honeycomb.io/blog/hard-stuff-nobody-talks-about-llm
Our follow-up on iterating on LLMs in prod: https://www.honeycomb.io/blog/improving-llms-production-observability
--------------- ✌️Connect With Us ✌️ -------------
Join our Slack community: https://go.mlops.community/slack
Follow us on Twitter: @mlopscommunity
Sign up for the next meetup: https://go.mlops.community/register
Catch all episodes, blogs, newsletters, and more: https://mlops.community/
Connect with Demetrios on LinkedIn: https://www.linkedin.com/in/dpbrinkm/
Connect with Phillip on LinkedIn: https://www.linkedin.com/in/phillip-carter-4714a135/
Timestamps:
[00:00] Phillip's preferred coffee
[00:33] Takeaways
[01:53] Please like, share, and subscribe to our MLOps channels!
[02:45] Phillip's background
[07:15] Querying Natural Language
[11:25] Function calls
[14:29] Pasting errors or traces
[16:30] Error patterns
[20:22] Honeycomb's Improvement cycle
[23:20] Prompt boxes rationale
[28:06] Prompt injection cycles
[32:11] Injection Attempt
[33:30] UI undervalued, charging the AI feature
[35:11] ROI cost
[44:26] Bridging ML and Product Perspective
[52:53] AI Model Trade-offs
[56:33] Query assistant
[59:07] Honeycomb is hiring!
[1:00:08] Wrap up
