Challenges testing and evaluating LLMs

“What I built today might be obsolete tomorrow.”

This is something I heard this week from a developer, and this is not uncommon given the warp speed nonstop advancement of AI models every week. We used to measure the rate of change in months or years. Now it’s days or weeks.

In this episode, I talk about why writing code is rarely hard part, and why having good taste and shipping things that people love is the most important things we can do.