Evaluating AI Models: Challenges and Innovations

On this week's episode of the podcast, freeCodeCamp founder Quincy Larson interviews Yifan Mai, a Senior Software Engineer on Google's TensorFlow team who left the private sector to go do AI research at Stanford. He's the lead maintainer of the open source HELM project, where he benchmarks the performance of Large Language Models.

We talk about: - Open Source VS Open Weights in LLMs - The Ragged Frontier of LLM use cases - AI impact on jobs and our predictions - What to learn so you can stay above the waterline

Can you guess what song I'm playing in the intro? I put the entire cover song at the end of the podcast if you want to listen to it, and you can watch me play all the instruments on the YouTube version of this episode.

Also, I want to thank the 10,993 kind people who support our charity each month, and who make this podcast possible. You can join them and support our mission at: https://www.freecodecamp.org/donate

Links we talk about during our conversation:

- Yifan's personal webpage: yifanmai.com 

- HELM Leaderboards: https://crfm.stanford.edu/helm/ 

- HELM GitHub Repository: https://github.com/stanford-crfm/helm 

- Stanford HAI Blog: https://crfm.stanford.edu/helm/