The Scaling Laws for Machine Learning

To wrap up the year we’re talking about what’s breaking the internet, again. Yes, we’re talking about ChatGPT and we’re joined by our good friend Shawn “swyx” Wang. Between his writings on L-Space Diaries and his AI notes repo on GitHub, we had a lot to cover around the world of AI and what might be coming in 2023.

Also, we have one more show coming out before the end of the year — our 5th annual “State of the log” episode where Adam and Jerod look back at the year and talk through their favorite episodes of the year and feature voices from the community. So, stay tuned for that next week.

Join the discussion

Changelog++ members get a bonus 12 minutes at the end of this episode and zero ads. Join today!

Sponsors:

• Fly.io – The home of Changelog.com — Deploy your apps and databases close to your users. In minutes you can run your Ruby, Go, Node, Deno, Python, or Elixir app (and databases!) all over the world. No ops required. Learn more at fly.io/changelog and check out the speedrun in their docs.
• InfluxData – With its data collectors and scripting languages, a common API across the entire platform, and highly performant time series engine and storage, InfluxDB makes it easy to build once and deploy across multiple products and environments. InfluxDB’s new storage engine allows developers to build real-time applications even faster and with less code. Sign up for the InfluxDB Beta program to give it a try.
• Retool – The low-code platform for developers to build internal tools — Some of the best teams out there trust Retool…Brex, Coinbase, Plaid, Doordash, LegalGenius, Amazon, Allbirds, Peloton, and so many more – the developers at these teams trust Retool as the platform to build their internal tools. Try it free at retool.com/changelog
• Fastly – Our bandwidth partner. Fastly powers fast, secure, and scalable digital experiences. Move beyond your content delivery network to their powerful edge cloud platform. Learn more at fastly.com

Featuring:

• Shawn Wang – Website, GitHub, X
• Adam Stacoviak – Website, GitHub, LinkedIn, Mastodon, X
• Jerod Santo – GitHub, LinkedIn, Mastodon, X

Show Notes:

• AI Notes
• Why “Prompt Engineering” and “Generative AI” are overhyped
• Multiverse, not Metaverse
• The Particle/Wave Duality Theory of Knowledge
• OpenRAIL: Towards open and responsible AI licensing frameworks
• Open-ish from Luis Villa
• ChatGPT for Google
• The Myth of The Infrastructure Phase

ChatGPT examples in the wild

• Debugging code
• TypeScript answer is wrong
• Fix code and explain fix
• dynamic programming
• Translating/refactoring Wasplang DSL
• AWS IAM policies
• Code that combines multiple cloud services
• Solving a code problem
• Explain computer networks homework
• Rewriting code from elixir to PHP
• Turning ChatGPT into an interpreter for a custom language, and then generating code and executing it, and solving Advent of Code correctly
• Including getting #1 place
• “I haven’t done a single google search or consulted any external documentation to do it and I was able to progress faster than I have ever did before when learning a new thing.”
• Build holy grail website and followup with framework, copy, repsonsiveness

For ++ subscribers

• Getting Senpai To Notice You
• Moving to Obsidian as a Public Second Brain

Something missing or broken? PRs welcome!