Benchmark Nuances and SWE Bench Concern

The Don’t Worry About the Vase Podcast is a listener-supported podcast. To receive new posts and support the cost of creation, consider becoming a free or paid subscriber.

* 00:00 - Introduction

* 02:42 - The Official Pitch

* 08:28 - Our Price Cheap

* 09:08 - Official Benchmarks

* 17:24 - SemiAnalysis Doublecheck

* 18:40 - Other People’s Benchmarks

* 21:51 - Vend That Bench

* 25:34 - Planning Is Essential

* 27:13 - Choose Your Fighter

* 28:53 - Cyber Lack Of Security

* 29:21 - You Get What You Give

* 30:27 - True Story

* 31:31 - Ethan Mollick Thinks GPT-five point five Is A Big Deal

* 32:00 - SemiAnalysis Loves GPT-five point five Especially In Codex

* 34:30 - Choose Your Fighter

* 35:04 - Positive Reactions

* 42:35 - Lazy and Literal

* 43:39 - Goblins, Gremlins and Trolls, Oh My

* 45:13 - Other Reactions

* 45:42 - Claude Ambition

* 46:04 - Other Notes

https://open.substack.com/pub/thezvi/p/gpt-55-capabilities-and-reactions?utm_campaign=post-expanded-share&utm_medium=web