The Physical Limits of Hardware Computing

The current rate of compute accumulation is enough for any of these large companies to do very weird things over the course of just a few years. Accumulating 10,000 GPT-3s wouldn't be trivial, but you're still talking about like five months of production at a price affordable to the hyperscalers. I don't actually feel that we need any significant improvements on the hardware side to reach AGI at this point,. cheaper and more efficient hardware does obviously make it easier. This section is my attempt to reason about how severe the apparent hardware cliff can get.