The Next Big Thing in AI Technology?

I expect scaling to continue to produce gains, but this is less obvious than it's been the past five years. Training GPT-4 will cost $100 million, which is a lot. In 2028, will traditional big tech be clearly ahead of AI-specific companies in AI technology? It's currently showing 43% chance. And another prediction? Gary Marcus can still figure out at least three semi-normal situations that are not solid gold Magikarp style situations. This is where you can give a language model an internet connection and tell it something like order cheap Chinese food off Uber Eats.