Is There a Limit to Scaling?

Lamey: I think it's possible that we still have many years ahead of surprising new capabilities by just scaling. The amount of data that these models need is at least three orders of magnitude above what say a five year old sees in their life with comparable competence or maybe an eight year old. Lamey: There's a more powerful way to get to those capabilities which humans showcase every human showcases by age three or five such capabilities with far less data ever needed to get there. Maybe for now, try to scale a bit more. But long term, try to build something more human-like, human level might require something different than just pushing to scale.