
"Why I think strong general AI is coming soon" by Porby
LessWrong (Curated & Popular)
00:00
The Importance of a Simple Feed-Forward Network
Transformers can even be thought of as a special case of graph neural networks. It's quite possible that some of the things that make a transformer a transformer aren't actually critical to its performance, and a simpler model could do just as well. The stronger your optimizer, or the more compute you have, the less you need to worry about providing a fine-tuned structure.
Play episode from 12:49
Transcript


