The Universal Approximation Theorem

I wanted to mention again the universal approximation theorem, at least i think that's how you described it in the course. It says we can't actually approximate any function that can be computed by building the appropriate neural network model. And there was an important theoretical basis for that. So yes, the non linearties are the critical factor that make the universal approximation work.