Al Feature Engineering and Machine Learning Is About Finding These Interpolative Representations

Deep learning models can be understood as unconstrained surfaces. They do have some structure coming from their architectural priors, but that structure only serves to restrict the space to a smaller space of unconstrained surface. If you want to generalize in a more systematic fashion, you either need a model with strong inductive prior, or a model which doesn't operate by empirically slicing up the ambient space like a deep learning model does. A discreet symbolic programme would be an interesting alternative for me. Just imagine the programme y equals x squared. It generalizes to any arbitrary number, right? Because it's highly structured. You can fit it with three training points, and once fit, it