Neural Networks and Tree-Based Models

Most of the gap between a tree-based models and neural networks exists when learning on numerical features only, apparently. And so, you know, this means that you can tweak and tune as much as you want, but most of the time a tree- based model will beat you to the ground. The second fact is about categorical variables. So we all believed and myself included here from an empirical examination of this scenario that categorical variables were the main weaknesses of neural networks. Apparently, categorical variables are not the main weakness of neural Networks.