Understanding the Multifractal Nature of Data and Neural Networks

Natural data exhibits a power law, multifractal structure, which neural networks learn to recognize. This explains their effectiveness in text and image processing but not in tabular data sets. The goal is to learn the correlations in the data, even the subtle ones that may be difficult to find using other methods like clustering algorithms. Weight Watcher is a tool that measures the fractal properties of data and neural network layers. By analyzing measures like the alpha metric, which indicates the amount of correlation, Weight Watcher helps identify layers that have not properly converged, allowing for adjustments in regularization, learning rates, and freezing of layers during training.