The Rise of the R-Lines in Machine Learning

The transformers have now entered all four domains and shown also there are so-called vision transformers that are used for computer vision. So it has been a kind of also unifying trend in machine learning to use the same basic architecture, which is a transform architecture. But people see more also that now that we have reached this level, people see that they have some problems like the quadratic scaling. And therefore, people are now looking again towards other architectures such as recurrent neural networks or attention mechanisms.