The Composition of Language Models

Cognitive scientists going back decades have suspected that these properties of composition and recursion are at the heart of our intelligence. So what is it about large language models we have today that allows them to engage in composition? And I think that the answer is as follows so the transformer is a little bit different to previous algorithms which have been developed for machine learning applications. The fact that it has separate representations of what and where is something which makes that algorithm more like what the human brain does. It's not actually possible to do unless you explicitly encode what and where or item and position if you like. We're often able to produce systems which can be very innovating on the algorithmic side, which