Token Predictors Capture Intelligence

Every iteration takes as input the previous tokens. It doesn't know whether they were from some external ground truth or the results of previous executions. This is most obvious when trying to execute discrete algorithms that are beyond the transformers' ability to express in a single step like arithmetic. That hallucination is accepted as the next token and collapses on certainty. Then future iterations will take it as input and drive straight into nonsense town. Would you have predicted this in 2016? I don't think I would have.