The Architecture of Chat GPT

The chat GPT API that opened AI launched like, I have to say it. What is the architectural evolution of this going to look like? Are we going to keep having models that use other models or eventually does this all become one big model? It's just so interesting to see this through that lens. There's another point along that axis and we don't know where that axis leads.