There will absolutely be microscopic differences. The question is not just do they chaotically spread through the system, but do they macroscopically amplify? I think that's going to just depend on a lot of physics that I don't specifically know. But guess what? With overwhelming probability from the macroscopic level, that box still looks like a box of gas with a uniform temperature and density,. So there's actually not a big macroscopic difference, even though microscopically things are very, very different.