The Nonlinear Library LW - Forget Everything (Statistical Mechanics Part 1) by J Bostock
Apr 23, 2024
04:45
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Forget Everything (Statistical Mechanics Part 1), published by J Bostock on April 23, 2024 on LessWrong.
EDIT: I somehow missed that John Wentworth and David Lorell are also in the middle of a sequence on this same topic here. I will see where this goes from here!
Introduction to a sequence on the statistical thermodynamics of some things and maybe eventually everything. This will make more sense if you have a basic grasp on quantum mechanics, but if you're willing to accept "energy comes in discrete units" as a premise then you should be mostly fine.
The title of this post has a double meaning:
Forget the thermodynamics you've learnt before, because statistical mechanics starts from information theory.
The main principle of doing things with statistical mechanics is can be summed up as follows:
Forget as much as possible, then find a way to forget some more.
Particle(s) in a Box
All of practical thermodynamics (chemistry, engines, etc.) relies on the same procedure, although you will rarely see it written like this:
Take systems which we know something about
Allow them to interact in a controlled way
Forget as much as possible
If we have set our systems correctly, the information that is lost will allow us to learn some information somewhere else.
For example, consider a particle in a box.
What does it mean to "forget everything"? One way is forgetting where the particle is, so our knowledge of the particle's position could be represented by a uniform distribution over the interior of the box.
Now imagine we connect this box to another box:
If we forget everything about the particle now, we should also forget which box it is in!
If we instead have a lot of particles in our first box, we might describe it as a box full of gas. If we connect this to another box and forget where the particles are, we would expect to find half in the first box and half in the second box. This means we can explain why gases expand to fill space without reference to anything except information theory.
A new question might be, how much have we forgotten? Our knowledge gas particle has gone from the following distribution over boxes 1 and 2
P(Box)={1 Box 1 0 Box 2
To the distribution
P(Box)={0.5 Box 1 0.5 Box 2
Which is the loss of 1 bit of information per particle. Now lets put that information to work.
The Piston
Imagine a box with a movable partition. The partition restricts particles to one side of the box. If the partition moves to the right, then the particles can access a larger portion of the box:
In this case, to forget as much as possible about the particles means to assume they are in the largest possible space, which involves the partition being all the way over to the right. Of course there is the matter of forgetting where the partition is, but we can safely ignore this as long as the number of particles is large enough.
What if we have a small number of particles on the right side of the partition?
We might expect the partition to move some, but not all, of the way over, when we forget as much as possible. Since the region in which the pink particles can live has decreased, we have gained knowledge about their position. By coupling forgetting and learning, anything is possible. The question is, how much knowledge have we gained?
Maths of the Piston
Let the walls of the box be at coordinates 0 and 1, and let x be the horizontal coordinate of the piston. The position of each green particle can be expressed as a uniform distribution over (0,x), which has entropy log2(x), and likewise each pink particle's position is uniform over (x,1), giving entropy log2(1x).
If we have ng green particles and np pink particles, the total entropy becomes nglog2(x)+nplog2(1x), which has a minimum at x=ngng+np. This means that the total volume occupied by each population of particles is proportion...
EDIT: I somehow missed that John Wentworth and David Lorell are also in the middle of a sequence on this same topic here. I will see where this goes from here!
Introduction to a sequence on the statistical thermodynamics of some things and maybe eventually everything. This will make more sense if you have a basic grasp on quantum mechanics, but if you're willing to accept "energy comes in discrete units" as a premise then you should be mostly fine.
The title of this post has a double meaning:
Forget the thermodynamics you've learnt before, because statistical mechanics starts from information theory.
The main principle of doing things with statistical mechanics is can be summed up as follows:
Forget as much as possible, then find a way to forget some more.
Particle(s) in a Box
All of practical thermodynamics (chemistry, engines, etc.) relies on the same procedure, although you will rarely see it written like this:
Take systems which we know something about
Allow them to interact in a controlled way
Forget as much as possible
If we have set our systems correctly, the information that is lost will allow us to learn some information somewhere else.
For example, consider a particle in a box.
What does it mean to "forget everything"? One way is forgetting where the particle is, so our knowledge of the particle's position could be represented by a uniform distribution over the interior of the box.
Now imagine we connect this box to another box:
If we forget everything about the particle now, we should also forget which box it is in!
If we instead have a lot of particles in our first box, we might describe it as a box full of gas. If we connect this to another box and forget where the particles are, we would expect to find half in the first box and half in the second box. This means we can explain why gases expand to fill space without reference to anything except information theory.
A new question might be, how much have we forgotten? Our knowledge gas particle has gone from the following distribution over boxes 1 and 2
P(Box)={1 Box 1 0 Box 2
To the distribution
P(Box)={0.5 Box 1 0.5 Box 2
Which is the loss of 1 bit of information per particle. Now lets put that information to work.
The Piston
Imagine a box with a movable partition. The partition restricts particles to one side of the box. If the partition moves to the right, then the particles can access a larger portion of the box:
In this case, to forget as much as possible about the particles means to assume they are in the largest possible space, which involves the partition being all the way over to the right. Of course there is the matter of forgetting where the partition is, but we can safely ignore this as long as the number of particles is large enough.
What if we have a small number of particles on the right side of the partition?
We might expect the partition to move some, but not all, of the way over, when we forget as much as possible. Since the region in which the pink particles can live has decreased, we have gained knowledge about their position. By coupling forgetting and learning, anything is possible. The question is, how much knowledge have we gained?
Maths of the Piston
Let the walls of the box be at coordinates 0 and 1, and let x be the horizontal coordinate of the piston. The position of each green particle can be expressed as a uniform distribution over (0,x), which has entropy log2(x), and likewise each pink particle's position is uniform over (x,1), giving entropy log2(1x).
If we have ng green particles and np pink particles, the total entropy becomes nglog2(x)+nplog2(1x), which has a minimum at x=ngng+np. This means that the total volume occupied by each population of particles is proportion...