The Nonlinear Library LW - Generalized Stat Mech: The Boltzmann Approach by David Lorell
Apr 12, 2024
32:42
Welcome to The Nonlinear Library, where we use Text-to-Speech software to convert the best writing from the Rationalist and EA communities into audio. This is: Generalized Stat Mech: The Boltzmann Approach, published by David Lorell on April 12, 2024 on LessWrong.
Context
There's a common intuition that the tools and frames of statistical mechanics ought to generalize far beyond physics and, of particular interest to us, it feels like they ought to say a lot about agency and intelligence. But, in practice, attempts to apply stat mech tools beyond physics tend to be pretty shallow and unsatisfying.
This post was originally drafted to be the first in a sequence on "generalized statistical mechanics": stat mech, but presented in a way intended to generalize beyond the usual physics applications. The rest of the supposed sequence may or may not ever be written.
In what follows, we present very roughly the formulation of stat mech given by Clausius, Maxwell and Boltzmann (though we have diverged substantially; we're not aiming for historical accuracy here) in a frame intended to make generalization to other fields relatively easy. We'll cover three main topics:
Boltzmann's definition for entropy, and the derivation of the Second Law of Thermodynamics from that definition.
Derivation of the thermodynamic efficiency bound for heat engines, as a prototypical example application.
How to measure Boltzmann entropy functions experimentally (assuming the Second Law holds), with only access to macroscopic measurements.
Entropy
To start, let's give a Boltzmann-flavored definition of (physical) entropy.
The "Boltzmann Entropy" SBoltzmann is the log number of microstates of a system consistent with a given macrostate. We'll use the notation:
SBoltzmann(Y=y)=logN[X|Y=y]
Where Y=y is a value of the macrostate, and X is a variable representing possible microstate values (analogous to how a random variable X would specify a distribution over some outcomes, and X=x would give one particular value from that outcome-space.)
Note that Boltzmann entropy is a function of the macrostate. Different macrostates - i.e. different pressures, volumes, temperatures, flow fields, center-of-mass positions or momenta, etc - have different Boltzmann entropies. So for an ideal gas, for instance, we might write SBoltzmann(P,V,T), to indicate which variables constitute "the macrostate".
Considerations for Generalization
What hidden assumptions about the system does Boltzmann's definition introduce, which we need to pay attention to when trying to generalize to other kinds of applications?
There's a division between "microstates" and "macrostates", obviously. As yet, we haven't done any derivations which make assumptions about those, but we will soon. The main three assumptions we'll need are:
Microstates evolve reversibly over time.
Macrostate at each time is a function of the microstate at that time.
Macrostates evolve deterministically over time.
Mathematically, we have some microstate which varies as a function of time, x(t), and some macrostate which is also a function of time, y(t). The first assumption says that x(t)=ft(x(t1)) for some invertible function ft. The second assumption says that y(t)=gt(x(t)) for some function gt. The third assumption says that y(t)=Ft(y(t1)) for some function Ft.
The Second Law: Derivation
The Second Law of Thermodynamics says that entropy can never decrease over time, only increase. Let's derive that as a theorem for Boltzmann Entropy.
Mathematically, we want to show:
logN[X(t+1)|Y(t+1)=y(t+1)]logN[X(t)|Y(t)=y(t)]
Visually, the proof works via this diagram:
The arrows in the diagram show which states (micro/macro at t/t+1) are mapped to which other states by some function. Each of our three assumptions contributes one set of arrows:
By assumption 1, microstate x(t) can be computed as a function of x(t+1) (i.e. no two microstates x(t) both evolve to the same later microstate x(t+1)).
By assumption 2, macrostate y(t) can be comput...
Context
There's a common intuition that the tools and frames of statistical mechanics ought to generalize far beyond physics and, of particular interest to us, it feels like they ought to say a lot about agency and intelligence. But, in practice, attempts to apply stat mech tools beyond physics tend to be pretty shallow and unsatisfying.
This post was originally drafted to be the first in a sequence on "generalized statistical mechanics": stat mech, but presented in a way intended to generalize beyond the usual physics applications. The rest of the supposed sequence may or may not ever be written.
In what follows, we present very roughly the formulation of stat mech given by Clausius, Maxwell and Boltzmann (though we have diverged substantially; we're not aiming for historical accuracy here) in a frame intended to make generalization to other fields relatively easy. We'll cover three main topics:
Boltzmann's definition for entropy, and the derivation of the Second Law of Thermodynamics from that definition.
Derivation of the thermodynamic efficiency bound for heat engines, as a prototypical example application.
How to measure Boltzmann entropy functions experimentally (assuming the Second Law holds), with only access to macroscopic measurements.
Entropy
To start, let's give a Boltzmann-flavored definition of (physical) entropy.
The "Boltzmann Entropy" SBoltzmann is the log number of microstates of a system consistent with a given macrostate. We'll use the notation:
SBoltzmann(Y=y)=logN[X|Y=y]
Where Y=y is a value of the macrostate, and X is a variable representing possible microstate values (analogous to how a random variable X would specify a distribution over some outcomes, and X=x would give one particular value from that outcome-space.)
Note that Boltzmann entropy is a function of the macrostate. Different macrostates - i.e. different pressures, volumes, temperatures, flow fields, center-of-mass positions or momenta, etc - have different Boltzmann entropies. So for an ideal gas, for instance, we might write SBoltzmann(P,V,T), to indicate which variables constitute "the macrostate".
Considerations for Generalization
What hidden assumptions about the system does Boltzmann's definition introduce, which we need to pay attention to when trying to generalize to other kinds of applications?
There's a division between "microstates" and "macrostates", obviously. As yet, we haven't done any derivations which make assumptions about those, but we will soon. The main three assumptions we'll need are:
Microstates evolve reversibly over time.
Macrostate at each time is a function of the microstate at that time.
Macrostates evolve deterministically over time.
Mathematically, we have some microstate which varies as a function of time, x(t), and some macrostate which is also a function of time, y(t). The first assumption says that x(t)=ft(x(t1)) for some invertible function ft. The second assumption says that y(t)=gt(x(t)) for some function gt. The third assumption says that y(t)=Ft(y(t1)) for some function Ft.
The Second Law: Derivation
The Second Law of Thermodynamics says that entropy can never decrease over time, only increase. Let's derive that as a theorem for Boltzmann Entropy.
Mathematically, we want to show:
logN[X(t+1)|Y(t+1)=y(t+1)]logN[X(t)|Y(t)=y(t)]
Visually, the proof works via this diagram:
The arrows in the diagram show which states (micro/macro at t/t+1) are mapped to which other states by some function. Each of our three assumptions contributes one set of arrows:
By assumption 1, microstate x(t) can be computed as a function of x(t+1) (i.e. no two microstates x(t) both evolve to the same later microstate x(t+1)).
By assumption 2, macrostate y(t) can be comput...