The Trouble with Entropy

If someone points out to you that your pet theory of the universe is in disagreement with Maxwell's equations-then so much the worse for Maxwell's equations. If it is found to be contradicted by observation-well, these experimentalists do bungle things sometimes. But if your theory is found to be against the second law of thermodynamics I can give you no hope; there is nothing for it but to collapse in deepest humiliation." (Eddington, A.S.,

……………..in The Nature of the Physical World

I suspect that at least 95% of physicists think that statistical mechanics and statistical entropy explain the Second Law of Thermodynamics. The rest of us are a bit less sure.

Entropy in statistical mechanics is proportional to logarithm of the volume of phase space occupied by the statistical ensemble in question. The volume occupied in phase space is just the volume of all accessible states of the system. A classic simple system is a perfect gas in box divided into two parts by a removable partition, with all the molecules initially on one side. When the partition is removed the volume of the box doubles, and the volume of accessible phase space increases by a factor of 2^N, where N is the number of molecules. The consequent entropy increase of N log 2 agrees well with experiment.

So where is there a problem? A fundamental theorem of Hamiltonian dynamics, called Liouville’s Theorem, says that in Hamiltonian evolution the volume of an ensemble of systems in phase space is unchanged. There is a little problem of the contrast between “unchanged” and “increases by a factor of 2^N.” The conventional explanation is that when the ensemble expands into the larger phase space, it does so by developing “fingers” that project almost everywhere into the larger volume while retaining their original volume. Thus, the initial space filling volume transforms itself into a dense network of tiny filaments. The entropy increase is captured only when you “coarse grain,” or divide up the phase space into large blocks and count as part of the new, larger, phase space any volume which has a trace of the old volume. The problem is that this seems to do a bit of violence to the notion that all accessible points in the phase space be occupied. I should mention that the above reasoning is based on an ideal, isolated system, an ideal not reached for real systems where various fudges are available. That last is not possible when we consider the Universe as a whole, where the problem of defining entropy becomes more acute.

Roger Penrose talks a bit about these matters in his book The Road to Reality. He is inclined to suspect that this subjective or arbitrary element in the definition and computation of entropy might be hiding important truths. As usual, I’m hoping my readers might have some insights that would clarify these thorny questions.