The Gibbs Entropy S = −k Σ p log p

Every state has a different probability

In the microcanonical ensemble every state happened as often as every other. The probability of one was therefore \dfrac{1}{W}, and the entropy came from counting alone: S = k \log W.

The canonical ensemble does not allow that. Each state carries its own probability \dfrac{e^{-\beta E_i}}{Z}, so counting how many there are will never tell you this shape.

The left of the figure is microcanonical and the right is canonical. On the left every bar is the same height; on the right the heights are all different. S = k \log W works only on the left, and the right has no formula for the entropy yet.

Weight by the probability, then add

So we write it this way: S = -k \displaystyle\sum_{i} P_i \log P_i. It is called the Gibbs entropy. Take the states one at a time and add them up, each weighted by its own probability.

The faint bars in the figure are the probabilities, the bright bars what each state contributes, -P_i \log P_i. A state with probability near 1 contributes little because \log P_i is near 0; a state with probability near 0 contributes little because P_i itself is small.

What counts most is the middle. That is, the less settled it is where the system sits, the larger the entropy. When entropy is described as an amount of spread, this weighting is what is being pointed at.

Equal probability brings back k \log W

Let us check that this does not contradict what came before. Put in the microcanonical case, where each of W states carries \dfrac{1}{W}.

The figure follows it through. The same term appears W times, so the sum is W times \dfrac{1}{W} \log \dfrac{1}{W}, and out comes S = k \log W.

So the new formula has not replaced the old one. The old one was the special case in which every state has the same probability. The Gibbs formula works for any ensemble at all, as soon as the probabilities are known.

It agrees with the free energy

Next put in the canonical probability. Since \log P_i = -\beta E_i - \log Z, taking the logarithm brings the exponent down.

Returning that to the sum, only two things are left, the mean energy and \log Z. We get S = k\beta \langle E \rangle + k \log Z. The figure is those four lines.

Rewriting the last line gives F = \langle E \rangle - TS. In the piece on the partition function, F = -kT \log Z was simply set down as a result. Only here is it confirmed to be the same thing as thermodynamics' F = E - TS.

The less settled it is, the larger it is

Finally, what the temperature changes. The figure is the entropy of a system with only two levels.

At low temperature it is 0. The system is settled in the lower level, so there is nothing left unsettled. Raise the temperature and the upper level starts to be used, and the entropy rises. At high temperature the two are used equally often and it approaches \log 2.

It never passes \log 2, because there are only 2 states. The number of states fixes the ceiling on the entropy, and the lopsidedness of the probabilities fixes how far below the ceiling it sits. The Gibbs formula puts both of those in one line. In the next chapter we turn all of this on systems close at hand.

Probability and large numbers
The microcanonical ensemble
The canonical ensemble
Applications
Phase transitions
Quantum statistics
Time and fluctuations