Suppose the levels of a system are known and the average energy has been measured. What, then, is the probability of being in each level?
The three in the figure are distributions with the same average. Every one of them agrees with what was measured, and yet their shapes are nothing alike.
Countless distributions agree with a measured quantity. To pick one from among them, something has to be settled about what was not measured. What to settle is the question of this piece.
There is one guiding rule: settle nothing about what is not known. Said another way, among the distributions that agree with the measurement, take the least slanted one.
What measures how unslanted a distribution is, is S = -k\sum p\log p. The figure is the case of two levels, where it is largest at p = 1/2. It falls as the weight leans to one side, and reaches 0 where one side is certain.
With nothing measured at all, this returns every state as equally likely. That is the principle of equal a priori probability itself. What was put in as an assumption in the microcanonical ensemble comes out here as a consequence of how we choose.
Two conditions are imposed. The probabilities must add up to 1. And the average energy must come out at the measured value.
Under those two, we make S as large as it can be. A constrained maximum is handled with Lagrange multipliers, and with two conditions, two multipliers come in.
The answer that comes out is p_i \propto e^{-\beta E_i}. The figure follows that working. \beta is the multiplier attached to the energy condition, and the other multiplier is the division that makes everything add to 1, which is the partition function.
The shape is familiar. It is the same Boltzmann factor found by counting the states of a bath in the piece on the weight from the hot side.
The figure is that distribution. Raise \beta and the weight gathers in the lower levels; lower it and the difference between the levels fades away.
The two derivations have nothing in common. That one counted the states of a bath. This one counted nothing at all, and used only the rule of not settling what is not known. They arrive at the same answer.
\beta was put in to make the average energy come out right. So far it has nothing to do with temperature. It is only a multiplier.
The figure is the average energy as \beta is moved. A larger \beta lowers the average, a smaller one raises it. They correspond one to one, so settling the average settles \beta.
Set that \beta against the \dfrac{1}{kT} of thermodynamics and they agree exactly. Temperature, it turns out, was the multiplier that came in to fit an average energy. Statistical mechanics can be built from counting a bath, or built from here, and arrives at the same place.