What is Entropy?

Entropy is the count of ways a system can be arranged without you noticing. It explains why ice melts, why gas spreads, and why time has a direction — all from one deceptively simple idea.

Estimated time: ~10 min
Difficulty: intro
Sources: 4 sources

Drop an ice cube in a warm drink. You have never seen the drink spontaneously freeze and the cube grow back. That asymmetry — things mix, spread, and warm up, but never the reverse — is entropy at work. Every such “one-way door” in nature traces back to the same counting argument.

The real question: how many ways?

Before any formula, here is the key move: stop asking where the energy is and start asking how many ways it could be arranged without you noticing the difference.

Gas in a box

Imagine a box with two compartments. You pump 4 gas molecules into the left side and seal the divider. Then you lift the divider. What happens?

The molecules can now be anywhere. There are 2⁴ = 16 ways to assign each molecule to a side. Only one of those 16 arrangements has all four molecules back on the left — a 6.25% chance. With 100 molecules, the probability of spontaneous return is 1 in 2¹⁰⁰ ≈ 10³⁰. With a mole (6 × 10²³) of molecules, waiting for that return would take longer than the age of the observable universe — many, many times over.

This is not a law the universe enforces. It is just combinatorics. The “disordered” states are overwhelming in number.

The widget below makes that counting visceral. Drag the slider from 1 to 20 particles and watch how fast the “all-on-one-side” probability collapses.

Each shuffle is a random arrangement. At N=20, the probability of all particles landing on one side is roughly 1 in a million.

Check your understanding

You have 10 gas molecules in a box with two equal halves. Roughly how many total arrangements exist?

The mechanism: Boltzmann's formula

Ludwig Boltzmann turned the counting argument into a single equation, carved on his tombstone:

Entropy (S) def.

S = k · ln(W), where W is the number of microstates (arrangements) consistent with the observable macrostate, and k = 1.38 × 10⁻²³ J/K is Boltzmann’s constant.

The logarithm does two jobs at once:

It makes entropy additive. If you combine two independent systems (W₁ and W₂ microstates), the combined microstates multiply: W = W₁ × W₂. Taking the log turns that multiplication into addition: S = k·ln(W₁ · W₂) = k·ln(W₁) + k·ln(W₂). Entropy of the combined system is just the sum of its parts — which is what you expect of any sensible “amount” quantity.
It tames astronomical numbers. A gram of air has on the order of 10^(10^23) microstates. The log compresses this to a number of order 10²³, which fits on a lab bench. ^{[Thermodynamics and an Introduction to Thermostatistics]}

Show how a worked number flows through the formula

For 4 molecules that can each be in one of 2 sides:

W = 2⁴ = 16
ln(16) ≈ 2.77
S = (1.38 × 10⁻²³) × 2.77 ≈ 3.82 × 10⁻²³ J/K

That tiny entropy value reflects the tiny, toy-sized system. A macroscopic gas at room temperature has S ≈ 10 J/K — implying roughly e^(10 / 1.38×10⁻²³) microstates. The log is carrying enormous freight.

Notice that doubling W adds a fixed increment k·ln(2) ≈ 9.6×10⁻²⁴ J/K — not a doubling of S. That is the logarithm at work.

Common misconception

Entropy is a measure of messiness or chaos in the everyday sense.

What's actually true

Entropy measures the number of indistinguishable arrangements — a precise combinatorial count. A well-shuffled deck of cards is “messy” to look at but has no more entropy than any other arrangement, because every specific ordering is equally improbable. A hot gas is high-entropy not because it “looks messy” but because there are exponentially more ways to distribute kinetic energy among its molecules than to have it concentrated in a few. ^{[The Second Law (Scientific American Library)]}

Check your understanding

If W doubles, by how much does S increase?

Where the Second Law comes from

The Second Law of Thermodynamics states that the total entropy of an isolated system never decreases. This sounds like a deep law of physics. It is actually a consequence of counting.

Analogy — card shuffling is like entropy increase

Shuffle a sorted deck. Almost certainly, it becomes less sorted — not because sorting is forbidden, but because there are vastly more disordered arrangements than ordered ones. Shuffling is random; it wanders through arrangement-space; arrangement-space is almost entirely “disordered.” The Second Law is the same argument applied to molecules.

The arrow of time — why you remember the past but not the future, why films of breaking eggs look wrong when played in reverse — follows directly. The past is special only because it was low-entropy. Evolution from low to high entropy is overwhelmingly likely; the reverse is not. ^{[Feynman Lectures on Physics, Vol. 1 — Ch. 46: Ratchet and Pawl]}

The simulation below shows this directly. Hot (red) and cold (blue) particles equilibrate — and entropy only climbs.

Try 'Reverse velocities' after equilibration. The simulation is Newtonian (time-reversible in principle), yet the system never un-mixes — numerical noise amplifies and the arrow of time reappears.

Check your understanding

You reverse the velocities of all gas molecules in a container at equilibrium. What happens to entropy?

A surprising consequence: Maxwell's Demon

Here is the threshold concept: if entropy is just counting, could a clever agent — one with perfect knowledge — reverse it?

James Clerk Maxwell proposed a thought experiment in 1871: a tiny “demon” sits at a partition between two gas chambers. It watches individual molecules. When a fast molecule approaches from the cold side, it opens a little door; when a slow one approaches from the hot side, it lets it through the other way. Over time, all fast molecules accumulate on one side. Temperature difference re-emerges from nothing. Heat flows from cold to hot. Entropy appears to decrease.

The demon does not violate energy conservation — it uses no engine. The paradox is aimed squarely at the Second Law.

The resolution came over a century later (Leo Szilard 1929, Rolf Landauer 1961): the demon must remember which molecules it let through. Memory has physical state. Erasing that memory — which must eventually happen, or the demon runs out of memory — takes work and dissipates heat. The entropy increase from erasing the demon’s memory exactly offsets the entropy decrease it produced in the gas. ^{[Irreversibility and Heat Generation in the Computing Process]}

The punchline: information is physical. Storing and erasing a bit of information has a minimum thermodynamic cost of k·ln(2)·T — exactly the entropy of one binary choice at temperature T. The demon’s paradox turned out to be the birth of information thermodynamics, a field connecting Shannon entropy to Boltzmann entropy.

Show the Landauer limit calculation

At room temperature (T = 300 K):

Minimum energy to erase 1 bit = k·T·ln(2)
= (1.38 × 10⁻²³ J/K) × 300 K × 0.693
≈ 2.87 × 10⁻²¹ J per bit

That is extraordinarily small — roughly 0.018 eV. Modern CMOS transistors currently dissipate about 10⁵ times this amount per operation; we are still far from the Landauer limit. But it is real, measured, and the floor below which no computation can ever go.

Check your understanding

Maxwell's Demon appears to violate the Second Law. Why doesn't it?

Entropy — end-of-lesson quizQ 1 / 4

A gas expands to fill a larger container. What happens to entropy?