In July 1982 the biologist Stephen Jay Gould learned that he had abdominal mesothelioma, a rare cancer usually linked to asbestos. As soon as he could walk after his operation, he went to Harvard’s medical library and looked up everything written about it. The papers agreed. The disease was incurable, “with a median mortality of only eight months after discovery.”
He sat stunned for about fifteen minutes, he wrote later. Then, in his words, his mind started to work again.
Most people, Gould thought, would read that sentence as “I will probably be dead in eight months.” Gould had spent years measuring how land snails in the Bahamas vary, and he knew what a median is: the one in the middle when everyone is lined up. So eight months didn’t mean that everyone died at eight months. It meant that half the patients lived less than that, and half lived longer. The question that mattered to him was which half he was likely to be in, and how much longer the longer half lived.
He worked out that the second answer had to be quite a lot. Nobody can live less than no time at all, so the shorter half was squeezed between zero and eight months. The longer half had no such wall. It could stretch out for years. When he found the data, it did: a long, thin tail of patients who had lived several years past the median. He was young, his cancer had been found early, and he was about to get the best treatment available. He saw no reason why he shouldn’t be in that tail.
He was. Gould lived another twenty years, and died in 2002 of a different cancer. In 1985 he wrote about the library in an essay with a title that says this whole chapter in five words: “The Median Isn’t the Message.”
Two kinds of middle
When a newspaper or a politician says “the average”, they usually mean one of two different things.
The mean is the one taught first. Add everything up and share it out equally. If five friends collect 100 sweets between them, the mean is 20 each. The median is the middle one in line. Put the five friends in order of how many sweets they actually have, and the median is what the third one holds.
For many things the two come out nearly the same. Darrell Huff, an American writer, made the point in his book How to Lie with Statistics in 1954: if you read that the average man somewhere is five feet tall, it hardly matters which kind of average was meant. Heights crowd around the middle, as many a little above as a little below. Money doesn’t. Most people have a modest amount, and a few have a great deal, and those few drag the mean up toward them. One millionaire in a village raises the mean income of everyone there, while the person in the middle is exactly as poor as before.
Huff made up a neighborhood to show what this lets you do. When he is selling you a house there, the average family income is a handsome $15,000 a year. A year later, when he wants the neighbors to fight a tax rise, the average is a measly $3,500. He hasn’t lied either time. The first is the mean, which three millionaires with weekend houses there pull up; the second is the median, the family in the middle. Same people, same incomes, two true numbers, and he gets to choose.
So the first question for any average is: which one? And when someone gives both and they are far apart, that gap is news in itself. It says that a few big values are pulling the mean away from most people. The sums behind all this, and a place to drag the numbers about yourself, are in Math, Going Further’s What does an average hide?
What a middle can’t tell you
Gould’s deeper point was that neither kind of middle is the real thing. The real thing is the spread: all the different cases, short and long, that the average was made from. An average is a summary of the spread, and a summary always leaves something out.
Nepal’s own numbers have a good example. By the UN’s estimates, in 1960 a baby born in Nepal could expect to live, on average, about 39 years. That sounds as if people were old at 39. They weren’t. In the same year, about one child in every three born in Nepal died before turning five. Each of those short lives counted in the average as much as a long one, and they pulled it far down. A child who got past those dangerous first years could expect a much longer life: someone who reached 15 in 1960 could expect to live to about 59.
The space between the two lines is mostly made of children who died young. As fewer of them died, the lower line rose to meet the upper one. Today a baby born in Nepal can expect about 70 years, and someone who reaches 15 about 73. Since 1960, life expectancy at birth has gone up by about 32 years, and a 15-year-old’s by about 13. The difference between those two rises is lives that no longer end in childhood.
So “people used to die at 39” is a misreading of a mean, and so is “a 15-year-old in 1960 would be dead at 39”. Both come from forgetting that an average comes from a crowd, and that a crowd can have two very different groups in it.
The spread hides in smaller places too. A school can say its average mark went up while the weakest students fell further behind, because a few top marks rose a lot. A river can be one meter deep on average and still drown you in the middle. Whenever one number stands for many, it is worth asking how much those many differ.
When the picture does the talking
A chart is a number drawn as a picture, and people read the picture first. They see a line shoot upward or a bar tower over its neighbor long before they look at the small print on the axis, if they ever do. That is what makes a chart so useful, and also what makes it so easy to bend. Huff put it well, writing of a chart drawn to look steeper: it “contains no adjectives or adverbs to spoil the illusion of objectivity.” It doesn’t need to say “collapse” if it can draw one.
The commonest bend is to cut off the bottom of the axis. Every year, Nepalis working abroad send money home, and the World Bank counts it. From 2018 to 2020, the total fell a little, from about $8.29 billion to about $8.11 billion, which is about 2% less. Here are those four numbers on one chart, with a slider for where its axis starts.
Not one number changes as the slider moves, and every one written on the chart stays true. Only the picture changes, and with it the story. With the axis at zero, the line is nearly flat, and a 2% dip looks like a 2% dip. Start the axis just under $8.1 billion and the same dip falls through most of the chart, the kind of picture that goes with a headline in capital letters.
Is a cut-off axis always a trick, then? No, and this is where it gets subtle. For bars, it is: a bar is read by its length, so its length has to stand for the whole number, which means starting at zero. For a line, what you read is where the points sit, and zooming in can be honest, even necessary. A child’s temperature going from 37 °C to 39 °C is a real fever, and on an axis that starts at zero it would look like nothing at all. The honest question isn’t whether the axis starts at zero. It is whether the picture makes the change look the size it really is, for whatever the change means. A 2% wobble in money from abroad is not a plunge. A two-degree rise in a child’s temperature is an alarm. The working of all this, with a way to measure how much a chart exaggerates, is in Math, Going Further’s Can a graph lie?
Uneven steps and growing pictures
A second bend is in the steps along the axis. A reader assumes that equal distances on a chart mean equal amounts, and that the years run in order. Break either, and the eye is fooled without anyone noticing why.
In May 2020, early in the COVID-19 pandemic, the Department of Public Health of Georgia, a state in the United States, had on its website a bar chart of new cases in the counties with the most infections, and the chart seemed to show new confirmed cases falling every single day for the past two weeks. Then people looked at the dates along the bottom. As The Atlanta Journal-Constitution put the questions: “Where does Sunday take place twice a week? And May 2 come before April 26?” The dates weren’t in order, and the counties didn’t keep the same place from day to day. The days had been arranged so the bars went down.
The health department said the chart was wrong because of “an error in how it sorted dates.” The governor’s office said the axis “was set up that way to show descending values to more easily demonstrate peak values”, and then added: “Our mission failed. We apologize. It is fixed.” Nobody outside the office can say whether it was a slip or something else. What anyone can say is how much a chart can claim through the order of its steps alone.
Steps can be uneven in quieter ways too. An axis that puts 1990, 2000, 2010, 2015, 2020 at equal distances squeezes ten-year gaps and five-year gaps into the same space, so a steady rise seems to speed up at the end. It is worth a glance at the numbers along each axis, to see that the gaps between them are the gaps they look like.
The third bend is in pictures that stand for numbers. Huff’s example is a gentle one. In 1860, he wrote, there were “something over eight million” milk cows in the United States, and by 1936 “more than twenty-five million”, about three times as many. Draw that as two cows, the second three times as tall as the first. It looks fair, since three times the cows gets three times the height. But a picture three times as tall is also three times as wide, so it covers nine times as much of the page, and a cow is solid, so it looks twenty-seven times as heavy. The eye takes in the size of the whole animal, not just its height. And, Huff added, a hasty reader may come away thinking that cows had simply got bigger.
The same goes for moneybags, houses, people, oil barrels, anything drawn bigger to mean more. A picture that grows in two directions to show a number that grew in one is telling a bigger story than its numbers. Honest picture charts keep each picture the same size and draw more of them, one cow for each million.
What it is being made to say
None of the tricks in this chapter needs a false number. Huff’s neighborhood had both its averages right. Georgia’s chart had real case counts. Huff’s cows multiplied by his own count, and the money from abroad really did dip by 2%. In each case what went wrong was the choice of which number to give, or how to draw it, and someone made that choice with a point in mind. Often an honest one, sometimes not. The way to read numbers and charts, then, is less like checking sums and more like reading an argument: what is this picture being made to say, and do the numbers underneath say it too?
A few plain questions do most of the work. Which average is this, and how far is it from the other one? How spread out are the cases behind it, and is there a long tail like Gould’s? Where does the axis start, and does the change look as big as it really is? Are the steps even, and in order? Is a picture being read by its height or by its whole size?
Every one of these numbers came from counting real people, cows or dollars, or more often from counting some of them and working out the rest. And that raises a question that no chart can answer on its face: who exactly was counted?
Found a mistake? Please tell us.
