In March 1951 a group of American intelligence experts finished a report with a plain, grim title: Probability of an Invasion of Yugoslavia in 1951. The question was whether the Soviet Union and its allies would attack Yugoslavia that year. A score or so of experts had worked over the draft, and its most important sentence said that an attack “should be considered a serious possibility.”
A few days later one of the men behind it, Sherman Kent, was talking with the head of a planning group at the State Department, who had read the report closely. Suddenly the official asked: “By the way, what did you people mean by the expression ‘serious possibility’? What kind of odds did you have in mind?”
Kent said that he himself put the odds at about 65 to 35 in favor of an attack. The official was jolted. He and his colleagues had taken “serious possibility” to mean odds much lower than that. Kent, troubled, went back and asked each member of the board that had approved the wording what odds they had meant. Each had something different in mind. The lowest had meant about 20 to 80, a one-in-five chance. The highest had meant 80 to 20. The same two words had covered everything from “probably not” to “probably”, and the people who agreed on them never noticed that they didn’t agree at all.
Nobody had lied, and nobody had been careless in the usual sense. Every one of them had a number in mind. The words just hid it.
A dial, not a switch
Most of the time we talk as if a belief had two settings, like a light switch. You believe a thing or you don’t. It’s true or it’s false. But very little of what you know is really like that. You are more sure of some things than others, and the difference matters.
A better picture is a dial that runs from 0 to 100. At 100 you are certain a thing is true. At 0 you are certain it’s false. At 50 you have no idea either way. Nearly everything you believe sits somewhere in between: that it will rain in Kathmandu at some point in July is up near the top, and that the next bus will be on time may be somewhere in the middle.
One way to find where the dial sits is to imagine a bet. Suppose someone offers you Rs 100 if you’re right and asks for Rs 100 if you’re wrong. If you’d happily take that, you’re above 50. Would you still take it if you had to pay Rs 900 when wrong, to win Rs 100 when right? Only if you were about 90 sure. The odds you would honestly accept are a good measure of how sure you really are, and much harder to fool than a feeling.
That’s the difference between a bet and a badge. A badge is something you wear to show which side you are on. Once a belief becomes a badge, changing it feels like losing. A bet is something you hold for a reason, and when the reasons change, you change the bet. Nobody thinks less of a cricket captain for changing the field when the batting side changes its plan.
The numbers on the dial mean something exact, which is the useful thing about them. At 80 you expect to be wrong about one time in five. At 95, about one time in twenty. So if you look back over a hundred things you were 95 sure of and not one turned out wrong, you weren’t really at 95: you were nearer 99 and should have said so. If twenty were wrong, you were nearer 80.
Fifty is the odd one, because it can mean two different things. A coin toss is a 50 even when you know everything there is to know about the coin. And “Which is older, the temple or the bridge?” can be a 50 because you know nothing at all. Both are honest. The first says the world is uncertain. The second says you are. Telling those two apart is half of what thinking clearly is about.
Kent and a colleague, a lawyer named Max Foster, set out to stop the words hiding their numbers. Kent wrote that some would remember Foster for his elegant writing, and that nobody would forget his oddities: he was the man who always wore his hat in the house. After a first try with eleven phrases, which they found far too fussy, they settled on five words, each with its odds.
Their colleagues split into two camps, which Kent called the mathematicians and the poets. The poets said that putting numbers on judgments like these gave them a precision they didn’t have, and that good writing needed its “may well” and its “distinctly possible”. Their strongest point was that readers are poets too: give them the general feel and it gets across, labor to be exact and it may not. Kent admitted the truth of it and then wrote one of the most honest lines in his essay: “I realize the truth in the above; I am not reconciled; I deplore it.”
He mostly lost. Some offices pinned the chart to a wall, some people laughed at it, and Kent dropped any hope of making everyone use it. But the idea came back. The world’s climate scientists, writing the reports of the Intergovernmental Panel on Climate Change, now give their words fixed meanings: “very likely” means 90 to 100 percent, “likely” 66 to 100, “about as likely as not” 33 to 66. Even that isn’t enough on its own. In one experiment, people read sentences from the panel’s 2007 report and said what number each word meant to them, and their numbers strayed well away from the panel’s, even when they had the definitions in front of them. What helped, in a later test across 24 countries, was to put the number right beside the word: “very likely (90–100%)”. Words are slippery that way. A number isn’t.
Keeping score
So suppose someone gives you a number. How do you tell a good 70 from a bad one?
Not from a single case. Every year, before the rains come, Nepal’s Department of Hydrology and Meteorology publishes an outlook for the monsoon, and it does the honest thing: it gives odds, not promises. In May 2025 it leaned toward a wet year. For most of Gandaki Province, eastern Karnali and northern Lumbini it gave a 55 to 65 percent chance of more rain than usual, and smaller chances elsewhere. It also said plainly that a season can’t be forecast with complete certainty.
Then the rains came. Most of the country had a normal or wetter season, while Madhesh and much of the southern plains were dry. Taken all together, Nepal got 90 percent of its normal monsoon rain, right on the line the department draws between a normal season and a dry one.
Was the forecast right? One season can’t say. A 60 percent forecast is supposed to miss about four times in ten, so a wet summer wouldn’t have proved it good and a dry one wouldn’t prove it bad. It’s the same with a forecaster who says “60 percent chance of rain” on a day that stays dry: nothing has gone wrong yet. To judge a forecaster you need many forecasts: gather every day they said 70 percent, and count how many of those days it rained. If it rained on about 70 in every 100, their 70 meant what it said. Forecasters call this being well calibrated. The department keeps score on itself in a small way: its 2025 outlook set the previous year’s forecast beside the rain that actually fell.
People really have counted. Two researchers in the United States collected about 169,000 forecasts of rain made by The Weather Channel, a big American forecaster, for 42 cities over 14 months. When it gave rain a chance between 40 and 90 percent for the next day or two, you could take the number at face value. Its low numbers were another matter. On days when it said 20 percent for that same day, it rained only about one day in eighteen.
You can even check people who never existed. The writer Julia Galef went through the Star Trek shows and films and counted every prediction made by Mr. Spock, the show’s famously logical science officer, and whether it came true. By her count, when Spock said something was impossible, it happened 83 percent of the time. When he gave something better than a 99.5 percent chance, it happened 17 percent of the time. His plain “likely” predictions, to be fair to him, did all right.
Real people do better than Spock, but not as well as they think. In experiments in the 1970s with questions of general knowledge, people who said they were completely sure of an answer were wrong about one time in five, sometimes nearly one in three. How big that gap is depends on the questions. Tricky ones make it large. Some researchers found that it nearly disappears when the questions are picked at random, and others that it doesn’t, and they still argue about it. What nobody disputes is that “certain” was often wrong, which makes it the setting most worth a second thought.
That doesn’t mean the safe thing is to say 50 to everything. Someone who answered every two-way question with “50 percent” would be right about half the time, so they’d be perfectly calibrated, and of no use to anyone. The aim is both at once: turn the dial up where you really know, down where you don’t, and be honest about which is which. That can be learned, by the same method the weather forecasters use. Make a lot of calls, write down how sure you were, and count.
A real answer
All of which makes “I don’t know” more respectable than it sounds. It’s a real answer, the honest 50. Kent made room for it too. He wanted the word “possible” kept for things that could happen but whose odds couldn’t be judged at all, used plainly and never dressed up. “The short and proper way out”, he wrote, “is to say that its happening is possible and stop there.” What annoyed him was “possible” with a word bolted on to make it sound like odds, a “serious possibility” or a “distinct possibility”, which is exactly what had gone wrong over Yugoslavia.
An honest “I don’t know” is useful to the person who hears it. It tells them to look elsewhere, or to wait. The damage is done by a 50 dressed up as a 95: the message forwarded with total confidence by someone who had no idea, or the friend who says “definitely” about a bus time they half remember. The words sound the same as real knowledge. Only the number, if anyone asked for it, would give them away.
And 100 is a setting to use very rarely. At 100 nothing could ever change your mind, because no evidence moves a belief that is already certain. That’s right for two and two making four. It is almost never right for a claim about the world, which is why careful people so seldom use it, and why the people most sure of themselves deserve a second look.
Kent went on making, in his own words, “a nuisance of myself” about it for almost fourteen years before he wrote it all down in an essay in 1964. His complaint was never with words as such. It was with words that seem to say how sure someone is while keeping the number to themselves. The fix is simple to say and takes practice to do: when you believe something, ask yourself how sure, and give the answer as a number, even if only in your own head. Then keep score.
Found a mistake? Please tell us.
