In 1904 crowds in Berlin came to see a horse who seemed to think. Clever Hans’s owner was Wilhelm von Osten, an old mathematics teacher who had taught him much as he would have taught a schoolboy, with carrots for rewards. Ask Hans what three and five made, and he tapped eight times with his right forefoot. He was said to add, multiply and divide, to read, to tell the time and to name musical notes. Von Osten never charged anyone to watch.
Experts came to catch the trick, and couldn’t. In September thirteen men, among them a circus manager, two retired army officers, schoolteachers and the director of the Berlin zoo, watched Hans and signed a report. They had found no signal of any kind. They had even tried questions whose answers von Osten himself didn’t know. They agreed that the case was worth a serious scientific study.
The study went to Oskar Pfungst, a psychologist at Carl Stumpf’s Psychological Institute in Berlin. Pfungst didn’t argue about whether a horse could think. He changed one thing at a time and counted, and for the first time he asked Hans questions that nobody present could answer.
First he held up cards with numbers on them and asked Hans to tap the number. Sometimes the questioner could see the card, and sometimes nobody in the courtyard could. When the questioner knew the number, Hans got 98% of them right. When nobody knew, he got 8%. Then Pfungst put large blinders on the horse, so that sometimes Hans could see the man questioning him and sometimes he couldn’t. When he could see him, he was right 89% of the time. When he certainly couldn’t, he got two answers right out of 35.
So Hans wasn’t doing sums. He was watching people. Pfungst then watched the people too, and found what thousands of visitors had missed. A questioner who had asked a question leaned forward very slightly to look at the horse’s hoof. When the count reached the right number, he made a tiny upward jerk of the head, and Hans stopped. Nobody was doing it on purpose. Von Osten himself never accepted the finding and went on showing his horse.
To make sure, Pfungst played the horse himself. In his laboratory he asked 25 people of all ages, children of five among them, to think hard of a number while he tapped with his hand. None of them knew what the test was for. All but two made the same small jerk of the head when he reached their number, and almost none of them knew they had moved.
Everyone in that courtyard was honest, and the crowd still saw what it expected to see. That is the whole problem of psychology in one horse. The thing being studied, a mind, can’t be seen. And the people studying it have minds of their own, which expect things and lean forward without being told.
Five ways to find out
Psychologists have a small set of ways round that problem, and each one catches something the others miss.
You can ask people, with a survey or an interview. You can watch them, as they go about their lives. You can look very closely at one person, in a case study. You can measure two things in many people and see whether they go together, which is correlation. Or you can do what Pfungst did, and change one thing on purpose to see what follows, which is an experiment.
The first three describe what people are like. Correlation lets you predict: if two things go together, knowing one tells you something about the other. Only an experiment can show what causes what, and so what would change a person’s behavior if you changed their situation. Observing, measuring, predicting and controlling are the four goals psychology sets itself, and each method reaches some of them.
People have other ways of knowing. Rationalism trusts reason: think hard enough from what seems certain and you will reach the truth. Intuition is knowing without being able to say how, the hunch that a friend is upset before they say a word. Spirituality finds knowledge in a faith’s teachings or in inner experience, such as meditation. Logical thinking draws a conclusion step by step, so that if every step is sound the conclusion must follow. Science uses two of these all the time: intuition to come up with a guess, and logic to work out what the guess predicts. What it adds is a last test. Reason, a hunch or a teaching can each be sincerely wrong about minds, so in psychology an answer has to survive being checked against what really happens, in ways anyone could repeat.
Whatever the method, a study runs in roughly the same steps. Someone notices a question (can this horse really count?). They make a guess that could turn out wrong, a hypothesis (he answers only when he can see someone who knows). They test it. Then comes verification: the result has to be checked, by repeating the test and by other people trying it, before anyone should lean on it. Pfungst’s blinders were his test. The checking was the 25 people in his laboratory.
Every study also needs its variables, the things that vary and get measured. In Pfungst’s card test, whether the questioner knew the number was what he changed, the independent variable. How many of Hans’s answers were right was what he measured, the dependent variable, because it depended on the first. Everything else that might sway the result, such as the weather, the crowd, or which man was asking, is an extraneous variable, and the art of a study lies in keeping those from muddying the answer.
Asking and watching
The quickest way to learn what people think is to ask them, and a survey asks many people the same questions in the same way. Nobody can ask everyone, so a survey asks a sample, a smaller group that stands for the whole population it wants to know about. If chance picks the sample, so that everyone in the population has a fair chance of being asked, it is a random sample, and it tends to look like the whole. A non-random sample, such as your classmates or whoever answers a post online, is easier to get and may be nothing like the whole. How a good sample is drawn is told fully in Thinking Clearly, in Who was asked.
Nepal’s first national survey of mental health, from January 2019 to January 2020, shows both the care it takes and the troubles. Chance picked wards in all seven provinces, giving bigger ones a bigger chance, then people within households. It set out to interview 9,200 adults and 5,888 teenagers face to face, with a standard psychiatric interview translated into Nepali and checked to make sure it made sense here. The 30 interviewers were trained for three weeks first. Almost 97% of the adults chosen finished the interview. Even so, the survey found the fewest problems in Province 2, and its own authors believe those were badly undercounted. One reason they give is that the questions were asked only in Nepali, in a province where the local language is spoken more than Nepali. A question can be asked perfectly and still not be heard.
The deeper trouble is that people’s answers about themselves can be wrong in two ways. They may tell you what sounds good, a pull psychologists call social desirability: fewer drinks, more homework, kinder thoughts. And they may simply not know. Asked how long they spend on their phones, people can sound sure. But in 2021 a team gathered every study it could find that had compared what people said about their screen time with what their phones or computers had logged. In only 3 of 49 comparisons did a group’s average answer come within 5% of the logged time, and groups were about as likely to say too much as too little. Nobody needs to be lying for a survey to mislead. Memory is a guess too.
Answers can be counted or told. A question with fixed choices (“strongly agree” to “strongly disagree”) gives numbers, which is quantitative data. An open interview gives people’s own words, which is qualitative data, richer and harder to add up. And either can be gathered fresh by the researcher, as primary data, or taken from records someone else kept, such as school results or hospital files, as secondary data. What all of it must be is empirical, collected by looking in a way someone else could repeat, not anecdotal, a story someone tells. The crowds around Hans had anecdotes. Pfungst had counts.
When you can’t trust what people say, you can watch what they do. In naturalistic observation researchers watch people, or animals, in their ordinary settings and write down what happens. Jane Goodall spent decades watching chimpanzees in the wild this way. When the American psychologist Pamela Cole wanted to know how adults in Tamang and Brahman villages in Nepal’s hills answered children’s feelings, her team didn’t ask the parents. Women from those villages sat in homes and wrote down what happened around each small child.
Watching has its own two troubles, and Hans had both. The watched may change because they’re watched. And the watchers see what they expect, which is observer bias. The usual guard is to decide beforehand exactly what counts as, say, a child looking ashamed, and to have two observers record the same scenes separately and see how often they agree. Watching also can’t make anything happen, so on its own it can’t show why.
The third way is to look very closely at a single person, a case study. A rare case can teach what no crowd can. A man known for decades only as H.M. lost the power to form new lasting memories after brain surgery in 1953, and was studied for more than fifty years. Much of what is known about how memory is divided up began with him. But one person may be like nobody else, a case can’t be repeated, and it rests on the honesty of whoever writes it up. A case is a good place to find a question. It is a weak place to settle one.
Two things that go together
Often what you most want to know can’t be watched in one person or asked straight out. Does a certain kind of upbringing go with how a child turns out? Then you measure both things in many people and see whether they rise and fall together.
The strength of that link is given by one number, the correlation coefficient, written r, which runs from −1 to +1. At +1 two things rise together perfectly; at −1 one falls exactly as the other rises; at 0 there is no straight-line link at all. Height and weight go together: taller people tend to be heavier, a positive correlation. Hours of sleep and tiredness go opposite ways, a negative one. Most links in psychology are much weaker than these. Between the screen time people reported and the screen time their devices logged, across 66 comparisons with over 52,000 people, r was about 0.38. That’s a real link, but loose enough that one number can’t stand in for the other.
A correlation is good for predicting. What it can’t do on its own is show what causes what. Ice cream sales and crime both rise in hot weather, so they go together, and neither causes the other. A third thing, the heat, drives both. And when two things do go together, it isn’t always clear which way round it runs.
That is why one of the most argued questions in child psychology is still argued. In 2016 two researchers pooled 75 studies of spanking, covering about 160,000 children. They defined spanking as hitting a child on the bottom or limbs with an open hand. It went with 13 of the 17 outcomes they looked at, and every one of those links was to something bad, such as aggression or mental health problems. Nobody can settle this with an experiment, since it would be wrong to assign some children to be hit. So it rests on correlations, and critics ask which way the arrow points: are children hit because they misbehave, or do they misbehave because they are hit, or both? Studies that follow the same children for years, called longitudinal studies, help, because they can see what came first. A 2013 review of only such studies found that spanking went with later behavior problems at r = 0.14, but once it allowed for how the children had behaved to begin with, that fell to 0.07, which its author called trivial. The 2016 team found their links about as strong whatever the kind of study. The two sides still disagree about how much of the link is the smacking. Why going together is not the same as causing, and how causes are found without an experiment, are told in Thinking Clearly, in Together is not because and Without an experiment.
Changing one thing
An experiment does what a correlation can’t. Instead of waiting to see what goes with what, you make one thing different, keep everything else the same, and see whether the result changes.
Say a teacher wonders whether ten minutes of exercise before a class makes students pay attention better. She could compare students who already exercise with those who don’t, but they might differ in a hundred other ways. So in an experiment she takes one group, the experimental group, through the exercise, and a second group, the control group, through ten quiet minutes instead. She has to say exactly what she will count as paying attention, perhaps a score on a short test of concentration at the end of class. That precise description is an operational definition, and it lets someone else repeat the study and get a fair comparison.
The most important step is to let chance decide who goes in which group, by drawing names or tossing a coin. This is random assignment, and it is not the same as random sampling. Random sampling decides who is in the study at all, so the result can speak for a wider population. Random assignment decides who gets what, so the two groups start out alike in everything, the keen and the sleepy, the strong and the shy, spread evenly by luck. Then, if the groups end up different, the exercise is the likeliest reason.
Hans’s lesson comes in at the next step. People who know what is expected of them behave differently, and so do the experimenters. Being given a treatment can change how people feel and what they report, even when it is a dummy with nothing in it, a placebo, which is why a control group may be given one that looks exactly like the real thing. When the people in a study don’t know which group they are in, it is single-blind. When the people running it don’t know either, it is double-blind, so they can’t lean forward at the right moment, as Hans’s questioners did. Pfungst’s card test, where nobody present knew the number, is the double-blind idea in a courtyard.
When the results come in, two questions matter. Could chance alone have made a difference this big? If that is unlikely, by a fixed rule, the result is called statistically significant. And how big is the difference? That is its effect size, and a significant result can still be a small one.
Experiments have weaknesses too. A laboratory is not a life, and people asked to do strange tasks in a strange room may not act as they would at home. Many questions, like the one about spanking, can’t be tested by experiment at all. And a single experiment can be a fluke, which is why the last step, checking, matters so much. A finding becomes something to trust only when other people, with other samples, have done it again and got the same answer. Psychology has learned, painfully, how often that step was skipped, and this book keeps asking of each finding whether it held.
Fair to the people in it
People in a study aren’t horses, and the rules for treating them fairly came late and were learned the hard way.
From 1932 to 1972 the United States Public Health Service followed 399 Black men in Alabama who had syphilis, along with 201 who didn’t, to see what the untreated disease did. There is no evidence that the men gave informed consent, and they weren’t offered treatment even after a cure was readily available. In psychology the warning cases were less deadly but still shocking. In 1963 Stanley Milgram reported an experiment in which ordinary people were told to give what they believed were painful electric shocks to a stranger. Many of them, he wrote, were seen to “sweat, tremble, stutter, bite their lips, groan, and dig their fingernails into their flesh.” The next year the psychologist Diana Baumrind argued that he hadn’t protected the people he studied, and the argument changed how psychologists thought about their work.
Today a study with people must pass an ethics board before it starts. In Nepal the national guidelines of the Nepal Health Research Council say that any health research with people must first be approved by an ethics committee the council recognizes, and they have a section for psychology and other studies of behavior. The rules come down to a few plain ideas.
People must give informed consent: told what will happen and what the risks are, and free to say no. In Nepal, for anyone under 18 a parent or guardian agrees in writing, and a child of seven or more must agree too. They keep the right to withdraw, to stop at any time without losing anything. They must be protected from harm, in body and in mind. What they share is kept confidential, and their privacy respected, so their names aren’t attached to their answers. Deception, keeping people from knowing what a study is really about, is allowed only when the study couldn’t be done any other way, and never when it could cause pain or severe distress. Afterwards comes debriefing: the researchers explain what the study was for, why anything was hidden, and correct any wrong ideas they left behind. Anyone who was deceived may then take their data back out.
Pfungst’s 25 people, who weren’t told what his test was for, would today be told afterwards.
So psychology finds out about minds the way Pfungst found out about Hans: by asking, watching, looking closely at one, measuring two things together and changing one thing on purpose, and counting the results instead of taking the crowd’s word. A survey is only as good as its sample and its questions, and what people say about themselves can be sincerely wrong. Watching shows what people really do, but the watchers bring expectations. A case study opens questions and can’t settle them. A correlation predicts without proving a cause. An experiment can show a cause, if chance decides the groups and nobody gives the answer away. Then someone has to do it again. Which leads to the next question: when someone did, who was in the study?
Found a mistake? Please tell us.
