Suppose you are in charge of your firm’s commuting policy and, after some research, you determine that 57 percent of your colleagues drive to work. How would you communicate that number to co-workers?
You could just tell them the number. Or you could characterize it in words. If you want to imply that 57 percent is a large number, you could call it “a clear majority,” or even “approaching two-thirds” (roughly 67 percent). However, if you seek to minimize 57 percent, you might call it “barely over half.”
Either way, when people are motivated to persuade their audience that a number is particularly high or low, they are considerably more likely to use language to describe the data, rather than simply airing the original numbers.
Now a new study co-authored by an MIT scholar quantifies this phenomenon: People motivated to persude others about statistical facts are 25 percent to 29 percent more likely to use words describing the numbers in question, instead of the numbers themselves. That is a large enough increase to be termed “substantial,” “significant,” or any number of other words.
“If I’m a well-informed expert and I have incentives to persuade my audience, I’m relatively more likely to choose to use language,” says co-author Victor de Chaisemartin, a doctoral candidate at the MIT Sloan School of Management.
The language that experts choose when seeking to persuade others is also more slanted than their numeric communications. As the authors note in the paper, people appear to be leveraging the imprecision of language in ways they feel are palatable: Whereas reporting a number incorrectly would be precisely false, it may be more psychologically comfortable to slant a message using language.
And that’s just the half of it. The study also shows that audiences find language-based descriptions of numbers convincing — even when they are told there is an agenda behind them.
“Language messages are particularly persuasive,” de Chaisemartin says. Telling an audience they are being spun, he notes, “actually doesn’t do very much to mitigate this. That’s interesting. It makes you wonder what’s going through people’s heads, and how they are dealing with information.”
The paper, “Numbers Tell, Words Sell,” is being published in the American Economic Review. The authors are de Chaisemartin; Michael Thaler, a lecturer in economics at University College London; and Mattie Toma, an associate professor at the University of Warwick.
Floating in data, reporting it in words
As de Chaisemartin describes it, the study’s origins come from a basic observation.
“The world has loads of statistics floating around in it,” he says. “But often if you pick up a newspaper or read something in the media, or if you’re reading a corporate report, you don’t usually see 50 statistics. You’re often going to see a lot of quantitiave data reported in words.”
Indeed, one study of media practices has found that 22 percent of news items include statistics, but that 24 percent of those stories only use words to describe the numbers. Another study of academic research has found that just 37 percent of papers include numbers in their abstracts.
To conduct the study, the researchers conducted two separate but parallel experiments. In the first one, using the research platform Prolific, 1,000 members of the general public were divided into 500 pairs of senders and receivers of information, communicating the results of simple games and observations. In the second experiment, set up through the University of Warwick’s Policymakers Lab, 146 scholars from academia were paired with policymakers from five countries — primarily central government officials.
Ultimately, the experiments show that when incentivized for accuracy, senders of information choose language over numbers only 13 to 14 percent of the time; that is, experts generally prefer numbers for communicating accurately. But faced with incentives to persuade, the likelihood of senders choosing language increases by 25 to 29 percentage points.
“It’s a clear-cut hypotheses, it makes sense, and the results bear it out,” de Chaisemartin says.
Applying the findings
While the study is specific in nature, it contains some broader implications. For one thing, de Chaisemartin notes, the findings can help organizations recognize the value of constructing consistent usage standards. Climate scientists, he notes, do this already regarding some forecasts. Security officials, medical researchers, doctors, and scholars may also have good reasons to adopt similar characterizations of likelihoods and other things.
“There are ways to think carefully about how we set up and report empirical data,” he says. “This could be viewed as one piece of information pushing in that direction.”
De Chaisemartin also suggests that the findings bear on the issue of media literacy generally, and could be used to help people analyze the statistical claims they read in the news. For his part, he is also interested in follow-up questions about what happens when people send information to the same audience repeatedly.
“That might introduce interesting new mechanisms in this process,” de Chaisemartin says, noting that senders of information might have additional incentives to protect their reputations for accuracy, while audiences could become more discerning over time.
To be sure, there are many subtleties involved in descriptions of statistics. For example, given that most drivers wear seatbelts, a policy intervention raising the usage rate by two percentage points might not seem huge in absolute terms, but could represent a substantial increase within the remaining room for improvement. In other circumstances, a change of two percentage points might just be a small fluctuation in a sea of data.
Language descriptions could therefore help contextualize numbers or aid audiences in understanding statistics. However, the present findings suggest caution when interpreting language, since it may also signal that the expert was trying to persuade their audience.
In any event, de Chaisemartin expresses hope that public discourse involving statistics can become consistently healthy and accurate.
“There’s a space for language and numbers to be complementary, and to provide people with both,” he says.
The research was supported, in part, by the British Academy/Leverhulme Small Research Grants, and the University of Warwick Behaviour Spotlight Small Grant Scheme and Primary Data Funds.