How to avoid being fooled by numbers — inspired by Daniel Levitin's A Field Guide to Lies and Statistics.
Before you check anyone's sources, sample sizes, or spreadsheets, there is a much cheaper test: how likely is this to be real at all? Plausibility is the smell test you run in your head before doing any actual maths. Most misleading statistics don't need debunking with data — they fall over the moment you ask what the world would have to look like for them to be true.
Statistics work the same way. A number can be delivered with total confidence, printed in a serious-looking report, and still describe a world that cannot exist. The first step to thinking critically should always be is it even remotely plausible.
You spot this in a newspaper. What would you make of this claim?
In the last 35 years, alcohol-related deaths in Ireland have doubled every year
A few pages later, the same paper runs a poll. Anything bothering you about this chart?
Which Newspaper do people trust most?
One more from the same paper — but careful, the test cuts both ways.
ACME has lost 95% of its value
"The average" sounds like a single, settled fact. It isn't — it's a choice between three different numbers, and the choice changes the story. When a report says average without saying which one, it almost always means the mean.
Income is the classic case. Because a handful of very high earners can pull the mean far above what a typical person makes (and unusually low values can drag it the other way), the mean is easily skewed — while the median simply shows the person in the middle. That's why honest reporting about salaries, house prices, or wealth usually quotes the median. Try it yourself:
You may have heard a statistic that is casually thrown around: that if the UK were a US state, it would rank 51st in terms of GDP per capita. This would rank the UK below a state like Mississippi, which is the poorest in the US. What is actually happening here is that the statisticians responsible for this data used the mean average rather than the median. There are a handful of very wealthy people in Mississippi, while around 18% of residents live below the poverty line.
Nine ordinary salaries — mean and median tell the same story.
There's one more way an average can mislead: when the data has two "typicals". Lunch spending in the City of London is a classic bimodal distribution — a tall spike of supermarket meal deals around £3–£5, and a second, broader bulge around £35–£40 where businesspeople are taking clients out or dining at higher-end restaurants. Any single "average" has to land somewhere between the two humps.
The "average lunch" is now £19.61. Point at the chart where those people are — there aren't any.
A chart can look precise while withholding the information needed to read it. Before drawing conclusions from its shape, check what each axis actually represents.
What is wrong with this chart?
A report uses this chart to celebrate an improvement. Does the second result really tower over the first?
Average house prices rose from £125,000 to £250,000 over 20 years. What impression does this 100-year chart create?
Probability is how we describe uncertainty without pretending it has disappeared. Levitin separates several meanings that are often bundled together: the known symmetry of a fair coin, the frequency with which something happens across many observations, and a judgement about how likely a future event seems. Trouble begins when we switch between those meanings or reverse the condition in a probability statement.
A fair coin has landed heads five times in a row. What is most likely on the sixth toss?
Suppose a report says that many road accidents happen during rush hour. That describes the probability of rush hour given that an accident happened. It does not tell us the probability of an accident given that it is rush hour.
Rush hour contains far more journeys than quieter periods, so it can contain many accidents while any individual journey remains very unlikely to end in one. Always ask which fact comes after the word given.
A headline says: "93% of women with breast cancer are in the high-risk group." What is the chance that a woman in the high-risk group has breast cancer?
Data does not have an agenda, but the person choosing which data you see may have one. A political campaign may want your vote. A company may want your money. A pressure group may want attention, donations, or action. The easiest way to move people is often to make them feel something first — fear, anger, pride, relief, or urgency — and let the number provide a coat of authority.
Two campaigns describe the same fictional labour-market report. Which headline can be true?
An investment advert gives you one impressive number. Is it enough to judge the product?