Numbers are not neutral
A statistic feels objective, and that is part of why it persuades. But every number comes from choices: what to count, whom to ask, how to summarize and how to display it. None of these choices is necessarily dishonest, yet each can change the impression the number gives. Reading statistics well mostly means asking what lies behind a figure.
All the numbers below are hypothetical and made up for illustration.
Averages can hide the picture
“Average” usually means the mean: add everything up and divide by the count. It is useful, but it can be pulled around by extreme values.
Suppose five people in a small group earn 30, 32, 35, 38 and 200 (in thousands, per year). The total is 335, so the mean is 335 divided by 5, which equals 67. Yet four of the five earn less than 40. The median, the middle value when the numbers are sorted, is 35, and it describes a typical person in this group better.
So when you see “the average,” ask which kind it is, and whether a few unusual values could be dominating it. The mean and median can also differ greatly in a skewed distribution, such as incomes or house prices, and a report that picks the more flattering one may be telling a one-sided story.
An average can also hide the spread. Two classes might both average 70 on a test, but in one nearly everyone scored near 70, and in the other half scored 40 and half scored 100. The same average, very different situations.
Samples: who was counted?
Most statistics come from a sample, a subset used to represent a larger group. Whether the conclusions apply depends on how the sample was chosen.
- Representativeness. A poll taken only among people who answer unknown phone numbers, or among visitors to one website, may not reflect the whole population.
- Self-selection. If people choose to respond, those with strong opinions are often overrepresented. A review page may show mostly very happy or very unhappy customers.
- Size. Small samples fluctuate more. If 3 out of 4 people in a tiny group like something, that is “75 percent,” but it says little about everyone else.
- Wording. The way a question is phrased can nudge answers.
A large sample does not fix a poorly chosen one. A million responses from an unrepresentative group can be less informative than a few hundred well-selected ones.
Percentages and base rates
Percentages sound precise but can hide the starting point. Imagine a hypothetical headline: “Risk doubles!” If the original risk was 1 in 10,000, doubling it makes 2 in 10,000. That is a 100 percent relative increase, but the absolute change is only 1 in 10,000. Both figures are true; they feel very different.
A good habit is to ask for the absolute numbers as well as the relative ones. “Up 50 percent” from a very small number may still be small, while “up 5 percent” of a very large number may involve many people.
Percentage points also get confused with percentages. If a rate goes from 10 percent to 12 percent, that is a rise of 2 percentage points, but a 20 percent relative increase (2 divided by 10). News stories use both, so check which one is meant.
Charts that mislead
Charts can be accurate and still give the wrong impression. Common things to look for:
- A truncated axis. If a bar chart’s vertical axis starts at 90 rather than 0, a difference between 95 and 98 can look enormous. Check where the axis begins.
- Stretched scales. Making a graph tall and narrow, or wide and flat, can make a trend look dramatic or calm.
- Cherry-picked time windows. Starting a line at a low point makes growth look steady; ending it at a peak hides the decline afterward.
- Missing context. A spike with no comparison may be normal for that time of year.
- Area and size tricks. When icons are doubled in height and width, their area grows fourfold, exaggerating the difference.
Look at labels, units and the source. If they are missing, treat the chart with extra caution.
Other common traps
- Correlation presented as cause. A relationship between two measures does not show that one produces the other, which is the heart of the difference between correlation and causation.
- Survivorship. Looking only at what remains, such as successful businesses, can leave out those that failed and produce an overly rosy view.
- Small changes called trends. Numbers fluctuate naturally. A small rise from one month to the next might be ordinary noise rather than a real shift.
- Precision that is too exact. A figure such as 37.82 percent suggests more accuracy than a rough survey can deliver. Ranges or margins of error are more honest.
Questions to ask
- Who produced this number, and what do they want you to conclude?
- What exactly was measured, and over what period?
- Who was included and who was left out?
- How big was the sample?
- Is this the mean, median or something else?
- What is the absolute number behind the percentage?
- What is the comparison, such as before, other places or other groups?
You do not need advanced mathematics to benefit from these questions. Asking them helps you treat a number as the start of a conversation rather than its end. Fields like data science exist partly to answer them with more care, but the questions are yours to use.