Misleading Statistics: Examples of Techniques Used

This article is an excerpt from the Shortform book guide to "How to Lie With Statistics" by Darrell Huff. Shortform has the world's best summaries and analyses of books you should be reading.

Like this article? Sign up for a free trial here .

What are the top five techniques used by liars to mislead the public and make statistics look better in their favor? How are these techniques used in practice?

There are many techniques that anybody with an agenda can use to manipulate statistics in their favor. In How to Lie With Statistics, Darrell Huff explains how liars can fudge the numbers and he gives examples of misleading statistics so you can learn how to avoid being manipulated.

Keep reading for misleading statistics examples and techniques that liars use.

Misleading With Statistics

Liars can pull or imply favorable numbers from existing data, without even having to change anything about the sample. Here are five techniques for fudging the numbers with misleading statistics examples:

Technique #1: Citing Misleading “Averages”

The first technique is using the word “average” without specifying what kind of average a figure represents. Each kind is calculated differently and gives different information (and a different impression) about the data:

Average Type #1: Mean. This number is the result of adding up all the sample’s numbers and then dividing by the number of samples.

(Shortform example: To get the mean income of five people, you’d add up all their incomes and divide by five: ($30,000+$50,000+$70,000+$70,000+$70,000)/5=$58,000)

This is a useful average for liars to use because it allows them to:

Make the number look bigger and better. (Shortform example: If a university wants to attract students, the larger the average income of its graduates, the more attractive it looks to prospective students. Even if there are just a few high salaries, the math will make the mean look higher than any of the other averages.)
Hide inequality. (Shortform example: If 90 employees at a company are paid $20,000 a year and the boss is paid $200,000, the mean is ((90*20,000)+(1*200,000))/91=21,978. The mean doesn’t show that one person is paid a lot more than everyone else.)

In turn, hiding that they’re using the mean, by simply using the more general “average” to describe the figure, benefits liars by obscuring the fact that they’re using such an unreliable calculation.

Average Type #2: Median. This is the number that falls in the middle when the sample numbers are arranged in numerical order. The median is a useful number for you as a seeker of the truth because it gives you information about the data distribution—half of the sample numbers are above the median and half are below.

(Shortform example: The median of the five people who make $30,000, $50,000, $70,000, $80,000,and $80,000 is $70,000.)

Average Type #3: Mode. This is the number that appears most frequently in a data set.

(Shortform example: The mode of the above list is $80,000.)

When the distribution of a data set is normal (most of the values fall in the middle, with just a few on the extremes), all of the averages will be similar. However, when the distribution isn’t normal, the averages can be wildly different. In this case, nefarious people can pick the number that suits them best and simply label it the average.

Technique #2: Giving Precise Figures to Appear More Reputable

Another number-fudging technique is to include a decimal to make a figure look more precise and therefore reputable. (For example, reading that most people sleep 7.84 hours a night sounds a lot more impressive than “about eight hours.”)

Liars can get decimals by doing calculations (for example, calculating the mean) on inexact figures that weren’t measured to the decimal point.

(Shortform example: If you ask 100 people how much they spent on groceries in the last month, they probably won’t remember exactly. Even if they give you round, approximate numbers, if you calculate the mean, you’ll likely end up with a decimal. For instance, (20+30+60)/3=36.66666… This number is meaninglessly more precise than the measures you started with, but it looks good.)

Technique #3: Using Percentages to Hide Numbers and Calculations

Like decimals, giving percentages instead of raw figures can make numbers look more precise and reputable than they really are. (Shortform example: If two out of three people prefer a certain cleaning product, this can be expressed as 33.333…%. The decimal adds precision and implies reputability.)

Here are some additional ways liars manipulate percentages and their associated terms for their gain:

1. Hiding raw numbers and small sample sizes. Percentages don’t give any indication of the absolute value of raw figures, so liars can use them to mask unfavorable numbers or suspiciously small sample sizes.

(Shortform example #1: If a stock was worth $1 yesterday and $2 today, that’s a 100% increase, which looks impressive. However, the actual difference is only $1, which is less impressive.)
(Shortform example #2: If one person uses cold medicine and is cured, the liar can cite the medicine’s success rate as 100%.)

2. Using different bases. Because percentages don’t give any indication of the raw figures (bases) used to calculate them, liars can compare percentages calculated off different bases to distort their results.

For example, The New York Times once reported that after taking a 20% pay cut last year, union workers got a 5% raise the next year, which gave them back one-fourth of their cut wage. This claim of it being one-fourth of their cut wage refers to 5% being one-fourth of 20%. However, the workers didn’t actually get 5% of their original wage back, they got 5% of their new, lower wage back, which is a smaller number. The 20% cut and the 5% increase were calculated off different bases, so weren’t directly comparable.

Liars can also combine percentages and averages while manipulating bases to mask the real data even more. For example, if milk has gone down from $2 a pint to $1, but bread has gone up from $1 to $2, liars can massage percentage math and choose different bases to prove the cost of living has gone up or down, depending on their agenda. To show costs went up, they can decide that last year’s prices were the base (100%). Milk’s price has halved (50%) and bread’s price has doubled (200%). The average of 50% and 200% is 125%, so prices have increased by 25% since last year.

To show costs went down, they can decide that this year is the base year (100%). With this base, milk used to cost 200% more and bread cost 50% less—you get the same average of 125%, but since the base is different, it shows a decrease of 25% since last year.

3. Adding up percentages. Percentages aren’t numbers—you can’t meaningfully add or subtract them.

For example, imagine you buy 20 vegetables at the grocery store and all of them cost you 5% more than they did last year. If you add together all of those 5% increases, you get a 100% increase (20*5%=100%). This could be reported as “the cost of living has gone up by 100%.” But in reality, it hasn’t—it’s gone up by 5%, and all products were affected.

4. Giving percentage points instead of percentages to confuse people. Percentage points are the difference between two percentages. For instance, the difference between 5% and 7% is two percentage points. If a liar doesn’t want to report how much money her company made, and her return on investment was 3% last year and 6% this year, she might say “return on investment rose three percentage points.” A three-point increase sounds much smaller than a doubling, even though they mean the same thing in this case.

Technique #4: Using the Most Favorable Form

The fourth technique is to report numbers in whatever form most exaggerates or minimizes them; whichever will further a liar’s agenda. For example, return on sales, return on investment, and increase or decrease in profits are all ways of reporting how much money a company made. Most people won’t realize that each type of measure tells only part of the story. For example, if you buy a stock every morning for $99 and sell it in the afternoon for $100, you’re making only 1% on total sales, which doesn’t sound like a great return. However, over 30 days, you’re making 30% on total money invested—a much better-sounding prospect.

Example #1: If a liar thinks it will look bad to report a high, raw profit value (perhaps because then employees will demand raises), she might report the return on sales instead if it’s lower.
(Shortform example #2: If 60 out of 90 people survive an operation, a liar who wants to discourage people from having the operation might choose to report this as “one-third of people who undergo the operation die,” rather than the equally accurate but survival-focused “two-thirds of people live.”)

Technique #5: Omitting Statistical Qualifiers

The last way to fudge numbers is to leave out information that puts caveats on their accuracy or further explains them. There are four types of information liars often neglect to include with their figures:

1. Probable error. Probable error is a measure of how reliable a figure is, expressed as a range that the true result will fall between. (It’s impossible to find the single number that represents the true result because measuring systems aren’t perfectly accurate.) If you’re presented with a single figure, and aren’t given any indication of how reliable it is or what the probable error is, it may not be accurate at all.

For example, if an IQ test has a probable error of 3 and you score 98, this means that your IQ is somewhere between 95-101 (98-3=95, and 98+3=101). The real number is equally likely to be any number in that range. So, simply telling someone that your IQ is 98 isn’t accurate.

2. Degree of significance. The degree of significance is a measure of how likely it is that results are due to chance. In most cases, for a figure to be statistically significant, the degree needs to be no more than 5%—this means that 95 out of 100 times, the results are real and not attributable to chance. If the degree isn’t given, it may be higher than 5%, which means the results could be due more to chance than anything else.

3. What the comparison is to. Some stats promise to “triple” the effectiveness of a product, or offer “25% more,” but don’t say what they’re compared against. A granola bar that contains 25% more protein than a competitor’s, versus a bar that contains 25% more protein than a rock, are two entirely different things.

4. Negligibility. While there may be mathematical differences between figures, sometimes, these differences are so small they don’t make any practical difference—but liars fail to point this out.

For example, the editor of Reader’s Digest solicited a study of cigarette smoke ingredients, and a lab produced a list of what ingredients made up the smoke from different cigarettes. All of the cigarettes were poisonous, but one, Old Gold, had slightly smaller quantities of poisons than the others. Technically, it was true that smoke from Old Gold cigarettes had fewer poisons (and Old Gold used this data, minus raw figures, to advertise), but the difference was negligible—smoking any cigarette was equally as unhealthy.

Misleading Statistics: Examples of Techniques Used