Misinterpreting Data: 6 Ways Bullshitters Try to Fool You

Can you distinguish causation from correlation? Do you notice when numbers have been taken out of context?

Unfortunately, some people manipulate data to mislead others. In their book Calling Bullshit, Carl T. Bergstrom and Jevin D. West argue that bullshitters make invalid inferences from valid data or visually misrepresent valid data in graphic form.

Keep reading to learn how bullshitters purposely misinterpret data to draw unjustified conclusions.

Shoddy Inferences From Sound Data

Bergstrom and West write that bullshitters employ various strategies to infer unsound conclusions from sound data. We’ll examine three such strategies for misinterpreting data: inferring causation from correlation, discussing numbers out of context, and presenting misleading percentages.

Strategy #1: Causation From Correlation

Humans naturally seek out causal narratives, write Bergstrom and West. For this reason, we often judge that one phenomenon causes another, even when our evidence only suggests the two phenomena are correlated. For example, it’s well known that increasing ice cream sales are positively correlated with increasing drowning rates. However, it’d be foolish to conclude that ice cream is causing the uptick in drownings—according to a more likely explanation, ice cream sales increase in the summer, and drownings also increase in the summer because more people swim when the weather is warmer.

Consequently, bullshit often arises when people make causal conclusions based on mere correlation. This bullshit is especially pervasive in mainstream news outlets, with one analysis finding that almost half of all news articles that report on research studies identifying correlations between two variables framed these studies as having established a causal link.

(Shortform note: In Thinking, Fast and Slow, Nobel laureate Daniel Kahneman argues that our propensity for inferring causation from correlation is a byproduct of system 1 thinking—our mind’s intuitive process of thinking, which is done without conscious effort. System 1 thinking, he clarifies, seeks to quickly make sense of the world in terms of coherent narratives, making it prone to assume causation between two related events. By contrast, system 2 thinking—our mind’s slower, more deliberate process of thinking—more carefully distinguishes between causation and correlation, making it helpful for fact-checking the causal narratives that system 1 hastily embraces.)

Strategy #2: Numbers Without Context

In a similar vein, Bergstrom and West contend that taking numbers out of context can lead to bullshit. For example, if a politician proudly announces that his economic plan has increased the average citizen’s income by $10,000, the economic plan could seem beneficial for the average citizen. However, if this plan only increased the income of the wealthiest 1% of citizens, thus inflating the average, then it may be less universally beneficial than it appears. Thus, by presenting the average income increase without the context of how this increase was distributed across the population, the politician can promote a false view of his economic plan.

(Shortform note: Statistics are regularly taken out of context in advertisements, where, without being strictly inaccurate, they can mislead consumers into purchasing a product. For example, in 2007 Colgate toothpaste advertised the claim that “80% of dentists recommend Colgate,” making it seem as if 80% of dentists recommended it over other brands. That implication was false: The 80% figure stemmed from a survey that allowed dentists to “recommend” as many brands as they wanted to.)

Strategy #3: Misleading Percentages

Further, Bergstrom and West explain that using percentages can obscure true values and promote bullshit. For instance, a tobacco company might point out that in 2022, of 330 million Americans, only 0.2% died of cancer. While technically correct, this percentage obscures the fact that cancer accounted for almost 20% of deaths among Americans in that same year, as it caused 600,000 deaths out of 3.2 million. Percentages can therefore be a powerful vehicle for disguising bullshit.

(Shortform note: Misleading percentages were especially common during the Covid-19 pandemic, in which those skeptical about Covid-19’s severity often claimed that 99% of patients with Covid-19 survived. While this claim implied that concerns about Covid-19 were vastly overblown, it obscures the fact that a 1% mortality rate is remarkably high for a virus—nearly 10 times higher than the seasonal flu, which has around a 0.1% mortality rate. Further, it obscures the fact that, because Covid-19 was sufficiently widespread, it still caused over one million deaths in the US.)

Misleading Data Visualizations

According to Bergstrom and West, bullshitters can further manipulate valid data by using strategies to create misleading representations. Although Bergstrom and West list an array of strategies for misleading viewers, we’ll focus on three particularly illuminating ones: prioritizing form over function, shoehorning data in suboptimal visualizations, and using disproportionate visualizations.

Strategy #1: Prioritizing Form Over Function

Bergstrom and West argue that prioritizing form over function can create bullshit that obscures the data that a visualization attempts to convey. For instance, imagine that you wanted to represent the proportion of Americans associated with a specific religious affiliation in graphical form. If, instead of representing this information via a simple bar chart, you chose to represent it via a shaded map of the United States, that shaded map would be neglecting function for the sake of form. After all, although the shaded map of the United States might look cooler, it would also be more difficult to interpret.

(Shortform note: Although graphs that prioritize form to the detriment of function are harmful, that doesn’t mean form should be given no consideration whatsoever. On the contrary, one team of researchers found that, when interpreting certain aesthetically appealing line graphs, viewers could extract the relevant information just as easily as they could on more austere line graphs. Moreover, these viewers rated the aesthetic graphs as more visually pleasing than the austere graphs, meaning they were more attractive without compromising functionality.)

Strategy #2: Shoehorning Data Into Inappropriate Visualizations

While an excessive focus on form can be innocuous, Bergstrom and West contend that shoehorning data into inappropriate forms is an unequivocal attempt to create bullshit. The clearest example of this is the proliferation of visual representations that parallel the periodic table; for instance, there are periodic tables of marketing techniques, cryptocurrencies, and typefaces. But, while the original periodic table was carefully designed to convey the chemical groupings of different elements, these mock periodic tables are not. Rather, they attempt to convey an appearance of rigor which their underlying data lack.

(Shortform note: Despite Bergstrom and West’s insistence that these faux periodic tables lend fake credence to unrigorous data, some of these tables have even slipped into formal academic research. For example, the periodic table of cryptocurrencies—an attempt to categorize various cryptocurrencies that exist on the blockchain—was first published in an academic paper.)

Strategy #3: Using Disproportionate Visualizations

Finally, the authors argue that certain data visualizations spread bullshit by violating statistician Edward Tufte’s requirement of proportionate visualizations. In his 1983 book, The Visual Display of Quantitative Information, Tufte argues that the size of regions that represent data should be proportionate to the values they represent. For instance, if you’re creating a bar chart that illustrates the number of Nobel Prize winners by country, then the bar representing Germany’s 111 Nobel laureates should be just over five times as large as the bar representing Austria’s 22 Nobel laureates.

(Shortform note: In his book, Tufte’s principle of proportionate visualization is one in a series of foundational principles for accurately representing data. Counterintuitively, another such principle is that tables are superior to graphs when dealing with smaller data sets since graphs can easily be manipulated whereas tables provide the data in a neutral manner.)

Bergstrom and West point out that, although the principle of proportionate visualization seems commonsensical, violations of it are common—any time a bar chart’s vertical axis doesn’t start at zero, it will violate this principle. Such charts can convey bullshit because they misrepresent the scale of the difference between two values, providing a ripe ground for false inferences. For example, imagine we created a bar chart measuring Nobel laureates by country that was evenly spaced but started at 20 rather than zero. Then, Germany’s bar would be 91 units tall (111 minus 20) while Austria’s would be 2 units tall (22 minus 2), creating a misleading picture of the difference between the two countries.

(Shortform note: Although bar charts must start at zero, the same principle doesn’t necessarily apply to line graphs. According to experts, line graphs that start at zero can obscure significant trends when the changes in data are, in some sense, small. For example, if we plotted the earth’s average surface temperature between 1900 and 2022—which has risen from about 57°F to 59°F—the increase would hardly be discernible if our line graph started at zero. Consequently, starting our line graph at zero would obscure an important trend in global temperature, since climate scientists consider the two-degree increase to illustrate the dangerous long-term trend of global warming.)

Misinterpreting Data: 6 Ways Bullshitters Try to Fool You