3 Dangers of Big Data You Should Be Aware Of

This article is an excerpt from the Shortform book guide to "Everybody Lies" by Seth Stephens-Davidowitz. Shortform has the world's best summaries and analyses of books you should be reading.

Like this article? Sign up for a free trial here .

What are the dangers of big data? How do businesses use data to exploit their customers?

In Everybody Lies, Seth Stephens-Davidowitz warns that big data can easily lend itself to exploitative practices by businesses and by governments. They do this by making inappropriate assumptions about individuals.

Continue reading for more details about the dangers of big data.

1. Businesses Exploit Customers

The first danger of big data is businesses exploiting customers. While A/B testing can help businesses optimize their products and services by identifying the most effective design choices, it can also help them make their offerings more addictive. He points out that from a business perspective, a site like Facebook is ultimately designed to get you to spend more time on Facebook. The more addictive they make the site and its services, the better, and A/B testing helps them find the best ways to keep users hooked.

Similarly, Stephens-Davidowitz argues that businesses can use the doppelganger method to extract the maximum profit from their customers. He gives the example of casinos, which can use data about customers like you to predict your pain point—the point at which you lose enough money that you won’t come back to the casino for a while, if at all. Once they know your pain point, they can let you lose money until you’re approaching that point, then intervene to offer you a free dinner or other perks. They come across as generous when in reality they’re manipulating you by stopping you from gambling now so that you’ll come back sooner.

(Shortform note: Some critics warn that data-based business practices might have even farther-ranging consequences than these. For example, in 21 Lessons for the 21st Century, Yuval Noah Harari argues that the increasing prevalence of computer algorithms threatens to erode our free will. He suggests that the more we learn to trust computer-generated suggestions, the less we trust our own decision-making capabilities—taken to an extreme, he argues that this trend could lead to humans entrusting their entire lives to computers and their data-driven recommendations.)

2. Minority Report

Stephens-Davidowitz warns of the temptation to use data to make inappropriate predictions about individuals. He points to studies of loan applications that identify which words are most correlated with future defaults and which are most associated with paying back the loan. He points out that it would be unfair for a lender to use this information to deny a loan in any one particular case, based only on the words an applicant used. That’s because data can only identify statistical likelihoods. This tells us, for example, that many people who “swear to God I’ll pay back this loan” default; this insight says nothing about any specific loan applicant’s likelihood of default (whether they “swear to God” or not).

Likewise, Stephens-Davidowitz says, it may be possible to correlate, for instance, a rise in searches for racist terminology in a specific neighborhood with the likelihood of racially motivated violence in that neighborhood. He argues that it may even be prudent to use this information to allocate police resources. But he rejects the idea of acting against individuals—just because somebody’s Google search suggests an interest in committing a hate crime doesn’t mean that person will actually commit any crime.

Stephens-Davidowitz acknowledges that these lines can be fuzzy: If police know that someone has been Googling where to buy guns and ammunition, how to modify those guns to make them fully automatic, and for information on a nearby school, should they intervene directly against that person? Should they inform the school? The answers aren’t clear.

3. How Governments Already Exploit Big Data

Another danger of big data is that government exploits it as well. Just as banks and insurance companies already use data to inform their decisions, governments and law enforcement already use data to surveil citizens and, in some cases, act against them. In Permanent Record, Edward Snowden details how he discovered a secret US government program for collecting data on individuals without warrants. Although this program’s public exposure was met with outrage and legal challenges, government surveillance and data collection have only expanded since Snowden blew the whistle in 2013. For example:

Data-mining company Palantir has come under criticism from human rights advocates and its own employees for maintaining a lucrative contract with the US’s Immigration and Customs Enforcement agency—which has used Palantir’s technology to detain and deport immigrants.
During the COVID-19 pandemic, the US and UK governments partnered with Palantir to track patients and map outbreaks.
With the overturning of Roe v. Wade, privacy advocates worried about the potential for the government to scrutinize pregnant women’s online activity and even track their locations to intervene against those who might seek abortions.

These real-life applications of big data bring to mind Harari’s concerns about who owns and has access to personal data. In 21 Lessons for the 21st Century, Harari argues that data in government hands poses the threat of authoritarian measures bordering on national mind control.

3 Dangers of Big Data You Should Be Aware Of