The Key to Getting the Most Honest Information Is Big Data

This article is an excerpt from the Shortform book guide to "Everybody Lies" by Seth Stephens-Davidowitz. Shortform has the world's best summaries and analyses of books you should be reading.

Like this article? Sign up for a free trial here .

Why does big data offer more honest information? How can big data spark hope?

In Everybody Lies, Seth Stephens-Davidowitz says that big data offers more honest information than we’ve ever had before. This information can be found through Google search results that reveal truths about humanity.

Learn why Stephens-Davidowitz believes this information can be used for good, and why others say differently.

Honest Information

Stephens-Davidowitz points out that people tend to lie in traditional surveys. For example, people are unlikely to be honest about their sexual habits when talking to another person—even if that person is a stranger administering an anonymous survey.

Respondents will probably give more honest information in an online survey than an in-person survey—but that still doesn’t solve the problem of self-deception. Stephens-Davidowitz argues that we’re often poor judges of our own thoughts and behaviors because we don’t want to acknowledge the less savory aspects of ourselves.

Google Confessions

Google searches and other internet activity, on the other hand, reveal truths that might never come out in traditional data-gathering methods like surveys. For example, Stephens-Davidowitz shows that in states whose laws oppose gay marriage, the percentage of self-reported gay men is much lower than the estimated average across the whole population. But, he says, if you look at searches on Google and porn sites, the percentage of male users looking for gay porn (or asking how to tell if they’re gay) is much closer to that average. Also, the percentage of gay men as defined by search results is roughly stable from state to state.

This suggests that search data is a more accurate—and honest—measure of gay male sexuality than traditional surveys. Similarly, Stephens-Davidowitz says that search results reveal truths about all kinds of topics that we have an incentive to lie about or hide in real life, such as:

1) Sexuality: Stephens-Davidowitz says that, in addition to the data on gay men, search results cut against common stereotypes about sexuality. For example, women are just as likely to ask Google why their husbands or boyfriends don’t want sex as men are to ask the same about their wives and girlfriends.

2) Prejudice: Stephens-Davidowitz argues that the prevalence of searches for racist terms and phrases reveals that there is a lot more explicit prejudice (as opposed to unconscious bias or systemic inequity) than traditional surveys suggest.

3) Child Abuse: During the 2007-08 financial crisis, experts predicted a rise in child abuse and neglect only to be surprised by a downturn in cases. Stephens-Davidowitz shows that searches like “my mom beat me” went up in heavily affected areas—suggesting that abuse and neglect increased, but that cases went unreported or uninvestigated because of lessened resources.4) Abortion: Stephens-Davidowitz explains that searches about self-induced abortion are more common in states with restrictive abortion laws.

Reasons for Hope

Stephens-Davidowitz argues that although a lot of this data seems depressing, it also offers causes for hope. He gives three reasons:

First, he says that big data suggests that you’re not alone. Internet searches show that in pretty much any case, a lot more people than you think share the kinds of concerns, troubles, and interests that you might never admit to in public.

Second, he argues that big data can point out suffering that we wouldn’t otherwise notice, as with the child abuse data mentioned above.

Finally, he says big data provides feedback we can use to solve problems. He explains that during an Obama speech decrying Islamophobia, Islamophobic searches actually increased—except after a line about Muslim athletes and soldiers, which seemed to prompt curious rather than hateful searches. Stephens-Davidowitz says that Obama’s speechwriters appeared to capitalize on this data by tailoring a future speech to focus on concrete examples of Muslim Americans rather than abstract calls for tolerance.

…And Reasons for Concern

While Stephens-Davidowitz focuses on the potential benefits of the honest information people reveal online, it’s also worth pointing out how some of these truths can be exploited for more cynical purposes.

For one thing, even as big data suggests the sheer diversity of opinions, preferences, habits, and interests, it offers up that diversity as a boon to advertisers: Google Trends—the same resource behind much of Stephens-Davidowitz’s research—positions itself in part as a marketing tool that promises to help businesses craft better campaigns by revealing the topics their prospective customers search for.
Unfortunately, the targeted ad techniques that have grown out of big data can easily be used to create racially discriminatory ad campaigns—for example, by illegally excluding racial and ethnic minorities from housing ads.

In extreme cases, some journalists have shown how easy it is to create ads specifically intended for “Jew haters” on Facebook while others have demonstrated how Google’s ad platform actually helps advertisers target racist search terms by offering related searches such as “black people ruin neighborhoods.” It’s important to note that Facebook and Google disable these sorts of abuses when they come to light, but recent studies suggest that the larger problem still exists.

The Key to Getting the Most Honest Information Is Big Data