This article is an excerpt from the Shortform book guide to "Bad Science" by Ben Goldacre. Shortform has the world's best summaries and analyses of books you should be reading.
Like this article? Sign up for a free trial here.
What are scientific controls? Why are they instrumental in experiments?
In Bad Science, Ben Goldacre explains that scientific studies are designed to test one treatment at a time, without any influence from other factors. To do this, good researchers make use of scientific controls.
Let’s go into more detail on a scientific control’s role in studies.
Scientific Controls Shape Results
What are scientific controls and why are important? Scientific controls are elements of a study’s design that are put in place to prevent specific factors from influencing results. By contrast, some researchers use weak experimental controls to manipulate their results.
(Shortform note: The practice of running experiments with controlled variables is called the scientific method. Scientists believe that the scientific method is what distinguishes scientific knowledge from other types of knowledge, such as the knowledge contained in philosophical treatises or religious texts. This doesn’t mean that these other kinds aren’t useful and meaningful. It simply means that only knowledge produced in controlled experiments can be considered scientific.)
All scientific experiments involve a control subject and a test subject. Ideally, your control and test subjects should be identical in every way except for the variable you’d like to test—in the medical field, this variable is usually a medical treatment.
Let’s demonstrate this with an example. Suppose you’re designing an experiment to test the effect of direct sunlight on pea plants. It wouldn’t be enough to simply put a pea plant in the sun and observe it, because you’d have nothing to compare it to. To really see how the sun affects plants, you’ll need at least one other plant, kept in a relatively dark location. Then, measuring against your control plant (the one kept in darkness), you’ll be able to evaluate the growth of your test plant (the one kept in the sun).
Of course, in the real world it isn’t possible to completely control variables. Researchers make decisions based on practicality, funding, ethics, and other factors. (For example, in our pea plant experiment, the different rooms in your house might be at slightly different temperature and humidity levels. Unless you were willing to spend lots of money remodeling your house, you’d be limited by money and practicality, just like a real researcher.)
(Shortform note: Experts note that one reason medical studies aren’t able to completely control variables is that health care and human bodies are both complex systems. Because these systems include so many variables, small differences in treatment as well as in patients’ bodies can lead to different medical outcomes. Even though many of these variables are difficult to spot, they can still have a huge impact on a study’s results.)
Spotting Bad Experimental Controls
Because controlling variables can be a complex process that includes compromise, you should carefully evaluate the studies you read. Studies that show a high level of detail and thorough commitment to controlling as many variables as possible are likely to be trustworthy. The best studies are aware of their own flaws and may even go so far as to suggest future methods that would improve those flaws.
By contrast, if a study doesn’t spend much time discussing how variables were controlled, Goldacre says you should take the study with a grain of salt. Its findings might be meaningful, but they might just as easily be the result of a variable the researchers conveniently left out.
(Shortform note: Lack of experimental controls isn’t always a bad thing. As experts describe, some studies don’t include experimental control because of the natural limitations of their experiments. For instance, in a study on the longevity of smokers as compared to nonsmokers, it’d be impossible to control aspects of the participants’ lives such as diet and exercise. However, researchers could still generate useful data from observing and comparing the two groups.)
———End of Preview———
Like what you just read? Read the rest of the world's best book summary and analysis of Ben Goldacre's "Bad Science" at Shortform.
Here's what you'll find in our full Bad Science summary:
- The strategies researchers, corporations, and journalists use to mislead the public
- The tools you need to identify and call out shady science when you see it
- Why media outlets have an ethical obligation to publish retractions
