Think Like a Rocket Scientist: Test as You Fly

This article is an excerpt from the Shortform book guide to "Think Like a Rocket Scientist" by Ozan Varol. Shortform has the world's best summaries and analyses of books you should be reading.

Like this article? Sign up for a free trial here .

What does the book Think Like a Rocket Scientist mean by “test as you fly”? How might you use this principle in your life?

In Think Like a Rocket Scientist, Ozan Varol explains how rocket scientists test equipment in space-like conditions. Similarly, successful people test ideas and products. Varol shares three criteria for successful tests and two suggestions for testing your ideas rigorously.

Keep reading to learn how to think like a rocket scientist and apply the test-as-you-fly principle to your own life, career, or business.

Think Like a Rocket Scientist

Before pursuing a new idea for achieving the seemingly impossible, successful people and companies conduct tests to ensure the idea will work as planned. According to Think Like a Rocket Scientist, successful tests must:

Have real stakes. If the idea fails your test, you need to be prepared to kill the idea rather than coming up with excuses to carry on anyway.
Take place under the same circumstances in which the full version of the idea will occur (not a laboratory) whenever possible.
Test the whole system, not just individual parts. Remember that each individual part might work well, but that doesn’t guarantee they’ll work well together.

(Shortform note: It might not always be possible to conduct tests that meet all of these criteria. For example, new medical products can’t be tested on humans until they’re proven to meet minimum safety criteria. Until that point, they must be tested in other ways, with lower stakes and less realistic context.)

How Rocket Scientists Test Ideas

Rocket scientists test equipment in space-like conditions before launching it into space. According to Varol, their goal is to expose the equipment to as much stress as possible in order to find the breaking points. That way, they can fix those problems from the safety of Earth rather than waiting for them to develop during a space mission. (Shortform note: This happened in January 2021, nine months after Think Like a Rocket Scientist was published. NASA tested their new deep-space rocket’s engines, which were supposed to run for eight minutes but shut down after just one minute on the launchpad. While some politicians interpreted this as a sign of NASA’s decline, those inside the organization said the test gave them helpful information.)

How You Can Test Your Ideas Thoroughly

Here are Varol’s suggestions for testing your ideas rigorously:

1) Conduct tests under real-life conditions. Don’t assume that you know how the tests will turn out, or that something like a survey (in which you ask people to imagine a certain outcome and predict how they’ll react) is actually representative of real life. Such tests can fall victim to the observer effect, or the fact that merely observing a situation tends to impact the outcome—in the case of an in-person survey, the results may be skewed because people modify their behavior when they know they’re being watched.

2) Don’t rely on one single tester. When you test something, Varol believes you either need multiple testers or you need a way to test the tester to ensure you can trust the results. For example, if you’re trying to lose weight but the numbers aren’t budging, it could be that your exercise plan isn’t working—or it could be that your scale is broken. Only by weighing yourself on a different scale will you be able to tell where the problem is.

(Shortform note: In The Design of Everyday Things, author Don Norman argues that testing should be an iterative process, not a one-time thing. In other words, test an idea with a small group of people (Norman recommends five to start), make any necessary adjustments, then test again with another small group. That way, you can run your idea by lots of people without having to gather a huge group all at once.)

How to Test Like Pixar

While testing under real-world conditions is ideal, there are tradeoffs to this type of testing—namely, it doesn’t allow you to control as many variables as laboratory testing. To strike a balance between real-world and controlled laboratory testing, you can create a miniature version of your ideal real-world conditions. You still get the benefits of real-world testing, but since the situation is only on a small scale, you have a better chance of controlling variables.

In Creativity, Inc., Ed Catmull describes how Pixar uses its iconic short films as testing opportunities for new directors or animation techniques they’re considering using in feature films. The process of making a short film is similar enough to the process of making a feature film to be realistic but is more condensed and requires fewer resources. This gives the benefits of both real-world and laboratory-controlled testing.

When you think like a rocket scientist and apply the test-as-you-fly principle, you’re more likely to make your own moonshots.

Think Like a Rocket Scientist: Test as You Fly