Why Did a Spacecraft Crash on Mars?

Use the widget above. The file the navigators get holds each push in lbf s, pound-force seconds, as it did in 1999.

• a.Fly to Mars month by month. Correct the course whenever the dashed curve has dropped below 226 km. What happens at Mars?
• b.Press Start again, set Real path to Shown, and fly again. Watch the gap between the two curves. When does it grow fastest?
• c.Now switch the file to N s, newton-seconds, the unit the two teams had agreed on. Fly again, correcting as before.

The real navigators did their job too. They corrected the course several times on the way, the last time on 15 September, eight days before Mars, aiming for 226 km. Every correction was worked out carefully, from the wrong picture.

A push and its unit

When a rocket engine fires, it pushes. How hard it pushes is a force, measured in newtons (N): a 100 g packet of biscuits presses on your hand with about 1 N. How much a push changes a spacecraft’s speed depends on how hard it pushes and for how long, so engineers multiply the two: force × time, in newton-seconds (N s). A push of 2 N for 3 seconds is 6 N s. This is called the impulse, and below we just call it the push.

The United States also uses an older unit of force, the pound-force (lbf): the weight of a 1-pound mass. One pound-force is about 4.45 newtons. So a push of 1 pound-force second (lbf s) is the same as about 4.45 N s. The same push can be written as two very different numbers:

A number on its own says nothing. 1 could mean 1 lbf s or 1 N s, and one is 4.45 times the other.

Why the spacecraft needed small pushes

To point its camera and antenna the right way, the Mars Climate Orbiter turned itself with reaction wheels: flywheels inside it. Spin a wheel one way and the spacecraft turns the other way, without using any fuel. But sunlight pushes, very gently, on whatever it shines on, and the orbiter’s big solar panel stuck out on one side. The sunlight kept twisting the spacecraft, and to hold it steady the wheels had to spin faster and faster.

Every so often the wheels were slowed while small thrusters fired to hold the spacecraft still: angular momentum desaturations, AMDs. Each also gave the spacecraft a tiny shove, so the navigators had to know every one. The report into the loss found that, because of the one-sided panel, these firings happened 10 to 14 times more often than the navigators had expected.

Two teams, one file

The spacecraft was built, and flown day to day, by Lockheed Martin Astronautics in Denver, Colorado. It was navigated by NASA’s Jet Propulsion Laboratory (JPL) in California. After each firing, a program on the ground at the spacecraft team, called SM_FORCES (“small forces”), worked out its push and wrote it into a file, the AMD file. The navigators’ program read the file and worked out how each push changed the path.

A written agreement between the two, the Software Interface Specification, said exactly what the file must hold, and that the pushes must be in newton-seconds. SM_FORCES wrote them in pound-force seconds. The navigators’ program read every one as newton-seconds, and so it thought every push was 4.45 times smaller than it really was. The real path sank toward Mars; the path on the navigators’ screens sank much less.

That is the whole bug. No line in either program is wrong; the two together were.

Why nobody saw it in time

The navigators did see that something was odd. For the first four months the AMD files had other format mistakes and couldn’t be used, so they worked from emails saying when firings happened. From April 1999 they used the files, and within about a week the numbers looked odd. Different ways of working out the path disagreed (those using only the Doppler measurements, below, kept putting it closer to Mars), but this was only reported informally, and the cause wasn’t found.

Part of the reason is how a spacecraft is tracked. Radio dishes on Earth can measure very exactly how fast it moves toward or away from Earth, from how its radio signal’s pitch changes (the Doppler shift, the same thing you hear when a siren passes). The firings pushed the spacecraft mostly sideways to that line, so the tracking hardly saw them.

In the last week, the navigators’ estimates of the pass began to drop: from 226 km to 150–170 km, and an hour before arrival, as low as 110 km. The board said the spacecraft could not survive a pass below 80 km. A last correction, called TCM-5, was talked about but not made. After the loss, the navigators worked the path out again with the right units. It had passed about 57 km above Mars.

With blocks

Here are the two teams’ parts as one small block program. Team 1’s part works out the push: a made-up thrust of 2 pound-force, for as many seconds as you type, so its push is in pound-force seconds. Team 2’s part says the push in newton-seconds, as the agreement says. Every block is right for its own team.

Did you find it? set push to (push × 4.45) turns pound-force seconds into newton-seconds: one block, once someone saw the two parts used different units.

Step 1: two programs, each right

Now the same in Python, with the real spacecraft’s mass (629 kg at launch) and made-up firings (the report doesn’t give their sizes). Team 1’s small_forces gives a firing’s push in pound-force seconds. Team 2’s drift_km takes a push in newton-seconds and works out how far it moves the pass by the time the spacecraft reaches Mars: push ÷ mass is the change of speed (m/s), and that × the time left is how far off it ends up (simplified: we pretend every push moves the pass straight down).

MASS = 629   # kg, the spacecraft at launch # Team 1, the spacecraft team: the push of one firingdef small_forces(thrust, seconds):    # thrust in pound-force, so the push is in pound-force seconds    return thrust * seconds # Team 2, the navigators: how far a push moves the pass at Marsdef drift_km(push, days_left):    # push in newton-seconds, as the agreement says    speed_change = push / MASS                   # m/s    return speed_change * days_left * 86400 / 1000 # each team tests its own part, and both tests are rightprint(small_forces(0.0888, 0.5), 'pound-force seconds')print(round(drift_km(1, 100), 2), 'km for 1 newton-second, 100 days out') # joined, as in 1999push = small_forces(0.0888, 0.5)print(round(drift_km(push, 100), 2), 'km, as the navigators worked it out')print(round(drift_km(push * 4.45, 100), 2), 'km, what the push really did')

0.0444 pound-force seconds13.74 km for 1 newton-second, 100 days out0.61 km, as the navigators worked it out2.71 km, what the push really did

A day has 86,400 seconds, and ÷ 1000 turns meters into kilometers. Both functions pass their own tests. Joined, one firing 100 days out moves the pass 2.71 km, and the navigators think 0.61 km.

Step 2: nine months of nudges

The journey took 286 days, from 11 December 1998 to 23 September 1999. Suppose (made up) a firing every 5 days, each like the one above. The loop adds up how far all of them move the pass, as the navigators think and as it really is, and prints the totals every 60 days.

MASS = 629 def small_forces(thrust, seconds):    return thrust * seconds def drift_km(push, days_left):    speed_change = push / MASS    return speed_change * days_left * 86400 / 1000 think = 0really = 0for day in range(5, 286, 5):    push = small_forces(0.0888, 0.5)    think = think + drift_km(push, 286 - day)    really = really + drift_km(push * 4.44822, 286 - day)    if day % 60 == 0:        print('Day', day, ': the navigators think', round(think), 'km, really', round(really), 'km') # the navigators corrected the course so that they expected 226 km,# so the real pass was lower by everything they didn't know aboutprint('Navigators expect', 226, 'km above Mars')print('Really', round(226 - (really - think)), 'km above Mars')

Day 60 : the navigators think 19 km, really 83 kmDay 120 : the navigators think 33 km, really 146 kmDay 180 : the navigators think 42 km, really 189 kmDay 240 : the navigators think 48 km, really 213 kmNavigators expect 226 km above MarsReally 57 km above Mars

(4.44822 is a closer value of the newtons in a pound-force; it is exactly 4.4482216152605.) The widget at the top runs this same arithmetic. With these made-up firings the pass comes out at 57 km, as the board’s own estimate did. Notice the totals: the first 60 days move the pass more than the last 100, because an early nudge has the most days left to grow.

Step 3: find the bug

Here is the join on its own, with line numbers.

push = small_forces(0.0888, 0.5)miss = drift_km(push, 100)print(miss)

Step 4: put the unit in the name

A simple habit makes this kind of bug visible: put the unit in the name of every variable and parameter that holds a measurement. Python also lets you name each value as you pass it, drift_km(push_n_s=…): a keyword argument, which works just like passing values in order, but says which is which.

MASS_KG = 629NEWTONS_PER_POUND_FORCE = 4.44822 def small_forces_lbf_s(thrust_lbf, seconds):    return thrust_lbf * seconds def drift_km(push_n_s, days_left):    speed_change_m_s = push_n_s / MASS_KG    return speed_change_m_s * days_left * 86400 / 1000 push_lbf_s = small_forces_lbf_s(0.0888, 0.5)print(round(drift_km(push_n_s=push_lbf_s, days_left=100), 2))    # looks wrong! push_n_s = push_lbf_s * NEWTONS_PER_POUND_FORCEprint(round(drift_km(push_n_s=push_n_s, days_left=100), 2))      # looks right

0.612.71

The first line still gives a wrong answer: Python doesn’t understand names. But a person reading push_n_s=push_lbf_s sees at once a pound-force number going where a newton number should.

Step 5: let the number carry its unit

Better still, make the program check. Here every push is a list of two things, the number and its unit: [0.0444, 'lbf s']. drift_km looks at the unit before using the number, and if it isn’t newton-seconds it stops the program with an error of its own: raise ValueError(...) stops the function and says why.

MASS = 629 def small_forces(thrust, seconds):    return [thrust * seconds, 'lbf s'] def drift_km(push, days_left):    if push[1] != 'N s':        raise ValueError('drift_km needs newton-seconds, but got ' + push[1])    speed_change = push[0] / MASS    return speed_change * days_left * 86400 / 1000 think = 0for day in range(5, 286, 5):    push = small_forces(0.0888, 0.5)    think = think + drift_km(push, 286 - day)print('Navigators expect', 226, 'km above Mars')

Traceback (most recent call last):  File "main.py", line 15, in <module>    think = think + drift_km(push, 286 - day)                    ~~~~~~~~^^^^^^^^^^^^^^^^^  File "main.py", line 8, in drift_km    raise ValueError('drift_km needs newton-seconds, but got ' + push[1])ValueError: drift_km needs newton-seconds, but got lbf s

It stops at the very first firing, on day 5, on the ground, nine months before Mars. Now add a function that turns any push into newton-seconds, and refuses a unit it doesn’t know:

MASS = 629 def small_forces(thrust, seconds):    return [thrust * seconds, 'lbf s'] def to_n_s(push):    if push[1] == 'N s':        return push    if push[1] == 'lbf s':        return [push[0] * 4.44822, 'N s']    raise ValueError('unknown unit: ' + push[1]) def drift_km(push, days_left):    if push[1] != 'N s':        raise ValueError('drift_km needs newton-seconds, but got ' + push[1])    speed_change = push[0] / MASS    return speed_change * days_left * 86400 / 1000 think = 0really = 0for day in range(5, 286, 5):    push = small_forces(0.0888, 0.5)    think = think + drift_km(to_n_s(push), 286 - day)    really = really + drift_km([push[0] * 4.44822, 'N s'], 286 - day)print('Navigators expect', 226, 'km above Mars')print('Really', round(226 - (really - think)), 'km above Mars')

Navigators expect 226 km above MarsReally 226 km above Mars

The navigators’ picture and the real path are now the same.

• Each part was right, and the whole was wrong. small_forces and drift_km each passed its own test. Joined, they put the spacecraft 169 km lower than the navigators thought: 57 km instead of 226. No single line of code was wrong; the bug was in the meeting place, where a number passed from one program to another without its unit.
• Small errors grow. Each firing’s mistake was a few kilometers at most, and the later ones much less. Fifty-seven of them over nine months came to 169 km, and the earliest mattered most.
• A number that carries its unit can’t be misread. Once each push said 'lbf s' beside its number, the program refused to use it as newton-seconds and stopped on day 5. The checking costs one line, and finds in a second what nine months of careful navigation didn’t.

Big programs are built by many people, in pieces that talk to each other. Testing each piece alone is not enough: the joins need checking too, and units first.

Units in the computer’s own languages. Some programming languages and libraries keep a unit with every number, as your Step 5 program did, and convert or refuse automatically.

Other mix-ups to try. Change Step 2 so that only the firings after day 120 are misread (as if the mistake had come in a later version of SM_FORCES). How much lower does the probe pass? Then try the opposite: what if the file had been in newton-seconds, but the navigators had read every push as pound-force seconds?

★ Say it in newton-seconds (blocks)

The program asks for a push and its unit. Finish it so that it says the push in newton-seconds: “44.5 N s” for 10 lbf s, “10 N s” for 10 N s. For any other unit it must say “Unknown unit: ” and the unit, never guess. Then press Test.

★★ Land in two systems (Python, from Grade 8)

A family’s land is written as a list of pieces, each a number and its unit: hill units (ropani, aana, paisa, daam) and Terai units (bigha, kattha, dhur). Write total_sq_ft(pieces) that gives back the total in square feet. If any piece has a unit it doesn’t know, it gives back 'unknown unit: ' and that unit instead.

• 1 ropani = 5,476 sq ft; 1 aana = 342.25; 1 paisa = 85.5625; 1 daam = 21.390625
• 1 bigha = 72,900 sq ft; 1 kattha = 3,645; 1 dhur = 182.25

def total_sq_ft(pieces):    # pieces: a list like [[2, 'ropani'], [3, 'aana']]    return 0 print(total_sq_ft([[2, 'ropani'], [3, 'aana']]))

★★★ The same kind of thing? (Python, from Grade 8)

Every unit of physics is built from a few base units: kilograms, meters and seconds. A newton is kg m/s² (a kilogram sped up by 1 m/s every second), so a newton-second is kg m/s. powers(name) below gives a unit’s powers of kg, m and s, as a list. Write same_kind(unit1, unit2) that says whether two units measure the same kind of thing. A unit is written as names with spaces between them, multiplied; the names after a / divide; a name may end in 2 or 3 for a power (m/s2 is meters per second per second).

def powers(name):    # a named unit as powers of [kg, m, s]    if name == 'kg':        return [1, 0, 0]    if name == 'm':        return [0, 1, 0]    if name == 's':        return [0, 0, 1]    if name == 'N' or name == 'lbf':    # forces: kg m/s2        return [1, 1, -2]    if name == 'J':                     # energy: N m        return [1, 2, -2] def same_kind(unit1, unit2):    # e.g. same_kind('N s', 'kg m/s') is True    return False print(same_kind('N s', 'kg m/s'))

• Grade 8, Functions: testing each function on its own, the first step; this topic is the second, testing where they join.
• Grade 10, A Project in a Team: agreeing how the parts of a program fit, in writing.
• Grade 12, Software Engineering: integration tests, from one function to the whole system.
• Can You Land on the Moon? Another flight worked out a step at a time, with a pilot of your own.
• What Is the Biggest Number a Computer Can Count To? Another famous kind of bug: counters that run out of room.
• Why Did a Rocket Destroy Itself 37 Seconds After Launch? Ariane 5’s program was right for one rocket and wrong for a faster one (Grades 9–12).