Why Did a Rocket Destroy Itself 37 Seconds After Launch?

Use the widget above.

• a.Play Ariane 5 with the conversion Unchecked. At what time does the first inertial unit stop? How long after it does the second stop?
• b.Switch to Ariane 4 and play it. How close does its curve get to the dashed line?
• c.Back on Ariane 5, switch the conversion to Checked and play again. What happens at 36.7 seconds now?

What “16 bits” means

The number that failed was being put into a 16-bit signed integer: a whole number kept in 16 bits, with the left bit worth minus its place value (two’s complement, from Grade 9). That gives −32,768 to 32,767, and nothing outside. Here are the 16 bits, near the top. Press +1 until you pass 32,767.

A counter that goes past 32,767 rolls round to −32,768, as the younger topic shows. Ariane’s program did something else: its conversion checked whether the number would fit, and when it didn’t, it raised an error. On a rocket, that was worse.

A float can hold numbers an integer can’t

The number didn’t start as an integer. It was a 64-bit floating-point number (a “float”), the kind a computer uses for measurements with decimals: a sign, an exponent and a few significant digits, like 3.2814 × 10⁴ written in binary Grade 11: Grade 11, Floating point. A float’s exponent lets it reach enormous numbers: a 64-bit float goes up to about 1.8 × 10³⁰⁸. So a float can always hold the measurement. The trouble starts when a program squeezes it into an integer.

This float has only 8 bits, and below it is what the same number gives as an 8-bit signed integer, which holds −128 to 127. Store each number.

The Inquiry Board, chaired by Professor Jacques-Louis Lions of France’s Academy of Sciences, published its report on 19 July 1996. It traced the failure backwards, from the explosion to its first cause. Here it is in the order things happened.

Two units with the same program

A rocket must know which way it points and how it moves. On Ariane 5 that was the job of the Inertial Reference System, called the SRI from its French name: a box with laser gyroscopes and accelerometers and a computer of its own, which worked out angles and speeds and sent them to the On-Board Computer. The on-board computer flew the rocket by swiveling the nozzles of the two solid boosters and of the main engine, the Vulcain.

There were two SRIs, with the same hardware and the same software: one active, one ready in case the first failed. If the on-board computer saw that the active unit had failed, it would switch to the other at once. Two of something is the usual way to make a machine safe: if one part breaks, the other carries on.

The alignment, and a number called BH

Before lift-off, an SRI must learn exactly how it is sitting: which way is down, which way is north. That is the alignment, and a part of the program did it. One of its results was a number called BH, the horizontal bias, related to the horizontal speed the platform felt, which showed how good the alignment was.

The report is plain about this part of the program: it “computes meaningful results only before lift-off. As soon as the launcher lifts off, this function serves no purpose.” Yet it kept running for about 40 seconds of flight. On Ariane 4 that had a reason: if a countdown was stopped in its last seconds, the alignment could carry on, and the count could start again quickly instead of waiting 45 minutes or more for a new alignment. (It was used once, on Flight 33 in 1989.) Ariane 5 is prepared for launch differently and didn’t need it. It was kept, the report says, “for commonality reasons”: the program worked on Ariane 4, and it seemed unwise to change it.

The conversion

Every cycle, the program converted BH from a 64-bit float to a 16-bit signed integer. Seven variables in that part of the program were found, when it was written, to be at risk of not fitting. Four were protected, with a check in the code. Three were not, BH among them, because the reasoning was that they were either physically limited or had a large margin of safety. The Board was told that one reason for leaving some out was that the SRI’s computer had been given a target of using at most 80% of its time. It was agreed not to put Ariane 5’s trajectory, its planned path, into the SRI’s requirements, so nobody checked BH against the path Ariane 5 would really fly.

Early in its flight, Ariane 5’s horizontal speed builds up five times more quickly than Ariane 4’s. BH grew far bigger than it ever had. At H0 + 36.7 seconds, about 30 seconds after lift-off, it no longer fitted in 16 bits, and the conversion raised an Operand Error.

What the error did

The SRI’s specification said what to do after any error: report the failure on the data line, store the details in a memory that keeps them without power, and shut the processor down. The design treated an error the way it treated a broken part: switch off the faulty unit, and let the spare take over.

But the spare had the same program, fed the same kind of numbers. It had already stopped, for the same reason, one data cycle (72 milliseconds) before. (In its list of findings the report gives the gap as about 0.05 seconds; either way, a blink.) Then:

• a.The active SRI, shutting down, sent a diagnostic bit pattern in place of flight data, and the on-board computer read it as flight data.
• b.From those numbers it saw a large turn that hadn’t happened, and swung the boosters’ nozzles, and then the main engine’s, to their full deflection, to correct it.
• c.The rocket turned abruptly. Its angle of attack, the angle between where it pointed and where it was going, passed 20°, and the air tore it apart.
• d.The boosters broke away from the main stage, which set off the self-destruct system, as designed, at H0 + 39 seconds, about 4 km up.

The report sums it up: the failure “was caused by the complete loss of guidance and attitude information 37 seconds after start of the main engine ignition sequence (30 seconds after lift-off). This loss of information was due to specification and design errors in the software of the inertial reference system.”

You will rebuild the failure in four steps, each running here on the page. One warning first: the numbers for BH are made up. The report gives no values of BH, so the program uses a formula picked to match what the report does say (Ariane 5 five times faster than Ariane 4, the limit passed at 36.7 seconds, and lift-off about 7 seconds after H0, worked out from the report’s “36.7 seconds after H0 (approx. 30 seconds after lift-off)”). The limit, the times and the logic are real.

Step 1: the conversion

Python’s whole numbers have no limit (try print(2 ** 100)), so to behave like a 16-bit integer, a program must make the limit itself. round(x) gives the nearest whole number to the float x; if it is outside −32,768 to 32,767, the function stops with an error of its own: raise OverflowError(...) stops the function and says why, as Ada, the language the SRI was written in, stopped with its Operand Error.

def to_int16(x):    n = round(x)    if n < -32768 or n > 32767:        raise OverflowError(f'Operand Error: {x} does not fit in 16 bits')    return n print(to_int16(1234.6))print(to_int16(-32768.0))print(to_int16(40000.25))

It prints 1235 and -32768, and then:

Traceback (most recent call last):  File "main.py", line 9, in <module>    print(to_int16(40000.25))          ~~~~~~~~^^^^^^^^^^  File "main.py", line 4, in to_int16    raise OverflowError(f'Operand Error: {x} does not fit in 16 bits')OverflowError: Operand Error: 40000.25 does not fit in 16 bits

The first two fit. The third stops the whole program, and nothing after it runs. That is what happened inside each SRI.

Step 2: fly both rockets

Now the made-up BH, worked out every tenth of a second from H0 to H0 + 47 seconds (the end of the alignment’s 50 seconds, which start 3 seconds before H0), and converted each time. To find out when the conversion fails without the program stopping, the loop puts it inside try: if anything in try raises an error, Python jumps to except, where the program can say what happened (Grade 10 has try and except in full).

LIFTOFF = 7      # seconds after H0, aboutK5 = 37.2        # made up, so that BH passes 32,767 at H0 + 36.7 sK4 = K5 / 5      # Ariane 4: horizontal speed builds up 5 times more slowly def bh(k, t):    s = t - LIFTOFF    if s <= 0:        return 0.0    return k * s * s def to_int16(x):    n = round(x)    if n < -32768 or n > 32767:        raise OverflowError(f'Operand Error: {x:.3f} does not fit in 16 bits')    return n def fly(name, k):    for step in range(471):      # every 0.1 s, H0 to H0 + 47 s        t = step / 10        try:            to_int16(bh(k, t))        except OverflowError as e:            print(f'{name}, H0 + {t:.1f} s: {e}')            return    print(f'{name}: every conversion fitted; BH reached {bh(k, 47):,.0f}') fly('Ariane 4', K4)fly('Ariane 5', K5)

Ariane 4: every conversion fitted; BH reached 11,904Ariane 5, H0 + 36.7 s: Operand Error: 32813.748 does not fit in 16 bits

The same function, the same line, two rockets. On Ariane 4, BH never gets beyond about a third of the limit in the whole time the alignment runs. On Ariane 5 it passes the limit 30 seconds after lift-off.

Step 3: why the spare didn’t help

A spare is a good defense against random faults: a wire that breaks, a chip that dies. Suppose (another made-up number) that each unit has a 1 in 100 chance of a random hardware fault during a flight. With two units, the flight is lost only if both fail, which should be about 1 in 100 × 100 = 10,000. This program flies 100,000 flights to see, and then adds the bug, which strikes every unit with the same program on Ariane 5’s path.

import randomrandom.seed(501) P = 0.01   # made up: the chance a unit has a random hardware fault in one flight def flight(units, same_bug):    working = 0    for u in range(units):        broken = random.random() < P        if same_bug:            broken = True        # the same program meets the same numbers        if not broken:            working = working + 1    return working > 0           # one working unit is enough for units, bug, name in [(1, False, 'One unit, random faults'),                         (2, False, 'Two units, random faults'),                         (2, True, 'Two units, the same bug')]:    lost = 0    for n in range(100000):        if not flight(units, bug):            lost = lost + 1    print(f'{name}: {lost:,} of 100,000 flights lost')

One unit, random faults: 949 of 100,000 flights lostTwo units, random faults: 11 of 100,000 flights lostTwo units, the same bug: 100,000 of 100,000 flights lost

A second unit cut the random losses from about 1 in 100 to about 1 in 10,000: a hundred times safer. Against the bug it did nothing at all. The report saw this as the deeper lesson: the Ariane program’s culture was “of only addressing random hardware failures”, and an error in software was handled as if it were one.

Step 4: the checks that would have caught it

The report’s recommendations name several ways out. Three are easy to program:

• Check the conversion and send the best value that fits instead of stopping (its recommendation R3: “Do not allow any sensor … to stop sending best effort data”).
• Switch the alignment off at lift-off (R1: “no software function should run during flight unless it is needed”).
• Test with the real trajectory before flying (R10: “Include trajectory data in specifications and test requirements”). A test that runs fly with Ariane 5’s numbers, on the ground, finds the failure at 36.7 seconds before anyone launches anything.

LIFTOFF = 7K5 = 37.2K4 = K5 / 5 def bh(k, t):    s = t - LIFTOFF    if s <= 0:        return 0.0    return k * s * s def to_int16(x):    n = round(x)    if n < -32768 or n > 32767:        raise OverflowError('Operand Error')    return n def to_int16_checked(x):         # the nearest number that fits    n = round(x)    if n > 32767:        return 32767    if n < -32768:        return -32768    return n def fly(k, convert, align_until):    for step in range(471):        t = step / 10        if t > align_until:      # alignment switched off            break        try:            convert(bh(k, t))        except OverflowError:            return f'both units stop at H0 + {t:.1f} s'    return 'both units keep working' print('As flown:            ', fly(K5, to_int16, 47))print('Checked conversion:  ', fly(K5, to_int16_checked, 47))print('Alignment off at lift-off:', fly(K5, to_int16, LIFTOFF)) def test_trajectory(k):          # run on the ground, before the flight    result = fly(k, to_int16, 47)    if result != 'both units keep working':        return 'TEST FAILED: ' + result    return 'test passed' print('Ground test, Ariane 4:', test_trajectory(K4))print('Ground test, Ariane 5:', test_trajectory(K5))

As flown:             both units stop at H0 + 36.7 sChecked conversion:   both units keep workingAlignment off at lift-off: both units keep workingGround test, Ariane 4: test passedGround test, Ariane 5: TEST FAILED: both units stop at H0 + 36.7 s

Each fix is a few lines, and the test, with the right numbers fed in, fails loudly, on the ground. The report says the same of the real thing: had such a test been done, “the failure mechanism would have been exposed.” The program had been tested a great deal, but never with Ariane 5’s flight path.

Your programs found, from one line of code and a made-up curve:

• a.The same code was right for one rocket and wrong for another. to_int16(bh(k, t)) never failed with Ariane 4’s numbers and failed 30 seconds after lift-off with Ariane 5’s (Step 2). Nothing in the code changed. What changed was an assumption about its numbers, which was true for Ariane 4 and was never written into the code.
• b.A spare doesn’t help against a bug. Two units made random faults a hundred times rarer and did nothing against the same program meeting the same numbers (Step 3).
• c.The fix was small; finding the need was the hard part. A five-line check, or one comparison, or one test with the right trajectory would each have caught it (Step 4).

That is why reusing code is both the best and the most dangerous thing in programming. Code that has flown many times feels proven, but it was proven only for the job it did. Move it to a new job, and every assumption it makes about its numbers must be checked again. The report puts it this way: “software should be assumed to be faulty until applying the currently accepted best practice methods can demonstrate that it is correct.”

What other languages do

Ada raised an error. Other languages do other things. JavaScript, putting a number into a 16-bit list, keeps it wrapped round; Java and Rust, turning a float into a 32-bit integer, keep the nearest limit. In C, a float too big for its integer is undefined behavior: the C standard doesn’t say what happens, so the program may do anything at all (the rule is set out on cppreference’s page on C conversions). None of these is right for every program. Python’s whole numbers have no limit, but the moment they go into a fixed-size store (a file format, a sensor, a library that works in 16 or 32 bits), the question comes back.

Range analysis

The report asks for every flight program to be checked for “the range of values taken by any internal or communication variables”. For a short program you can do it by hand: if BH is K × s², and s is at most 40 seconds of flight, then BH is at most 1,600 × K, a sum you can do before writing any code. Challenge ★★★ below does it the other way round.

An open question to think about: in a program of hundreds of thousands of lines, where numbers pass through many functions, how could a computer find every variable that might ever leave its range, without running the program on every possible flight? Trying every flight is impossible, so the answer must come from reasoning about the program’s text, as you did above for BH, done by a program.

Three challenges, each checked on the page: write your function in the program above the checker (press Edit), then press Check my program. The examples show what your function should give; there are more tests you can’t see.

★ The nearest number that fits

Write clamp16(x): round the float x to the nearest whole number with round, and if that is outside −32,768 to 32,767, give back the nearest limit instead.

def clamp16(x):    # Round x, then keep it within -32768 to 32767.    return 0 print(clamp16(40000.25))

★★ When does it first fail?

A list holds a variable’s values, one per cycle, as floats. Write first_failure(values, bits): the position (counted from 0) of the first value that, rounded, doesn’t fit in a signed integer of bits bits, or -1 if they all fit. A signed integer of 16 bits holds −32,768 to 32,767; of 8 bits, −128 to 127.

def first_failure(values, bits):    # Where the first value that doesn't fit is, or -1.    return -1 print(first_failure([100.0, 20000.5, 32767.4, 32767.6, 50000.0], 16))

★★★ How much faster can it fly?

You are reusing a program on a new rocket. On the old rocket, a variable took the whole-number values in values (some may be negative, at least one isn’t 0). On the new one, every value will be k times bigger. Write largest_speedup(values, bits): the largest whole number k for which every k × value still fits in a signed integer of bits bits.

def largest_speedup(values, bits):    return 0 print(largest_speedup([0, 1500, 6000, 11904], 16))

• Grade 9, Two’s complement: signed integers, and why the negative side has one more number than the positive.
• Grade 11, Floating point: how a float keeps a sign, an exponent and a mantissa, and where every machine runs out.
• Grade 12, Tests that run themselves: choosing test data, so the test that would have saved Ariane 5 gets written.
• What Is the Biggest Number a Computer Can Count To?: counters that roll over, from Pac-Man’s level 256 to YouTube’s view counter.
• How did a rounding error stop a missile defense?: a number that couldn’t be stored exactly, a clock that drifted, and a missile missed.