by Charles Murray & Catherine Bly Cox
My rating: 5⁄5
This year is the 50th anniversary of the moon landing, so our book club at Apple voted to read this book. I love this book. It tells the story of the entirety of the Apollo program, from inception to completion. A lot can be learned from this fantastic tale.
Quoted texts are taken from the book. Emphasis mine.
1. Russians can be very cheeky
The United States … was working on a grapefruit-sized satellite weighing three and a half pounds. … launch this puny competitor to Sputnik in front of television and newspaper cameras from around the world. The Vanguard rocket being used for the launch rose four feet into the air, fell back, and crumpled onto the pad in a spectacular explosion. A few days later, the Soviet delegate to the United Nations inquired solicitously whether the United States was interested in receiving aid earmarked for underdeveloped countries.
2. Keep it simple
a.k.a. YAGNI principle
In the early day of NASA, they have a limited budget and haven’t built a sophisticated transport system for the spacecraft. They just lined up an old pickup truck with mattresses and got the job done. Too bad I couldn’t find the picture.
Pictures still in the files attest to the story, showing a battered pickup with a spacecraft on mattresses pulling up to the pad where an Atlas rocket stands waiting.
3. Building a construction marvel is easier than conveying its scale to the general public
To house the gigantic Saturn V rocket, NASA built a mega-warehouse that’s really massive in scale. The construction went fine, but they have trouble taking a photo to show the size.
Yet conveying this size in ways other than numbers turns out to be impossible. Ray Clark, who ran the Cape’s support facilities in the 1960s, recalled that as the V.A.B. was nearing completion, NASA wanted to show the public how big it really was. But nothing their photographers produced captured the immensity. NASA got in touch with Life magazine, whose editors replied loftily that NASA need not worry, Life’s photographers knew how to do that sort of thing. The Life photographers flew down and set up shooting positions from every conceivable angle. They shot hundreds of rolls of film and went back to their labs in New York. “A while later they came back to see us,” Clark remembered. “They said, ‘We don’t know how to do it either. There’s just no way to portray photographically, or any way we know of, what the building’s really like.’”
4. Planning for the impossible makes it easy to imagine building the extremely hard
If they had not been talking about the Nova, the huge Saturn would have seemed impossible. “The fact that we could postulate Nova with somewhat of a straight face automatically made the five-eighths-size Saturn credible.” It is one of the tricks engineers sometimes use on themselves, Maynard said, to maneuver themselves into taking on tasks that would otherwise be terrifying.
5. Rocket science is more uncomplicated than car
Liquid fuel is pumped into a combustion chamber in the presence of liquid oxygen and a flame. It burns. That’s all there is to it. There are no crankshafts to turn, no pistons to drive. The burning fuel produces energy in the form of gases that exit through the rocket’s nozzle. The force the gases produce against the top of the engine is called thrust. The thrust is transmitted through the rocket’s structure and, if it is greater than the weight of the rocket, the rocket lifts off. Put in its most basic terms, for any rocket to work there are two things that must be done extremely well: The propellants must be brought together, and then they must burn smoothly.
6. Exploding a bomb inside of the engine is a totally valid way to test the engine’s stability
The engineers were having a problem reproducing an engine instability problem. The problem arises seemingly in random. In software engineering, we call this Heisenbug . So how would they reproduce the issue consistently? To plant a bomb inside of it, of course.
… for testing purposes they must “be able to initiate this instability at our command,” as Castenholz put it. Nature, left to herself, wasn’t producing instabilities frequently enough for the team to learn how to control them. Therefore, they decided to explode a bomb in the combustion chamber while the test was in progress—if that didn’t drive the engine into instability, then nothing would, they reasoned. Then they could concentrate on making the engine damp out the instabilities they had induced.
After numerous bomb explosions and iteration, they got a system that would mend itself.
They fiddled and nudged, and the incidence of instability decreased still further until, by late 1964, they weren’t getting it anymore. The bombs would explode, the pressure in the combustion chamber would skyrocket—and then the engine would be running smoothly again, not just within the 400-millisecond goal the team had set for itself, but within 100 milliseconds.
7. The best way for higher up to know what’s going on is to communicate directly with the engineers
Joe Shea (Deputy Director of NASA) would get each engineering team to provide a weekly report. He would read and annotate it. This is the way they come out with to let the higher-ups stay in touch with the ground-level work.
Markley went away to produce the first edition of the loose-leaf notebook that he and his staff would prepare for Shea every week for the next 165 weeks. Each notebook ran to more than a hundred pages and was on Shea’s desk by close of business Thursday. Shea would get up at 4 A.M. Friday and start to annotate it, usually with technical comments and instructions, occasionally with remarks such as “You’ve gotta be kidding.” He would work on the notebook intermittently from Friday morning through the weekend and return it to Markley on Monday morning. By Monday afternoon, everyone in the ASPO network would have gotten his response from Shea. With the annotated notebook in his hands, Markley became the action officer. “I’d take this notebook, and then I’d fan it back out very quickly and say, ‘Hey, we’ve got these questions and I want the answers in the notebook by this Thursday.’ We’d get the answers back and then we’d take Joe’s marked-up notebook and start off in the front, anything he had marks on. We’d have the answers there. And then we’d have the brand-new section with the newest status too.” The scrawled notations to and fro were a way of keeping up “a running communication,” in Shea’s words, without having to type up long memos. “Even if I didn’t see [the project officers] directly, I’d at least been in touch with them, and they’d been back in touch with me.”
8. “If you understand it, you can make me understand it.”
And then Joe would begin to grill you. There was no point in trying to sidestep or fudge an answer. It made no difference if Shea wasn’t a specialist in your area. “If you understand it, you can make me understand it,” Shea kept saying, and you had better understand it absolutely cold or Shea would dissect you.
9. Keep it sweet and simple
a.k.a avoid over-engineering.
the people who wrote the specifications for the fuel gauges had reasoned that, because it was so important, they needed a highly accurate fuel gauge system. The way they would do it was to build a Geiger-counter arrangement that would infer the bulk of fuel left in the tank by the attenuation of radiation sources that would be transmitted through the fuel. It was a challenging bit of technology, no doubt about it—so challenging that the gauge they had developed so far was accurate only to about 10 percent and it kept failing quality tests for reliability. Shea let this process run for a couple of months and then stepped in. “This thing doesn’t make any sense to me,” he said. “We don’t need anything that complicated.” Shea had a Karmann Ghia that didn’t even need a fuel gauge, because it had a little reserve fuel tank that he could switch to. And that’s what they ended up doing for the R.C.S. fuel tanks. They had a big tank that they used during the mission, and a little tank, not to be touched unless the big tank ran out, but that could get the crew home safely. No more nuclear gas gauge.
10 .“The better is the enemy of the good”
This discussion comes up surprisingly often in tech. Should we rewrite our software in this newer, sexier framework/language? The answer is usually no because if you had time to ask this question, often you would already have a satisfactory working product.
“Because if we fail a qual test and if I have to send it back to redesign,” Shea explained, “then I have a less mature product when it comes back than I had already gotten up to that point.” The better is the enemy of the good.
11. Trust in reducible complexity
The system can be complicated, but you can tackle it by understanding each of the pieces.
“Hey, it isn’t that complicated. It is very understandable. The engines work this way, the guidance system works that way, the transistors work this way, so don’t get yourself in a state of mind thinking that it’s too complex. It really is very simple. It’s piece by piece.” It’s awfully big, that’s all, Shea kept saying. Piece by piece, it’s simple.
12. Even during the busy time, we need to have a calmer period to reflect
The most significant accident of the Apollo program happened during the test launch of Apollo 1. A catastrophe where all the crew were killed. The grief gave everyone time to pause and reflect.
“once the fire occurred, the flight schedule came to a screeching halt and everybody stopped and took stock.” People then had the time to go back and work in a less pressured fashion on “all of these things that everyone had in their back pocket that they should have worked on, and hadn’t had a chance to.”
13. “If you don’t know what to do, don’t do anything.”
“If you don’t know what to do, don’t do anything.” But it was simple as a koan is simple, for flight controllers were trained and conditioned to solve problems; the temptation was for a controller to think that he knew what to do when he really didn’t. True wisdom in flight control lay in being able to recognize one’s own ignorance. Much of the training of the flight controllers consisted of showing them how they could be fooled.
14. Hire smart people & trust them to make a right decision
Flight had to listen carefully to every word and nuance of the answers. “The most likely error we can make in the business is not listening,” said Kranz. “We’ve got very smart people [in the controllers]. We breed them to be very smart, we breed them to give opinions. We breed them to work in an arena of conflict. Sometimes they may disagree with you. Sometimes they may not be too smooth in words. Sometimes they may come in on an untimely basis and disrupt your train of thought.” But none of that can interfere with your listening and understanding precisely what it is that you are hearing.
Then another during the Apollo 13 accident.
For Glynn Lunney, standing alongside Kraft, the reaction of the men in the viewing room was inspiring. The option Kraft was recommending had the obvious disadvantage of leaving the crew at risk for twenty-four hours longer than the alternatives, but there was no quibbling. Lunney: “These men who we all grew up thinking were kings, absolute tops in our business, sat in this room and said to us, ‘Look, we’re here to support you guys, and we just want to hear what’s going on and we want you to know that if there’s anything at all that you need, you just let us know.’” Then and in the days to come, Lunney recalled, “they never second-guessed, never reversed anything, never even came close to that. No cover-your-ass. The whole set of interactions was one of the most supportive and reassuring you can imagine.” The gathering quickly ratified Kraft’s choice: The Apollo 13 crew would come home more slowly, but, they hoped, more surely.
15. Don’t mess with the rocket fuel tank
One day, a technician at White Sands working with a fully fueled test article was filling out a report and absentmindedly clicked the end of his ballpoint pen on a fuel tank. The pen exerted just enough extra pressure on the tank to open a pinprick leak. Eventually, or so the story goes, they found the pen embedded in a fence post, along with a nub of finger bone. The wire-thin stream of propellant had sliced through flesh and bone as neatly as a scalpel.
16. “The Eagle has landed”
When Neil Armstrong and Buzz Aldrin landed on the moon, they announced like so:
“Houston, Tranquility base here,” Neil Armstrong announced. “The Eagle has landed.” “Roger, Tranquility, we copy you on the ground. You’ve got a bunch of guys about to turn blue. We’re breathing again. Thanks a lot.”
17. Anything that could go wrong will go wrong, so always be prepared
Apollo 13 accident was so unlikely that they never even practised it in the simulator.
Such experiences led to a few procedures that could be pulled off the shelf during Apollo 13. Still, the flight control team had never developed procedures for an accident of the magnitude of Thirteen’s. They had imagined losing one of the two oxygen tanks, or a battery, or even the S.P.S. engine. But never had the simulations postulated that the C.S.M. would be completely dead. Surely, everything thought, anything that knocked out both oxygen tanks would also destroy the spacecraft. Apollo 13 brought a change in simulation policy. FIDO Dave Reed summarized the new ground rule: “They can throw anything at us they want and we won’t object.”
18. In unusual circumstances, forget the rulebook and improvise
When you encounter something that has never happened before, it might mean that you will have to do something you’ve never done before.
To conserve water and power, the engineers in the MER went systematically through the spacecraft, looking at every component to see whether it could be turned off completely. If not, they calculated the lowest voltage at which any given piece of equipment would operate and the minimal configuration in which a system would continue to perform the functions it absolutely had to perform. Forget the specs, forget the rulebooks, forget the operations manuals, Arabian announced. Go back to the physics of the design that no one knows better than you, the knowledge that’s not written down anywhere, and decide what’s the least we can really get away with.
19. Rely on checklists
NASA has checklists for everything . When the Apollo 13 accident happened, they needed the astronaut to solve some problem, and of course, they created a checklist for them too.
The checklist system is also widely used in medicine .
So as the White Team assembled Wednesday morning, they were a disheveled and weary crew, with a little more than two days left before splashdown. Under ordinary circumstances, preparing an entry checklist for a lunar mission was a three-month job.
The creation of the checklist was a controlled frenzy in which solutions were developed, rejected, and replaced in such rapid-fire sequence that it was difficult at any one moment to be sure how far they had to go.
20. Astronauts jobs are really, really hard
During the Apollo 13 accident, the astronaut had to survive on a meagre amount of water & sleep.
By Thursday, the temperature in Odyssey was thirty-eight degrees, about that of a refrigerator. Until then, the crew had used Odyssey as the bedroom of a two-room apartment; now they huddled together in the little LEM, which was marginally warmer. But Aquarius, too, was uncomfortable. The walls and windows of the LEM were moist with condensed water. The men were averaging three hours of fretful sleep per day. Also, concerned about having enough water to cool the electronics, all three astronauts had cut their fluid intake to near zero. They deliberately ignored the Surgeon’s instructions to drink—since they didn’t feel that thirsty anyway (one doesn’t, in space), they chose to save water for the equipment. The result was that, without knowing it, all three were becoming dehydrated. Haise had developed a kidney infection and was running a fever. Fatigue. Cold. Dehydration. As they neared time for entry, the normal performance levels of all three had deteriorated. Now they were going to have to assimilate a hastily written entry checklist, perform a crucial midcourse correction, and then bring the spacecraft home without any of the backup systems that ordinarily provided some margin for error.
21. If you want to maintain a good family relationship, don’t work too much
Work-life balance is imperative, especially when you have spouse and kids.
In many cases, families didn’t survive the stress—during the mid-1960s, the towns around K.S.C. had the highest divorce rate in the country.
The consensus is that this is no coincidence. Under the heat of Apollo, your marriage either melted or annealed.