By Philip Weber
Ground crew veterans at Kennedy Space Center still talk about what they call “the summer of hydrogen”—the long, frustrating months in 1990 when the shuttle fleet was grounded by an elusive hydrogen leak that foiled our efforts to fill the orbiter’s external fuel tank.
On August 9, 1990, Columbia (left) rolls out to the pad while Atlantis rolls back to the Vehicle Assembly Building.
Photo Credit: NASA
I was the external tank/solid rocket booster project engineer on the ground crew during that painful stretch of mainly trial-anderror efforts to locate and solve the problem. It drove us crazy for more than six months—a Florida-length summer. As painful as it was, the experience demonstrated the incredible dedication and persistence of the workforce and, we eventually discovered, showed the importance of designing tests that match flight conditions as exactly as possible.
Columbia (STS-35) was on Launch Pad A for a scheduled May 30 launch when we discovered the hydrogen leak during tanking. The external fuel tank is loaded through the orbiter. Liquid hydrogen flows through a 17-inch umbilical between the orbiter and the tank. During fueling, we purge the aft fuselage with gaseous nitrogen to reduce the risk of fire, and we have a leak-detection system in the mobile launch platform, which samples (via tygon tubing) the atmosphere in and around the vehicle, drawing it down to a mass spectrometer that analyzes its composition. When we progressed to the stage of tanking where liquid hydrogen flows through the vehicle, the concentration of hydrogen approached four percent—the limit above which it would be dangerously flammable. We had a leak.
We did everything we could think of to find it, and the contractor who supplied the flight hardware was there every day, working alongside us. We did tanking tests, which involved instrumenting the suspected leak sources, and cryo-loaded the external tank to try to isolate precisely where the leak originated. We switched out umbilicals; we replaced the seals between the umbilical and the orbiter. We inspected the seals microscopically and found no flaws. We replaced the recirculation pumps, and we found and replaced a damaged teflon seal in a main propulsion system detent cover, which holds the prevalve—the main valve supplying hydrogen to Space Shuttle Main Engine 3—in the open position. The seal passed leak tests at ambient temperature but leaked when cryogenic temperatures were applied. We added new leak sensors—up to twenty at a time—and tried to be methodical in our placements to narrow down the possible sources of the problem. We even switched orbiters, sending Columbia back to the Vehicle Assembly Building and bringing outAtlantis, scheduled to fly as STS-38. Two shuttles on their mobile launchers passing in the night was a majestic sight, but not one you want to see if you’re trying to get an orbiter launched. None of this told us where the leak was, or if we were dealing with more than one leak source.
One member of the ground crew even volunteered to sit in the aft fuselage during fueling wearing an oxygen supply so he could carry a sensor around from point to point until he found the leak. It’s no surprise that his proposal was vetoed on safety grounds, but he was ready to do it—that’s how frustrated and determined we were.
Eventually, since nothing else had worked, we put a series of leak detectors outside the orbiter, near where the umbilical connected with the external tank. We found the greatest concentration of hydrogen there, so we knew, finally, that the leak had to be at the seal we had changed out and so rigorously inspected. Now that we knew the leak was there, we were able to figure out what was happening. We knew from the testing that the leak would appear when the liquid hydrogen, which is much colder than the gaseous hydrogen used to chill the system, was flowing through the vehicle. The extremely cold liquid hydrogen made the metal of the joint contract unevenly, creating small gaps that the hydrogen escaped through. The fix seems counterintuitive: we added spacers outboard of the bolts in the umbilical flange; when the bolts were tightened, the inside diameter of the flange squeezed down tighter on the seal. We also slowed the loading sequence to reduce the cold shock created when the liquid arrived at the joint.
But how had the seal passed all its tests at the contractor? Why didn’t they see the leak then? Since they were working so closely with us, they were able to supply the answers as soon as we understood the problem. They had tested the seals with liquid nitrogen, not liquid hydrogen. They had a good reason for that choice. Their facility in Downey, once fairly isolated, had seen Los Angeles grow around it. With schools and offices nearby, testing with hydrogen had become too dangerous. Liquid nitrogen was the safe alternative. But liquid hydrogen, at about -253°C, is much colder than liquid nitrogen, which liquefies at about -196°C. Also, hydrogen atoms are many times smaller than nitrogen atoms. So the seal worked fine with liquid nitrogen, but liquid hydrogen created gaps it could slip through. (Hydrogen atoms are so small, they can even escape through a weld.)
The lessons we took from this experience, in addition to seeing that persistence and dedication eventually pay off, are these:
- Don’t take anything for granted.
- Stay in constant communication with the hardware manufacturer.
- Test as you fly.
On October 6, 1990, Discovery took off from Launch Pad 39B, the first launch since April. Other successful launches would occur in November and December. The summer of hydrogen was over.
