Genesis: Learning from Mistakes

Download PDF

By Kerry Ellis
On the morning of September 8, 2004, in a desert in the middle of Utah, two helicopters waited to pluck Genesis gently from the air and return it safely to the ground. Genesis: Learning from MistakesWhen the capsule hurtled into the Earth’s atmosphere and crashed into the desert at 193 miles per hour, it was clear something had gone wrong. Genesis’s drogue, parachute, and parafoil were supposed to slow the 500-lb. capsule enough to allow the helicopters to capture it. Their failure to deploy was the obvious cause for the hard landing. Discovering the root cause of that failure—why those parachutes did not deploy—was what Mike Ryschkewitsch and John Klein set out to do as part of the mishap investigation.

The process of landing Genesis was not a simple one. First, the capsule had to be directed to a tiny spot in Utah, the Utah Testing and Training Range. Then, gravity-switch devices inside the capsule had to sense the braking caused by high-speed entry into Earth’s atmosphere and initiate the timing sequence that led to deployment of the drogue parachute and parafoil, approximately nineteen miles above ground. Moments later, when the sample return capsule (SRC) was nearly one and a half miles above ground, a larger rectangular-shaped parafoil—much like a hang glider—would allow the capsule to glide downward at approximately ten miles per hour. The Genesis team had carefully designed, coordinated, and tested the recovery process in order to minimize the risk involved in retrieving the solar-wind samples unscathed and uncontaminated.

After Genesis was recovered and the remnants analyzed, the Mishap Investigation Board discovered that the gravity-switch devices’ (or g-switches) installation had been designed incorrectly. The switches flipped, but they were in the wrong position inside the electronic configuration; they could not make the connection needed to start the timing process for parachute deployment. Once the immediate cause for the parachute failure had been identified, Ryschkewitsch and Klein investigated further to discover why the switches had been installed incorrectly.

Comet Collector Cell  A scientist holds up the Genesis collector array in a NASA Johnson Space Center cleanroom.

Comet Collector Cell
A scientist holds up the Genesis collector array in a NASA Johnson Space Center cleanroom.
Photo by NASA / JPL

“You can’t just say, ‘don’t install g-switches upside down,’ as a lesson learned from Genesis,” said Ryschkewitsch, who was Director of Engineering at Goddard Space Flight Center at the time and chair of the Genesis Mishap Investigation Board. Now Deputy Center Director at Goddard, one of his goals is to share what they discovered about Genesis after its crash and help other projects avoid the same pitfalls. NASA management called together a large group of NASA experts to discover the cause of the mishap.

Many people were surprised when Genesis returned the way it did. The project’s design was inherited from Stardust, which had launched successfully in February 1999, two years before Genesis. Stardust’s systems had been thoroughly tested, so the problem of drogue deployment evident with Genesis had been unexpected. Those involved with the Stardust project suddenly feared the same problem might happen when Stardust returned a little more than one year after Genesis’s arrival.

Upon further investigation, the Mishap Investigation Board discovered that while the design had been inherited from Stardust, it had also been altered. The box that held the g-switches for Genesis was different than the one for Stardust, so the electronics had to be placed in a different configuration to make them fit. The new design for Genesis was analyzed and tested to ensure the switches still flipped, but it was not tested to ensure the switches would start the process for parachute deployment.

I hear in proposals all the time that “I don’t have to test because it’s heritage,” which is a dangerous thing to say. It’s rare you will build something exactly the same way every time. Things aren’t coming off a production line at NASA; everything is one of a kind. It’s good to reuse hardware, but not to assume it will work the same way every time.

 

“There was confusion around what testing needed to be done and what the tests meant,” Ryschkewitsch said. “One part of the team understood that repackaging the electronics meant losing the ‘heritage’ from Stardust. Other parts never clearly understood that, and it was never clearly communicated between those teams.” So when one team verified that the switches did what they were supposed to—which to them meant flipping from one orientation to another—another team thought full testing had been done, ensuring the switches flipped and made the correct connections.

This lack of communication and assumption about heritage design are not unique to Genesis. “I hear in proposals all the time that ‘I don’t have to test because it’s heritage,’ which is a dangerous thing to say,” said Ryschkewitsch. “It’s rare you will build something exactly the same way every time. Things aren’t coming off a production line at NASA; everything is one of a kind. It’s good to reuse hardware, but not to assume it will work the same way every time.”

Ryschkewitsch and Klein have been sharing the tough lessons learned from Genesis in a variety of ways. At the time of Genesis’s return, several new missions were currently in development at NASA, including Swift, Deep Impact, and New Horizons. “Mike and I have given individual briefings to various projects in various stages—even before the Mishap Investigation Board report was released—so lessons learned from Genesis are carried on to the new projects,” said Klein, who was Manager of Autonomous Spacecraft Division at Jet Propulsion Laboratory during Genesis and is now Deputy Project Manager for the Mars Science Laboratory. “We’ve also given presentations to new project managers so the lessons learned are incorporated at the beginning of their new projects, such as the proposed Discovery, Scout, and Explorer missions,” he added.

They share the lessons learned more broadly as well by having training sessions that discuss in detail the traps Genesis fell into and how to avoid them. These lessons have been shared at systems engineering seminars as well.

Genesis on the Ground  The impact of the Genesis sample return capsule occurred near Granite Peak on a remote portion of the Utah Test and Training Range.

Genesis on the Ground
The impact of the Genesis sample return capsule occurred near Granite Peak on a remote portion of the Utah Test and Training Range.
Photo by USAF 388th Range Sqd

In addition, they have been addressing formal changes to NASA processes. Genesis was built during the same time as the Mars ’98 failures, so the Genesis recommendations are an amplification and sharpening of the lessons that came from Mars ’98. “What it comes down to is making sure you understand requirements, having the requirements written down clearly, and ensuring there is enough dialogue going on between all levels so they all understand what they should be doing and what the risks are,” Ryschkewitsch explained.

Evidence of their success in communicating how mission teams can improve their processes and likelihood of success is in Stardust’s safe return and landing in January 2006. The lessons from Genesis had an immediate impact on the Stardust team’s ability to mitigate and prepare for risk in bringing their sample return capsule home. Though nothing could be done about Stardust’s hardware since it was already in space after Genesis crashed, the Stardust team started training early for their capsule’s return. They reviewed additional risk scenarios and created emergency action plans that clearly outlined who needed to make decisions and when should a problem occur with the SRC landing.

The Genesis mission has also been labeled a success, despite mistakes made on the project. In April 2005, scientists announced they were able to recover samples intact from Genesis’s solar-wind collectors. Scientists will analyze the samples to measure solar-oxygen isotopic composition, the highest-priority measurement objective for Genesis. The data may hold clues to increase understanding about how the solar system formed.

About the Author

Share With Your Colleagues