Artist's conception of Lunar CRater Observation and Sensing Satellite (LCROSS).
Image Credit: NASA
As project manager of the Lunar CRater Observation and Sensing Satellite (LCROSS) Mission to the moon, I (re)learned the hard way that nothing comes for free.
The mission of the 2009 LCROSS was to slam the upper stage of an Atlas rocket into a permanently-shadowed crater at the moon’s south pole and analyze the lunar rocks and regolith that got kicked up. We were looking keenly for the presence of water-ice, and found it, rewriting the textbooks about what we know about the moon.
To guide the Centaur upper stage impactor to its target, our shepherding spacecraft employed a typical Attitude Control System (ACS) featuring both an Inertial Measurement Unit (IMU) to measure velocity and a star tracker to determine position. There was nothing unusual about these devices or our approach. Fairly late in the design process, we came to understand that our star tracker had the additional capability of being able to provide a computed velocity based on changes in position over time, effectively performing a similar function to the IMU. If the IMU failed, we could switch over to the star tracker for velocity measurements, giving us a “free” additional capability to fortify our design and reduce risk.
Except that this “free” capability nearly cost us the mission.
Two-thirds of the way through the mission, just a few weeks prior to impact, the operations crew was reacquiring signal from the spacecraft following a planned polar-orbit communications outage. Once the telemetry started flowing from the spacecraft, the team saw all-red indicators, with the thrusters firing erratically. Most of the fuel was gone, and it was unclear if we could finish the mission with what was left.
After all the work we put into this mission, it could now be over.
Fortunately, we were able to halt the thruster firing to end the propellant consumption and move our focus to understanding what had caused the problem. After many grueling, high-tension hours, we discovered that our supposed risk-reducing “free” capability of back-up velocity measurements was the culprit. Worse yet, the problem was totally preventable!
Here’s what happened: A spurious IMU fault indicator caused the ACS to declare that the IMU was compromised. Our updated system design correctly switched to using the star tracker velocity signal, enabling the new data to flow flawlessly. The lurking problem was that the star tracker’s computed velocity signal was (understandably) “noisy,” as it is not a direct measurement of velocity but a calculated velocity based on changing position over time. Our high-performance ACS interpreted the computational noise as actual velocity error that needed to be neutralized, which is exactly what the ACS set out to do. The ACS expended a large amount of fuel, firing thrusters, trying to get rid of inconsequential noise.
In the end, we were able to decrease the sensitivity of the ACS so that it no longer tried to nullify computational noise. We were even able to resume IMU functionality so that we could fly the system as originally intended. We updated the mission plan to be very miserly with our remaining fuel, and successfully got into the history books.
The moral of the story: The gift of the “free” velocity measurement was so attractive that we missed the fact that we were actually growing the complexity of our system. If we had never employed this capability, we would have been fine during an IMU fault, simply losing a little bit of positioning accuracy, but successfully flying with only star tracker position data.
Further, because this alternate velocity option was discovered relatively late in the design flow, we neglected the “Testing-101” requirement: “test as you fly; fly as you test.” If we had tested this backup capability on the ground, the ACS’s response to the noise would have been immediately apparent and could have been easily addressed.
So the old adage, “If it seems too good to be true, it probably is,” appears to have been proven once again.
