By Karen Halterman
Scientific instruments for robotic NASA space missions are usually designed for flight on a specific satellite with a planned launch date. Sometimes multiple copies of instruments are developed to fly on several satellites. Occasionally, the last instrument in the series is launched years after originally planned and may be decades old when it finally reaches space.
It might seem that almost all work related to flying a series of instruments should be finished after the flight hardware is built, with only minimal effort needed later to launch the stored instruments. In reality, maintaining flight hardware for many years prior to flight is a major challenge. The experiences of the Goddard Space Flight Center Polar Operational Satellites (POES) in providing earth-sensing instruments developed for the National Oceanic and Atmospheric Administration for flight on the European Meteorological Operational (MetOp) spacecraft teaches some important lessons about that challenge.
These instruments were procured from Goddard instrument contractors at an average cost of about $14 million for each instrument at delivery in the late 1990s. The last set of these instruments will be almost twenty years old at the time of their planned launched on MetOp-C around 2018.
Programmatic Issues
Some of the issues faced by POES and other projects responsible for launching old instruments are programmatic, relating to how the missions are supported, staffed, and managed.
Obtaining Sufficient Funding
Problem: When a decision is made to develop a series of satellites and instruments, the need for the mission is well defined. Strong financial arguments are made for copies of the same instrument—savings come from having a single design for multiple units and mass parts purchases. Many years later, after all but the last unit of the series are on orbit, it is much harder to obtain needed funding. Typically, the original schedule is out of date and the final launch has been delayed many years. Government managers who started the mission have transferred to other projects or retired, and savings from procuring multiple copies are being eroded by the sustaining engineering costs for the unexpectedly long mission lifetime. Those now responsible for budgets are probably struggling to fund missions currently in development and may be less familiar with the old one.
Mitigation: Decision authorities need to be aware of the problems of flight hardware maintenance and should advocate for the necessary budget. The POES project has addressed this by identifying the increased risks to mission success without the necessary sustaining engineering funding. While it is expensive to lengthen the old mission, its longer lifetime often defers the costs of a follow-on mission to later years.
Retaining Knowledge
Problem: It is difficult to preserve detailed understanding of the design, manufacturing, and testing of flight hardware twenty years after the last unit was delivered. Folks move to new projects, relocate, change employers, or retire. Companies no longer produce spaceflight hardware or go out of business. Records are archived somewhere in yellowing paper or in obsolete, unreadable electronic media. Documentation is incomplete or ambiguous. Important unwritten lessons have been forgotten.
Mitigation: Continuity of a core team is essential to maintaining old instruments and new people must be added when needed. On-orbit support for the launched instruments can provide a focus for training new personnel. The POES project has maintained high-fidelity engineering units on all instrument contracts. They bring their essential expertise to rehearsals for working on the flight unit and investigating anomalies by recreating the test conditions or on-orbit environment. Frequently, the company that provided the unit is developing newer instruments in the same family type and has a pool of skilled individuals who can assist the older project. Key positions in the new project are often filled by veterans of the previous instrument generation. Retirees from the old project have been an excellent knowledge source for the POES project; they often gladly work part time and can support essential activities like prelaunch reviews. Incentives to keep experienced people with the old project include interesting assignments during slow periods, participation in spacecraft-level testing, attending the final launch, and bonuses.
Extending Contracts
Problem: It takes about five years from contract start to deliver the last unit in a series. Assuming another twenty years until the final launch, flight hardware contracts can be active for twenty-five years or longer. Corporate takeovers and mergers as well as changes to major subcontractors and suppliers are likely to affect contracts that old. Company restructuring usually creates new mandated processes and procedures that are disruptive to an old project. Government contract administration is required until the last instrument is launched. The continually updating NASA flight-project requirements need to be evaluated to determine which ones apply to flight hardware built so many years earlier.
Mitigation: Appropriate levels of government contractual, financial, and technical effort are necessary to keep the instrument contracts active. One POES project instrument contract, executed in 1988, has had more than 375 modifications, including several sole-source performance period extensions. General NASA requirements should be grandfathered to when the instruments were delivered unless there is compelling reason and funding to add new ones to the contract.
Technical Issues
POES and similar projects also have to deal with technical issues related to the age of instruments.
Extended Storage
Problem: Had it been known at project start that the final instrument would be launched twenty years after delivery, long-term maintenance would have been a design requirement. Easy replenishment of limited-life items or storage orientation to minimize gravitational effects are the kinds of measures that would have been taken. Problems with flight hardware resulting from many years on the ground include expiration of materials’ shelf lives, lubricant creeping from bearings and reservoirs, relaxation of mechanical preloads, cracking of stakes and bonds, and contamination of detectors by moisture or organics.
Mitigation: Flight hardware should be kept in an appropriately clean, controlled environment. POES project instruments are purged with gaseous nitrogen to minimize the degradation of materials over time. They are activated periodically to ensure they function properly and to exercise motors and mechanisms. If testing before launch indicates that performance has degraded and there is insufficient budget or time for repair, then project leaders must recognize that all instrument requirements may not be met on orbit.
Making Repairs
Problem: If flight hardware is twenty years old at launch, it was probably modified after it was built. Instruments can require rework for many reasons: to correct problems found on earlier units in the series, in response to parts alerts, handling or shipment damage, test failures, spacecraft interface changes, contamination, overstressing during testing, alignment changes, and other factors. There is risk in repairing an old instrument without skilled people, good documentation, and the necessary ground-support equipment (GSE). A full set of spare parts or assemblies may not be available because they were never originally procured or were installed in earlier flight units. Acquiring new parts for old flight hardware is arduous; parts may no longer be manufactured, may be unaffordable due to minimum lot requirements, or be only available from an unreliable off-shore supplier. The design may contain unique parts developed by a niche company that went bankrupt. Existing parts probably need re-life testing. If new parts are used to replace obsolete or unavailable parts, redesign will be necessary. Replacement parts may not fit the original footprint, interfaces, or thermal characteristics. The company’s manufacturing capabilities may have been modernized since the instrument was built. For example, surface-mount techniques may be the sole method used today for electronics boards, so repair of boards designed for older techniques may not be possible.
Mitigation: If a problem is minor, it may be better not to risk repairing an old instrument. Spare parts should be replenished when used so that a complete set remains available until the last unit is launched. If repairs are necessary and some parts are missing, aggressive efforts will be needed to locate them. Some parts may still be in stock at the company or within NASA. POES project contractors have acquired new parts suppliers when the original parts were unavailable and undertook the formal flight qualification of the replacement parts.
Updating Ground-Support Equipment
Problem: GSE is often ignored when an instrument waits twenty years for flight. Computers used for structural analysis and to control instrument tests and process science data when the instruments were built probably have been superseded numerous times. Equipment needed for environmental testing, including special fixtures, thermal-vacuum chambers, and thermal-vacuum targets, may not be found.
Mitigation: Implementing a realistic GSE refresh plan is essential. EBay can sometimes be a source of antiquated equipment. The POES project has modernized GSE when sufficient spare parts are unavailable to repair existing GSE.
Meeting the Challenge
As the POES experience shows, maintaining old flight hardware is challenging. A variety of programmatic and technical issues must be dealt with to successfully launch flight hardware decades after manufacture.
Early planning for the possibility that many years may pass before an instrument is launched can help avoid or mitigate problems later on. That may happen to many science missions, not only those that feature instruments built for multiple flights. In today’s era of constrained budgets, such delays are likely to be common. The lessons of POES can help those other missions meet the challenge.
About the Author
 |
Karen Halterman has been the POES project manager since 2001, interrupted by five years starting in 2006 when she served as the Magnetospheric Multiscale project manager. |
