Back to Top
GALEX: Managing the Unexpected

Download PDF

By James Fanson

 

They say that good things come in small packages, and this has certainly been true for NASA’s Explorer Program. Explorers are among the lowest-cost missions flown by NASA, but they can pack a big scientific punch.

The Andromeda Galaxy, or M31, is our Milky Way’s largest galactic neighbor. The entire galaxy spans 260,000 light-years across—a distance so large, it took ten GALEX images stitched together to produce this view of the galaxy next door.

The Andromeda Galaxy, or M31, is our Milky Way’s largest galactic neighbor. The entire galaxy spans 260,000 light-years across—a distance so large, it took ten GALEX images stitched together to produce this view of the galaxy next door.
Photo Credit: NASA/JPL-Caltech

Such is the case with the Galaxy Evolution Explorer, or GALEX, a mission designed to map the history of star formation over 80 percent of the age of the universe. Since its launch nearly nine years ago, GALEX has transformed our understanding of how and when galaxies formed over time. Along the way, as the team anticipated, several unexpected and intriguing scientific discoveries have been made. What we did not anticipate was the gauntlet of technical and programmatic challenges that had to be overcome to get GALEX into orbit.

There’s a widely held belief that smaller projects are easier projects. In reality, the difficulty of a project is measured by the ratio of available resources to the challenges being faced. Explorer missions by necessity have small budgets, so its critically important to assemble the strongest possible team and ruthlessly constrain the magnitude of the challenges. This is easier said than done. Take staffing for example. Smaller missions have fewer team members, so each is more crucial. One needs the sharpest individuals with unusually broad skills. It can be difficult to recruit such people in organizations where importance is measured by the size of the budget managed or the number of direct reports.

In reality, the difficulty of a project is measured by the ratio of available resources to the challenges being faced.

Similarly, smaller missions have fewer instruments and components, but each element therefore tends to be mission critical. GALEX is a single-string, single-instrument design with very limited redundancy. This has two immediate implications: each element must have high reliability, and virtually no component can be eliminated. One of the most important tools available to the manager is the ability to “descope” items in order to contain cost. When each element is mission critical, the descope tool is limited to accepting lower performance from an element rather than eliminating it outright.

Recognizing these facts, GALEX’s principal investigator, Chris Martin of Caltech, proposed a mission that was simple in concept and built around a team of experienced individuals and institutions. NASA selected the mission for implementation in late 1997. Thus set sail a hardy band of explorers into what would turn out to be an unusually stormy sea.

Keeping It Simple

The idea for GALEX was to fly a 0.5-meter-aperture telescope with a wide field of view together with two photon-counting detectors, one optimized for the near ultraviolet and one for the far ultraviolet. The detectors would simultaneously image a region of the heavens 1.5° in diameter via a dichroic beam splitter. A filter wheel would enable a grism—a combination of a prism and grating—to be rotated into the beam to produce spectra that could also be imaged on the detectors. The spacecraft would point the telescope at the desired location during the night side of each orbit and orient the solar panels to recharge the battery during the day side. Over a period of twenty-eight months, virtually the entire sky would be imaged. In practice there were some complications, such as not imaging stars bright enough to damage the detectors, but in general the mission design and architecture were quite simple.

An ultraviolet mosaic from NASA’s Galaxy Evolution Explorer shows a speeding star that is leaving an enormous, 13-light-year-long trail. The star, named Mira (pronounced my-rah), appears as a small white dot in the bulb-shaped structure at right, and is moving from left to right in this view.

An ultraviolet mosaic from NASA’s Galaxy Evolution Explorer shows a speeding star that is leaving an enormous, 13-light-year-long trail. The star, named Mira (pronounced my-rah), appears as a small white dot in the bulb-shaped structure at right, and is moving from left to right in this view.
Photo Credit: NASA/JPL-Caltech/C. Martin (Caltech)/M. Seibert (OCIW)

We also felt that the mission required no new technology, but we were to learn otherwise. To achieve the required ultraviolet sensitivity, we needed photon-counting microchannel plate detectors. These make use of specially prepared, thin, porous glass plates supported at their edges, stacked and held at an electrical potential of several thousand volts. An individual photon of light striking the front surface of the detector produces a shower of electrons at the back. The location of the electron shower is measured by timing the arrival of electrical pulses with an array of very high-speed electrical circuits. While these techniques had been used in earlier detectors, they had not been implemented individually or in combination on detectors of the size required for GALEX.

In the end the detectors proved very difficult to manufacture, even by the group at University of California-Berkeley, considered the best in the business at this type of device. By the time we collectively recognized this fact, it was too late to reduce the size of the detectors, so we persevered and accepted delivery many months behind schedule. We learned an important lesson: scaling technology is sometimes as difficult as maturing the basic technology in the first place.

Paradigm Shift

GALEX began implementation during the height of NASA’s faster-better-cheaper era, a period characterized by the desire to find innovative approaches to reduce development cost, even if it meant tolerating and managing increased risk. In keeping with this paradigm, the GALEX implementation plan featured many cost-saving aspects, some of which involved cost and schedule risk. Interestingly, GALEX was confirmed with what today would be considered an absurdly low level of cost reserves: 10 percent, or about $5 million in total.

What we did not see coming was the dramatic change in risk acceptance following the loss of the Mars 98 and other smaller missions around this time. These mission failures sent a shock wave through NASA, which responded by overhauling the underlying implementation processes to be followed by every mission under development. By 2000, the paradigm had firmly shifted from “faster better cheaper” to “mission success first.”

Many of the cost-cutting approaches taken by GALEX were no longer considered acceptable. The team came under major scrutiny by outside groups of reviewers trying to reduce mission risk. NASA attempted to compensate the team for the cost of these changes, but it was difficult to estimate what the budget ramifications of the new processes would be. In particular, we knew that buying risk down after the fact would be difficult, as key design decisions and part selection had already been made and implemented. It was an unpleasant transition for all concerned.

Another important lesson was learned: when faced with the dilemma of shoring up a flagging effort with new help at the expense of an added delay to bring that help up to speed, the right answer is nearly always to bite the bullet and bring in the additional help.

Manage What You Can

Since we did not have the resources to oversee the detailed design of every subsystem or component independently, we relied on the expertise of the team members to identify where difficulties required additional attention or assistance. This worked well in most instances, but we dropped the ball in one important area—the detector readout electronics. While the Berkeley team was busy solving how best to build the photon-counting detector assemblies, the detector readout electronics were being designed and prototyped by a single design engineer. There were clear indications that he was overloaded, so the Berkeley teams solution was to isolate him from outside distractions. Being close to Silicon Valley during the dot-com boom, it was difficult for them to hire qualified electronics engineers at university pay scales, and they didn’t want to lose their one key designer. Efforts to carry out peer review of his work or bring in outside help were resisted on the grounds that they would slow down the effort. We left the engineer alone. It was a mistake.

Close to the time the flight electronics were due to be delivered, the engineer suddenly resigned and left the university. When we looked at the state of the electronics, we discovered why: an important portion of the readout electronics didn’t work and contained serious design flaws. The designer had been misleading us and his management about the status of the development. It took a crash program working with Southwest Research Institute to develop a replacement element. Another important lesson was learned: when faced with the dilemma of shoring up a flagging effort with new help at the expense of an added delay to bring that help up to speed, the right answer is nearly always to bite the bullet and bring in the additional help.

Survive Bolts from the Blue

Every project faces unknown unknowns, things that cant be anticipated. These events call upon the resilience and creativity of the team to overcome. GALEX faced an unusually large number of “bolts from the blue.” One event on our mission particularly illustrates how extensive the consequences can be.

Our ITAR-Baby

In order to reduce cost, the spacecraft bus supplier, Orbital Sciences Corporation, selected radio equipment manufactured by a company in Britain. The S-band receivers and transmitter were delivered successfully, but the X-band transmitter was more challenging and took a bit longer to complete. Within weeks of the scheduled delivery, Orbital received a phone call from the company stating that they had declared bankruptcy and were being liquidated. If we wanted the incomplete X-band transmitter, we should show up
with a final payment and take delivery at their loading dock.

Artist’s concept of the Galaxy Evolution Explorer. Its mission is to study the shape, brightness, size, and distance of galaxies across 10 billion years of cosmic history. The 19.7-inch telescope onboard sweeps the skies in search of ultraviolet-light sources.

Artist’s concept of the Galaxy Evolution Explorer. Its mission is to study the shape, brightness, size, and distance of galaxies across 10 billion years of cosmic history. The 19.7-inch telescope onboard sweeps the skies in search of ultraviolet-light sources.
Photo Credit: NASA/JPL-Caltech

We dispatched a contingent to pick up the hardware and as much design documentation as possible, and returned it to the United States. We approached the radio experts at the Jet Propulsion Laboratory, who explained that what remained to be completed was the staking and tuning of the circuitry, something that could only be done by someone intimately familiar with the design—in other words, they couldn’t do it without the engineer who had designed it.

By this time the design engineer was employed with another company in Britain. We explored shipping the unit over to him but discovered that our export license was only for the company that had gone bankrupt; we could bring the radio into the United States, but we couldn’t legally ship it back out. The next best thing was to bring the designer to the United States.

This is when we discovered that the designer was a dual British/Iranian citizen. Export-control regulations prohibit providing technical assistance to non-U.S. persons. Orbital explored the possibility of obtaining a Technology Assistance Agreement from the State Department to work with the designer but given his Iranian citizenship, they were encouraged not even to apply.

The last option was to bring the designer to the United States, set him up in an empty lab with a soldering iron and oscilloscope, and let him complete the staking and tuning. At the end of this exercise, we inspected his workmanship and concluded that the unit had been rendered unusable.

Being very short on time to find a replacement X-band transmitter, Orbital identified a potential replacement unit on another NASA spacecraft in their clean room. Several weeks of negotiation produced permission for us to cannibalize this hardware. The only problem was that it operated at a different frequency than our National Telecommunications and Information Administration license specified, and it used a different modulation scheme. One would think relicensing to another frequency would be straightforward, but it turned out that the new radio transmitted in a part of the X-band spectrum reserved for “downward”-looking vehicles (toward Earth) while GALEX was an “upward”-looking vehicle. We eventually got a special non-interference-based waiver approved. This left only the modulation problem—the way GALEX’s data was packaged for transmission—which was solved by a crash program to build new demodulators for the ground stations located in Australia and Hawaii so the data could be unpackaged correctly.

Make It Work

The GALEX spacecraft before its launch in 2003.

The GALEX spacecraft before its launch in 2003.
Photo Credit: JPL

The team found creative ways to survive various other bolts from the blue during GALEX: a vacuum-chamber failure that back-streamed diffusion-pump oil and contaminated the spacecraft bus; another missions in-flight failure of the gyro we had selected, forcing the crash refurbishment of a replacement gyro we found; the sudden loss of liquid-nitrogen supplies on the eve of instrument thermal-vacuum testing because the Enron-driven electricity crisis in California shut down the liquid-air production plant.

Once every development challenge had been successfully overcome, we launched GALEX in April 2003 aboard a Pegasus XL rocket. The scientific return has surpassed our expectations. Looking back on the experience, I appreciate the tremendous training value of the GALEX development; just about every type of problem that could arise did arise. It taught me many lessons I’ve applied to missions that followed, and gave me a true appreciation that small projects can be just as difficult as the big ones.

Note: This work was carried out at the Jet Propulsion Laboratory, California Institute of Technology, under a contract with the National Aeronautics and Space Administration. 2012 California Institute of Technology. Government sponsorship acknowledged.

Related Links

 

About the Author

 

 

James Fansons twenty-five-year career at the Jet Propulsion Laboratory has spanned technology development, instrument development, and flight project implementation. He was part of the team that repaired the Hubble Space Telescope; let the team that produced the preliminary design of the Spitzer Space Telescope; and, as project manager, led two telescope missions (GALEX and Kepler) to launch and early science operations.

James Fanson

 

 

 

About the Author

Share With Your Colleagues