By Lee Graham
As I walked though the gates at the Naval Research Laboratory (NRL) for the first time, I couldn’t help but smile to myself. Six years previously, while doing a tour at the Office of Spaceflight at NASA Headquarters, I had driven by NRL many times and had often thought, “That sounds like an interesting place to work.” Little had I realized then that, as a deputy project manager for NASA, I would get a chance to be the senior NASA manager on site.
The final integrated vehicle is tested in the anechoic chambers at the Naval Air Station-Pauxtent River.
Photo Credit: Marshall Space Flight Center/Jose Matienzo
With the Russians in the assembly critical path, the International Space Station (ISS) program had decided earlier that it needed to develop an “insurance policy“ to protect vehicle development. The program reviewed many options and settled on the solution of slightly modifying an existing NRL flight system. The project became the development and delivery of the vehicle to provide low-cost propulsion for the space station and was to be called the Interim Control Module (ICM). Budget and schedule were predicated on existing NRL processes, minimizing the use of NASA processes and, most important of all, not imposing new and unique system requirements to “human rate” the vehicle. It was to be “used as is.” None of this was formally documented; it was mentioned by some senior NASA officials while visiting NRL, with a few other NASA personnel in attendance. This lack of formal definition would cause many of the management and design problems the project was to face. Spelling out those expectations in a written agreement would have avoided many of the requirements, design, and integration issues we were to see in the following two years. Or at least it would have given us ammunition to defend our position and start the discussions. It’s a lesson I’ll never forget.
I joined the project prior to the preliminary design review, and I had an overall understanding of the requirements the vehicle needed to meet. Armed with this knowledge, I met the NRL program manager, Al Jacoby; his deputy, Bob Towsley; and the entire NRL team within an hour of coming through the gate. As I talked to them about how they were organized and how they operated, a number of points became obvious to me:
- They were a confident and technically competent organization.
- They were a “skunk works”-type organization—very flat with a lot of delegated responsibility.
- Man, this is going to be fun. (Not, how do we integrate this fast-moving, rapid-prototyping, minimum documentation approach with the document-and process-heavy bureaucratic approach of the ISS program?)
- The use-as-is approach, using NRL processes, was probably overly optimistic, but it was a good starting position.
- The program-provided requirements were not very detailed and would require some evolution later.
As the project matured over the following several years, the requirements and the resulting design changed and evolved. And changed, and changed. Our budget and schedule never really stabilized. We eventually completed the vehicle—just in time for the program to mothball it, since the Russians did fulfill their commitments. We also ended up addressing virtually all the points I had seen at the beginning.
Our NRL teammates were also civil servants, so they wanted to be treated as partners, not contractors. I learned that they, and only they, could call themselves the “Nuts Running Loose.” In addition, while NRL was very competent, it quickly earned a reputation with us as an overly confident, even at times cocky, workforce. (Sounds similar to us.) The phrase I most often heard from NASA folks was “but we have lives at stake in our missions, they don’t.” When I got to know the NRL people personally and learned about their integrity and some of their flight history, I discovered that they often had literally tens of thousands of lives depending on their missions. So their confidence began to be understandable. This joint understanding of motivations was the start of mutual respect that began the true team-building we needed. But in getting there, I spent a lot of time soothing troubled waters and getting folks on both sides to understand the other’s viewpoint.
One day I received a call about a heated exchange between our NASA civil servant quality assurance representative and one of the outstanding NRL technicians. I called all members of the on-site NASA office together to get the details and to calm everybody down. I reminded our folks that the technicians in this particular area were very close to completing their tasks and, since there was no other immediate work in sight, were probably going to be laid off in the near future. It was therefore understandable that the technician in question would be on edge; we needed to be sensitive to that and act accordingly. The folks involved in the argument apologized to each other later that day, and we got on with the job (and the NRL technician was picked up by another in-house project and kept working).
On another occasion I was called and told we had a “problem” with some of our NASA-provided government-furnished equipment. I found that a small air conditioner had malfunctioned and dumped some condensate water onto our EVA handholds. It turned out not to be a major problem. We were able to fix the air conditioner so it wouldn’t happen again, log the incident, dry the hardware, and press on. But as I walked out into our “high bay” area, and with this increased sensitivity to water damage, I noticed our powered-up flight avionics and power decks sitting in the open, right next to our only set of government support equipment that we would have to use in pre-launch and day-of-launch processing. When I pointed out the potential of a water leak destroying this hardware, Al Jacoby reminded me that we were in a large building inside a larger building. It was literally a separate building with about six feet of separation between the roofs. He also mentioned that the odds of any leak finding our hardware in the nearly 100,000 square feet of production space were unbelievably remote.
I still felt uncomfortable and wasn’t willing to let it go. So Al got an estimate for protective covering. The cost-benefit trade-off was a no-brainer for me, so we installed it. Some of the NRL engineers good-naturedly joked to me about “Lee’s Folly.” Fast-forward a couple of months: a hurricane came up the East Coast and hit the Washington, D.C., area with 60 mph winds that tore off a ventilation cover. Rain poured in. The area of the inner roof right on the edge of one of the protective covers turned out to have a hole in it. Sober expressions replaced the laughter. In addition, people started to ask questions about what other protection could be put in place. I saw covers appear around non-NASA flight projects near us. They had never had anything like this happen before, and they probably won’t again, but they learned to trust our opinions.
On yet another occasion, during automated acceptance testing of one of our flight star trackers, we had an unexpected reset of the instrument. No analysis of the test data nor examination and additional testing of the flight hardware itself gave us any clues about the cause. We assembled a group of outstanding engineers from Marshall Space Flight Center, along with NRL ICM Chief Engineer George Flach and myself, and set off for the vendor. We spent two difficult days going over all the manufacturing and test records trying to find the cause. We eventually settled on a “most probable” cause that fit all the circumstances, since we couldn’t find definitive data. We unanimously agreed to make the unit our spare and only fly it when absolutely necessary. Even more importantly, the in-depth technical discussions increased both groups’ respect for the other’s technical abilities.
I was shocked at how fast NRL could make changes to the flight hardware and software. We could discuss a change in the morning, make a decision, and have the new drawings and procurements out by that afternoon. Because of this, it became obvious that the NASA folks needed to be tied in tightly with their NRL counterparts to stay aware of what was happening, and why, and to concur with the changes. Constant contact quickly became our standard way of doing business and helped keep the project on track. I, and many others, spent a lot of time on the phone making sure we stayed coordinated across disciplines.
The ICM is inspected at the Naval Research Lab to ensure the adapter support posts are level with each other.
Photo Credit: Marshall Space Flight Center/Jose Matienzo
Getting this fast-moving, bare-bones-approach project integrated into the program was a lot more difficult, however. We ended up creating new documents and products that the program absolutely needed but we hadn’t planned on. We also ended up killing some documents that the program team felt were initially needed, but only after a lot of face-to-face meetings and teleconferences. They were often deleted because the program people didn’t understand the functionality or interfaces of the ICM to ISS, and the documents they wanted didn’t make sense.
Just as important to me was how we worked with our NASA counterparts across the Agency. We were moving so fast that a delay of a few days, while people went off to study minor issues, would be devastating to the project. So I sat down and developed a set of parameters that would allow us to make some changes and corrections without having to ask permission.
For Material Review Board dispositions, I developed a list of delegated authority criteria. The agreed-to approval authority would not require additional off-site NASA approval or review if
- The change didn’t affect the required performance of the item.
- No new or re-do of any analyses of any kind were required.
- There was no impact to any ISS interface(s).
- There was no impact to any NASA-provided hardware interface(s).
- There was no effect on the form, fit, or function of the item.
As you would guess, a certain level of trust had to exist between NASA and NRL, and even between the NASA on-site folks and other NASA personnel, to allow this to work. This trust was absolutely critical to avoid stop-work conditions on the shop floor. Just as critical was a desire to make the project a success. We did eventually reach an acceptable level of trust and commitment. On one occasion, we were scheduled to lift some major structural components for final positioning when NRL discovered their lifting equipment needed recertification. Neither they nor I were willing to lift the flight hardware with uncertified hardware, so a quick resolution was needed. I hastily called some friends up the road at Goddard Space Flight Center who agreed to loan us the necessary hardware. I signed the necessary paperwork that morning, loaded it in the back of my pickup, and headed back to NRL. The lift went off without a hitch.
The biggest impact to the project came from the changing requirements, though. The program had originally created the ICM project to ensure the ISS could continue to function should the early Russian modules not show up. With political changes in the wind, and as the technical requirements evolved from the original idea of use-as-is, those of us building the vehicle started to see a fluctuating set of requirements from the program. The vehicle design had to change accordingly. This continued for most of the project life cycle. At the build level, we really had no requirements control because the program was responding to external factors. As a consequence, we developed a lot of hardware that wasn’t used in the final configuration. Parts of wiring harnesses, power converters, and other black boxes were designed, built, tested, and certified but never used. It was a great experience but not the optimum project management model.
A number of us NASA human space flight folks gained a wealth of experience working directly with the NRL. We were able to get hands-on experience developing and assembling a complete space flight vehicle, including being test planners and directors on some development and integration tests. We also were able to gain valuable experience developing an Agency-wide distributed team that included outstanding can-do folks from Marshall, Johnson, and Goddard, among others.
I learned a lot. If I had to pick the top five lessons, they would be these:
- Trust is the key to making any distributed or cross-organizational team work. It may take time to build between people working together for the first time, but it is always worth the effort.
- Requirements stability is critically important. There will always be some level of requirements instability, but we need to be ever vigilant to avoid requirements “churn.”
- Communication, communication, communication. E-mail is okay, but the telephone is better. Face to face is best.
- Project management folks need to be aware of and sensitive to the mind-sets and experiences of the different organizations supporting their effort. An encouragement approach might work well for one organization yet be an utter disaster for another. That’s one reason why project management is as much an art as it is a science.
- It is possible to integrate a “skunk works”-type project into a “normal” program, but it requires both sides to be flexible. Just as important, it requires someone that understands both organization types and processes to guide the integration process.
Would I do it again? In a heartbeat.
About the Author
 |
Lee Graham joined the Constellation Program office after twenty-two years of working at Johnson Space Center or in support of NASA. His experience includes participation and leadership roles in both large program offices and in small “skunk works” project offices. His expertise includes systems engineering and integration, safety and mission assurance, and real-time operational flight support. |
