By Terry Little
My experience, both first- and second-hand, has been that people have misused prototyping almost as often as they have used it wisely. I will try and cite some of the ways I have seen people abuse the concept.
“Rope-a-dope”
I had a boss once who gave me some advice about how to get support for a new program. He said that you label some obscure aspect of the program as “high risk” when, in fact, you know that it is eminently low risk. Then, he continued, you get someone to give you limited money to support a risk-reduction prototype and, voila the prototype demonstration is successful! You then use the prototype’s success, ideally with videotape and loads of “data,” to secure funding for a major new program or project. The sad part about his strategy is that it often works. Technologists use it all the time as a way of getting funding that they could not get otherwise. I call the strategy prototype “rope-a-dope” because it is deliberately misleading. The legitimate use of prototyping is to find out something you don’t know — not to demonstrate something you do know. Others may differ with me, but I fail to see marketing as a legitimate use of prototyping — at least not when the government is paying the bill.
“Kluging”
Another pitfall is a belief that basic system engineering principles can go out-the-window when you design and build prototypes. There may be rare instances where “kluging” together a prototype (like my high school science fair projects) makes sense, but usually one should build a prototype with an eye toward making a smooth transition to beyond the prototype stage. There is nothing worse than having a successful prototype demonstration and then having to start again from scratch to build something that’s affordable and serves some useful purpose. Some years ago an Air Force program spent several hundred million dollars to build missile prototypes for a competitive “fly-off.” The prototypes worked just fine, but the designers had to completely redesign the missile to make it into something that anyone would want to buy. That redesign had some major cost and technical problems that almost led to the program’s demise. I am quite sure that had the prototype design been more thoughtful and systematic, the transition would have been much less painful.
out something you don’t know — not to demonstrate something you do know.
Risk-averse Beware
It’s OK for a prototype to fail. In fact, if there isn’t a non-trivial likelihood of a failure, then why build a prototype at all? The purpose of building prototypes is to reduce risk and, sometimes, to find problems that you can only find from a prototype. “Try-fail-fix-try-fail-fix…” is a legitimate and sound prototyping strategy, but hasn’t always been acceptable where I have worked. Perhaps NASA is different, but my experience is that “higher-ups” tend to be fine with risky ventures so long as the ventures succeed. It reminds me of people who are happy making high-risk investments so long as they don’t lose any money. There isn’t much that we can do about others’ attitudes except to make sure that all the higher-ups understand the risks and to regularly remind them as the effort unfolds. It is easy to get so mesmerized by a prototyping project that we lose our objectivity and become less than sober about assessing risk. It’s always a critical mistake to take a path and underestimate the number of opportunities to stumble along the way.
Seeing Forests Instead of Trees
Finally, I have seen plenty of instances where someone built a prototype of the wrong thing — for example, producing a system prototype when only a subsystem prototype was necessary. Building a prototype when a model or simulation would have yielded a similar result is also common. The missile program I mentioned earlier should have built seeker prototypes and tested them in hardware-in-the-loop simulations and captive carry. There was no real need to go to the expense and time to prototype the entire system, because 90% of the risk was in the seeker. However, the program succumbed to external pressures to shoot down an aircraft. The money and time required to do that stunt would have better gone to packaging the seeker prototype in a more production-representative configuration. It was that packaging challenge that later threatened the program. The lesson here is to carefully craft any prototyping effort to address the most salient risks or unknowns. Spending 90% of the money to address a 10% risk area is just not good use of taxpayer money.
Knowing Your Tools
Overall, I love prototyping as a tool. As with any tool, it’s important to use it wisely. (You don’t want to hammer a nail with a screwdriver.) When prototyping is the right tool, it can be a powerful means to identify challenges, reduce risk or prove a hypothesis. It is a superb way to learn what we don’t know. Prototyping can give us the confidence that there is a way ahead, or the knowledge that there isn’t — either way, it’s a worthwhile investment.
About the Author
 |
Terry Little is the Director of the Kinetic Energy Boost Office at the Missile Defense Agency. One of the most seasoned program managers in DoD, he is also a regular contributor to ASK Magazine. |
