Ask OCE — February 8, 2006 — Vol. 1, Issue 4

NASA would benefit from consistent, agency-wide project management practices and standardized product development criteria across its centers, according to a December report by the General Accountability Office (GAO). GAO called upon the Agency to support its flight systems and ground support projects with comprehensive, cross-center policies that encourage best practices used by successful project developers in other industries.

GAO said that in the past, NASA experienced difficulty in realizing cost, schedule, and performance objectives for some projects due to a lack of clearly defined project requirements and an inability to correctly quantify resources. Though NASA recently revised its policy concerning development of flight systems and ground support projects, it still lacks key criteria and major decision reviews that are the hallmarks of what GAO calls “effective knowledge-based acquisition frameworks.”

A knowledge-based approach to product development, as defined by GAO, “enables developers to be reasonably certain, at critical junctures or ‘knowledge points in the acquisition lifecycle, that their products are more likely to meet established cost, schedule, and performance baselines and, therefore provides them with information needed to make sound investment decisions.”

According to this model, the Non-Advocate Review (NAR) meets this standard of a knowledge point, but later technical reviews such as the Critical Design Review (CDR) do not, because they are not major decision reviews with the scope of a NAR. GAO also pointed out that NPR 7120.5c does not establish success criteria for the CDR.

GAO found that while NASA does employ some design best practices, its policies do not require projects to demonstrate high levels of technology readiness before winning formal approval at the NAR. By failing to establish minimum levels of technological maturity, NASA increases the probability of costly design changes at later phases of product development, which are typically more costly than changes that take place earlier in the design process.

GAO observed that NASA centers do not have a common standard for product development criteria. As a result, some centers have product development criteria that represent best practices in a knowledge-based acquisition framework, while others do not, and centers report varying degrees of knowledge and knowledge levels at key decision points in the design process.

GAO also noted that the absence of consistent, agency-wide project management criteria may result in design and manufacturing decisions that are based on inconsistent information. NASA relies on senior project managers and systems engineers to employ good project management and systems engineering practices. However, in light of “the loss of seasoned project managers and systems engineers and the decline of in-house systems engineering and technical capabilities,” GAO concluded that this reliance “could be problematic.”

Read the GAO report: NASA: Implementing a Knowledge-Based Acquisition Framework Could Lead to Better Investment Decisions and Project Outcomes.