Participants in the NASA Virtual PM Challenge on independent assessment included (L to R) Director of the Strategic Investments Division in the OCFO Cristina Guidi, Deputy Associate Administrator for Policy and Plans for HEOMD Greg Williams, Program Executive, Program and Project Management Division of the OCE Ellen Stigberg, Director of the Integration Management Office for the ARMD Tony Springer, Deputy Associate Administrator for Management of the STMD Prasun Desai, Deputy Associate Administrator for Programs in the SMD Greg Robinson, Senior Advisor to the NASA Associate Administrator James Ortiz, and moderator Ramien Pierre.
Photo Credit: NASA APPEL
In a recent Virtual Project Management (PM) Challenge, Dr. James Ortiz and leaders from across the agency discussed NASA’s new decentralized independent assessment process.
Independent assessment is a project management tool used to supply unbiased information on program or project progress to key decision makers as a means of enhancing the probability of success. At NASA, the independent assessment process includes analysis of schedule, cost, and technical risk at major project milestones. For years, the agency relied on the Independent Program Assessment Office (IPAO) to conduct its assessments. That changed in 2015, when NASA updated the independent assessment function by disseminating it from a central office to the four mission directorates. In doing so, the agency entrusted responsibility for the process to those most invested in the success of NASA programs and projects, with the goal of further ensuring the likelihood of mission success.
The role of independent assessment and the benefits of the agency’s new approach were discussed during the Virtual PM Challenge No One Grades Their Own Homework: Independent Assessment Under the New Decentralized Model. Over the course of the event—which featured live interviews, video segments, and a Q&A panel—moderator Ramien Pierre spoke with former IPAO Director Ortiz and eight other key agency personnel involved in managing the implementation of the new model: Deputy Associate Administrator for Management of the Space Technology Mission Directorate (STMD) Prasun Desai, Director of the Strategic Investments Division in the Office of the Chief Financial Officer (OCFO) Cristina Guidi, NASA Associate Administrator Robert Lightfoot, Langley Research Center (LaRC) Deputy Chief Counsel Pete Polen, Deputy Associate Administrator for Programs in the Science Mission Directorate (SMD) Greg Robinson, Director of the Integration Management Office for the Aeronautics Research Mission Directorate (ARMD) Tony Springer, Program Executive, Program and Project Management Division of the Office of the Chief Engineer (OCE) Ellen Stigberg, and Deputy Associate Administrator for Policy and Plans for Human Exploration and Operations (HEOMD) Greg Williams. The Virtual PM Challenge is now available to view online.
Now Senior Advisor to the NASA Associate Administrator, Ortiz served as director of the IPAO for six years and remains a key figure in transitioning the agency to the decentralized construct. He and his IPAO staff have played critical roles in ensuring the agency continues to conduct effective independent assessment reviews during the transition phase and in helping the mission directorates and other agency leaders implement the new model. Because of this, Ortiz has a unique perspective on the value of independent assessment and why it makes sense for the agency to adopt a new approach at this time. APPEL News sat down with him after the Virtual PM Challenge to learn more about the facts and misconceptions regarding the independent assessment process at NASA.
APPEL News: During the Virtual PM Challenge, you and your co-presenters talked extensively about the transition to the new independent assessment model and the value of the new process. If the audience could take away just one thing from your presentation, what would you want that to be?
James Ortiz: What I’d like the agency’s project management community to take away is that the new independent assessment model is an evolution of the way NASA does independent assessment. Just because the process is changing, it doesn’t mean the value of independent assessment is changing. It will continue to be a very important part of program and project success at the agency. And we will continue to mold the process, to adjust it, and to improve it as we go forward to ensure it best supports the success of our agency programs and projects.
APPEL News: What is the biggest misconception about the new independent assessment process that you would like to see corrected?
Ortiz: The biggest misconception is that somehow or another it is less robust now: that having the mission directorates take ownership of the process makes it is less rigorous. Some people believe that the mission directorates will be more lenient than the IPAO, which was doing it before. I don’t agree. I believe the rigor will continue because at the end of the day, as we mentioned during the Virtual PM Challenge session, independent assessment is done for the benefit of the projects. It is designed to help the projects succeed. Everyone at the agency wants projects to succeed, so putting them through a somewhat stressful activity that in the end produces a stronger project approach, which reduces risk, is good for the projects and it will continue. So that’s the misconception: that somehow or another, because we don’t have a centralized approach and the mission directorates are doing it instead, the process won’t be as rigorous as it was before. That’s not true.
APPEL News: There must be some risk associated with introducing a new approach to independent assessment at NASA. Why institute a new process when the original one was effective?
Ortiz: An important thing for people to know is that although there are risks in introducing a new model, we have put in place mitigations and a built-in system of checks and balances to mitigate those risks, as I explained during the Virtual PM Challenge. As for why we decided to do this, the mission directorates are very influential with the programs and projects, so having them own independent assessment should facilitate programs and projects accepting and responding to the SRB findings as compared to when the process was conducted by the IPAO, as IPAO was further removed from project execution. Robert Lightfoot improved the process by distributing the responsibility and accountability for independent assessment with the mission directorate management, who are ultimately responsible for the success of the programs and projects. This way, they are better positioned to take full advantage of independent assessment as part of their oversight of their programs and projects.
Senior Advisor to the NASA Associate Administrator James Ortiz and moderator Ramien Pierre.
Photo Credit: NASA APPEL
APPEL News: Was there anything not addressed during the Virtual PM Challenge that you wish had been?
Ortiz: I think we touched on all of the things that were really important. One of the things that we wanted from this session was for project managers to hear from different people and to hear the different perspectives. To hear Robert’s perspective, to hear my perspective as the focal point for the transition on behalf of Robert, and then to hear from the people implementing it today. They heard from mission directorate management, who were quite candid about how it’s working so far and what lessons we have learned and what is yet to be done. They also heard from the OCFO, because they are playing a significant new role in the new process. And they heard from the legal folks who are helping us make sure that all of the Standing Review Board [SRB] teams conducting the independent assessments are as pristine as possible to maintain the integrity of our independent assessment efforts. So I think we touched on everything. The Q&A section of the Virtual PM Challenge took us to another level because we explored some areas that we did not cover directly during the event. Overall, I feel that for an hour and a half session, we covered a lot of ground. I do worry, though, that people who attended the Virtual PM Challenge but are not from the agency may not understand some of the basics about independent assessment. In that case, I would encourage them to Google the SRB Handbook to do a little bit of reading in order to get some of the basics. [The NASA Standing Review Board Handbook can also be accessed here.]
APPEL News: Speaking of that, many non-NASA agencies registered to attend your Virtual PM Challenge. Do you have any advice for organizations that may be considering instituting a similar approach to NASA’s for independent assessment?
Ortiz: I recently had a conversation with someone from the Department of Commerce who was asking some basic questions about independent assessment at NASA. Previously, I had conversations with the Smithsonian Institute about it as well. My recommendation to them is if they’re interested in learning about how we do independent assessment at NASA, the SRB Handbook is a good place to go. There, we have captured what we know about independent assessment. Also, we’re very candid about the fact that we continue to evolve: it’s an art.
I do think that some of the principles about how we do independent assessment could be adapted and transferred to other applications. For instance, how to keep the review team membership independent and what we consider are the attributes of good reviewers. Also, how review teams can operate and report autonomously and unimpeded, so that their findings are clearly communicated to the people that need to hear and act on the findings, such as the projects or programs and their leadership. Those principles are quite universal. So my advice to an external organization is to examine the principles and then think hard about how they could apply them to their domain. The implementation might be different, but the principles could be applied to many other applications.
APPEL News: What was the biggest challenge you faced in transitioning the independent assessment process from the IPAO to the mission directorates?
Ortiz: It was primarily a leadership challenge for me, personally, because I was and am still very much invested in the people from my old organization. The IPAO personnel have been outstanding. Their dedication and professional attitude after the announcement about the new process is the reason we were able to complete all of the reviews that were part of the transition and to capture and transfer the information and corporate knowledge to the mission directorates. They worked with the mission directorates to ensure a smooth transition of activities so that agency programs and projects were not affected. They did all of this while working on finding new assignments after the IPAO closed. So that was a challenge for me, personally, and for all of them. I was quite encouraged to see how well the organization performed and how easy it was for them to find new—and very good—assignments. It is a testament to the caliber of people that were part of IPAO. Overall, it was a challenge as we were working through it, but I think the outcome has been positive.
APPEL News: You said the independent assessment transition will be complete by the end of the year. What’s next in store for you?
Ortiz: I’m working now for Robert and Lesa Roe (his deputy) as part of their staff in the office of the Associate Administrator, and I’ve already begun doing other duties for them. They are keeping me busy! It’s quite interesting to get a different view of the agency, to go from independent assessment into other areas where I can help Robert and Lesa overcome agency-level challenges. For me, it’s been an interesting and a growth-producing experience.
APPEL News: Is there anything else you would like to mention about independent assessment or the transition process?
Ortiz: One thing probably worth talking about is transitions in general. This was a significant agency transition—and a successful one. Something we learned that is not new, but needs to be reemphasized, is that to do a transition like this successfully takes people coming together with the objective of helping the agency move forward. You have to take off your personal or organizational hat and come together as a unit to go off in a new direction for the good of NASA. If you do that, and if you have good communication, then the transition will be successful in the end and it will be done in a manner that is not overtaxing. It will go smoothly. This is crucial, particularly in significant transitions like this involving changing personnel and responsibilities in a function that directly impacts the operational tempo of programs and projects. It is like working on an engine while the engine is running. It takes working together for the good of the agency’s future.
The Virtual PM Challenge No One Grades Their Own Homework: Independent Assessment Under the New Decentralized Model can be viewed online.
To learn more about independent assessment at NASA, download the latest edition of NASA’s Standing Review Board Handbook.
To receive email notifications about upcoming Virtual PM Challenge sessions and recorded session availability, please subscribe to the Virtual PM Challenge email list.
____________
Dr. Ortiz is a Senior Advisor to the NASA Associate Administrator. In that role, he is leading the transition of the agency’s independent assessment function for spaceflight programs and projects from a centralized construct to a new decentralized model. Dr. Ortiz was the Director of the IPAO from 2009-2015 at NASA Headquarters. As such, he had responsibility over the independent assessment of the agency’s highest-priority spaceflight projects. Prior to joining NASA in 1990, Dr. Ortiz served in the U.S. Air Force as a flight test engineer for electronic warfare systems on high performance aircraft. He holds a PhD in Electrical Engineering (EE) from the University of Houston, a Master of Engineering in EE from the University of Florida, and a Master of Science in Systems Engineering from the Naval Postgraduate School in Monterey, CA. He received his bachelor’s degree in EE from Los Andes University in Colombia, South America. He is also a graduate of the Senior Managers in Government Program at the Kennedy School at Harvard University. Dr. Ortiz is a member of the U.S. Government Senior Executive Service.
