In November’s Virtual Project Management (PM) Challenge, Doug Comstock, Kristin Van Wychen, and Mary Beth Zimmerman offered practical advice on cost estimating and reporting.
The Virtual PM Challenge, Why Don’t They Just Give Us Money? Project Cost Estimating and Cost Reporting, is now available to view online. People who were unable to attend the live event can watch as cost experts Comstock, Director of NASA’s Cost Analysis Division (CAD), and Zimmerman, Branch Chief for NASA’s Portfolio Analysis Branch in the Strategic Investments Division, along with co-presenter Van Wychen, Senior Analyst in the Government Accountability Office (GAO) Acquisition and Sourcing Management Team, provide unique insights and candid advice concerning the critical roles of cost estimating and cost reporting. They talk to moderator Ramien Pierre about the ways in which taking a proactive approach to these activities can help project managers better control life cycle costs, maintain stakeholder confidence, and protect current and future projects.
After the Virtual PM Challenge, APPEL News sat down with Comstock and Zimmerman to learn more about the resources available at NASA to support project managers and the challenges projects may face as they tackle cost management from an estimating and reporting perspective.
APPEL News: In your Virtual PM Challenge presentation, you made clear that there are a lot of resources available to NASA project managers to support them in developing cost estimates and reports. If they could only take away one thing from your presentation, what would you want that to be?
Doug Comstock: I’m going with two things. First, they can download the NASA Cost Estimating Handbook for more background. But I think the number one thing would be that there are a lot of experts throughout the agency who are ready, willing, and able to help the projects with cost estimating. Points of contact are identified for each center in the presentation.
Mary Beth Zimmerman: I’d like them to take away that cost reporting explains to the Office of Management and Budget (OMB) and Congress what resources are needed to support successful projects. My hope is that this process is more transparent after seeing the Virtual PM Challenge.
APPEL News: If you had more time to talk to project managers about cost estimating or cost reporting, what would you want them to know?
Comstock: I would like them to know that there’s a huge amount of data out there on technical parameters and risks in addition to cost. This is data that we collect through our Cost Analysis Data Requirements (CADRes) and is available in the One NASA Cost Engineering (ONCE) database. I think that information could be very useful to project managers and their systems engineers because they can very quickly look at analogous projects to see what’s been going on with systems and subsystems and similar spacecraft.
Zimmerman: During the Virtual PM Challenge, we weren’t able to get into much depth on how reporting requirements are tailored to both the size of the project and where it is in its life cycle. Projects that are less than $250M in life-cycle costs have fewer requirements. In addition, reporting during formulation includes fewer elements than during development. I think it would be helpful for people to have a better sense of that. They can come talk to us if they have questions.
APPEL News: Mary Beth, during your presentation, Ramien brought up the truism, “Time is money.” You turned that around to “Money is time.” Can you tell us why that’s important to consider?
Zimmerman: When projects do their detailed cost estimates, everything is focused on the year that they spend the money. Working effectively with their business managers to make sure that we request the money from Congress in time for it to be appropriated and obligated is important for it to be available for them to spend when they need to spend it. This can help avoid schedule delays. It’s sort of an extra step that isn’t always considered, and this cash flow can really help a project keep on schedule.
Another way to look at this is in how available budget affects schedule. Budgets are developed, and pretty well set, a couple of years ahead of time. This means that the work that can get scheduled in the short run is limited by the dollars previously planned for this time frame. Keeping an eye out for what cash flow is looking like will help with devising schedules that make the most use possible of these resources. The important thing is to keep a feel for cash flow, both as you’re planning and executing.
APPEL News: Doug, you gave the Virtual PM Challenge audience a few ways to think about issues involved in estimating cost. We hope people will view the presentation online to see them all. But, for now, can you highlight one of your points here?
Comstock: There’s an example of a point estimate versus a probabilistic analysis in the presentation that we can put into everyday terms. Let’s say you have a 20-mile commute to work. If you catch every light and there’s no traffic, maybe you can make it in 25 minutes. That’s the record. When you leave the house, you may always have in your mind that you can make it in 25 minutes as your point estimate. That’s just inherent optimism. Before leaving the house, there are discrete risks that you may know of and take into account. You check the weather: is it going to be raining or snowing? You check the GPS maps to see which roads are backed up. You can consider those analogous to discrete risks that you can take into account in your analysis. But there are unknown risks out there, too. Let’s say you anticipated the rain, but there was an accident right in front of you in an intersection so you’re stuck there for an hour, waiting for the accident to clear. You didn’t anticipate that that was going to happen. So there’s a range of times on the commute, from 25 minutes to an hour and a half, and the average time may be 40 minutes. That’s analogous to the range of potential costs for a project. The fact is, you do that kind of analysis in your head all the time when you’re getting ready to drive to work. For a project, you’ve got to have a point estimate, but in order to really understand the risks associated with the project, a point estimate is not sufficient. You have to have a probabilistic analysis that takes into account risk and uncertainty.
APPEL News: You both provided a lot of practical advice in your presentations. Let’s say a project manager whole-heartedly follows all of it. What would you say is a challenge they may still face? And what should they do?
Comstock: Even when a project manager has been diligent about managing known risks and actively mitigating those risks, there are unknown risks out there. In the methods that we use for estimating the cost, we incorporate discrete risks to the extent that they are known. But our estimating also takes into account unknown risks through uncertainty. It is very important to have good communication channels and management systems in place so that when an unanticipated issue comes up, you’re equipped to deal with it promptly. And hopefully, given the construct we have for the cost estimating and the budgeting, there should be resources available for those unknown unknowns through the UFE [unallocated future expense].
Zimmerman: I’d say an inherent challenge in projects that are as complex as those NASA undertakes is in keeping a pulse on how the many moving pieces are shaping overall cost and schedule performance at the highest levels of the project. In between the life cycle reviews that Doug described, the cost reporting and the cost assessment that we do from our office is at a much higher level than project managers’ day-to-day management considerations. It’s valuable for project managers—and everyone involved with the success of a project—to be able to keep an eye on the top line as well as the details. I know it’s not an easy thing to keep doing that roll up. We all tend to focus on the things that are right in front of us. And the more complex and challenging the project, the harder it is to carve out that space to look at the big picture, but it really does help us all see the forest for the trees.
APPEL News: You generously took the time to participate in a Virtual PM Challenge. Why do you believe activities like this are of value to NASA’s practitioners and the general public?
Comstock: The Virtual PM Challenge is a really good vehicle for communication. Everybody at NASA is busy: they’ve got their heads down, focused on their project. So it’s a good opportunity for them to lift their heads up a little bit and benefit from information they might not have. From the perspective of the presenters, you get a large audience so there’s a lot of leverage for the message. And it’s nice that it’s captured by recording so that people who aren’t able to see it live can go back and look at it. For instance, I made reference in my presentation to another Virtual PM Challenge from a couple of years ago on the Joint Confidence Level (JCL). That’s still available for people to see and learn from.
Zimmerman: For NASA practitioners—which includes me—it’s very helpful to get up-to-date information about how other aspects of our program and project management are getting accomplished. For the audience beyond NASA, the Virtual PM Challenge is a means of sharing NASA’s lessons learned or ways of doing business. It’s been interesting to see how many people outside of NASA have participated in these Virtual PM Challenges. That, to me, is a measure of the value: that even if they’re not responsible for the specifics of NASA’s policies and procedures, people can take away some value from how we do business.
APPEL News: Thanks so much for your time today—and for the information and insights you shared during your Virtual PM Challenge. Before we go, is there anything you’d like to add?
Comstock: I want to remind people that we have a number of resources available to help them with cost estimating and reporting, including experts at all centers. We have a NASA Cost Estimating Handbook and a CADRe brochure. There’s also the ONCE database, which is a portal to the CADRe data and other tools. Civil servants and NASA contractors can request access to that database. People outside of NASA can access historical papers and other tools on the CAD website.
Zimmerman: I would encourage project managers who are thinking about how to handle cost and schedule reporting to take a look at their project’s pages in the agency’s annual budget so that they can see for themselves how the information they provide helps inform Congress and how this information changes as their project progresses through its life cycle. I think being able to picture how the information is going to be used helps a lot in developing it. NASA’s budget can be found by going to any NASA web page and clicking the Agency Financial Reports link at the bottom of the page.
The Virtual PM Challenge Why Don’t They Just Give Us Money? Project Cost Estimating and Cost Reporting can now be viewed online here or at the bottom of this page.
___
Doug Comstock is the director of NASA’s Cost Analysis Division (CAD), which has responsibility for NASA’s cost estimating policy and advancing NASA’s cost estimating methodologies, tools, and databases. Prior to leading the CAD, Mr. Comstock led the NASA Innovative Partnership Program (IPP) office. Previously, he was the NASA comptroller and the founding director of NASA’s Strategic Investments Division. Before joining the agency, Mr. Comstock spent four years with the Office of Management and Budget (OMB). He began his career in the private sector with General Dynamic’s Space Systems Division, and as Director of Engineering with Futron Corporation.
Mary Beth Zimmerman is the Branch Chief for Portfolio Analysis in the Strategic Investments Division (SID) within the Office of the Chief Financial Officer (OCFO) at NASA, where she leads agency-level analysis supporting strategic and budget decision making, cost reporting, and monthly portfolio assessment for the CFO and the agency’s Baseline Performance Review (BPR). Before joining NASA, Ms. Zimmerman was with the Office of Energy Efficiency and Renewable Energy (EERE) at the Department of Energy (DOE). She has also worked with the Alliance to Save Energy and the Energy Committee of the National Governors’ Association.
