For NASA programs using risk-based independent verification and validation (IV&V), detailed NASA-developed/supported simulation of key flight phases provides deeper government insight and certification ability.
A detailed, independently developed simulation of deorbit, entry, descent, and landing (DEDL) phases proved invaluable to the Commercial Crew Program (CCP) in providing government insight into and certification of the flight systems. The independent DEDL simulation also informed the CCP by allowing government investigation into areas of concern with the flight system design and/or operation during DEDL. Rapid post-flight support of a CCP spacecraft in-flight anomaly was possible with the independent DEDL simulation, demonstrating its potential application for future flights.
Recent application of risk-based IV&V on NASA flight programs such as CCP and the Artemis Human Landing System relies heavily on government insight and certification of contractor-developed flight systems. One element to insight involves government review of contractor-provided detailed verification objectives demonstrating flight system acceptability relative to program-defined constraints. Review of contractor-provided flight system descriptions and results alone provides a limited understanding of key flight system elements and operation and may not provide sufficient insight for certification.
Lesson Number: 31901
Lesson Date: August 2, 2020
Submitting Organization: NASA Engineering and Safety Center
HIGHLIGHTS
LESSONS LEARNED
- The process of developing an independent 6 degree-of-freedom DEDL simulation, using contractor-provided detailed descriptions of the flight system and its operation and using the simulation to independently generate critical, detailed verification objective results, provided the depth of understanding needed for government insight and vehicle certification.
- Once developed, the independent DEDL simulation allowed NASA to investigate challenging cases in the flight system operational space without distracting the contractor during final flight system development and operations unless an issue was uncovered.
RECOMMENDATIONS
- NASA programs using risk-based IV&V to certify a contractor-provided flight system such as CCP and the Artemis Human Landing System should independently develop detailed simulations in key flight phases (e.g., DEDL, ascent, and rendezvous/proximity operations) to model, investigate, and confirm detailed verification objective results to provide the insight necessary to understand the system for certification.
- Interaction between government and contractor engineers should be encouraged during the development process to confirm that simulation models and results are consistent between the government and contractor simulations.
Consult the lesson learned for complete lists.
Jeremy Shidner
Credit: NASA
NASA Human Landing System Deorbit, Descent & Landing Mission Segment Lead Jeremy Shidner on the importance of this lesson learned:
We were very successful building relationships with SpaceX for the certification of Dragon, so we wanted to take that lesson forward. Looking at a design and asking questions from your experience may not always be pertinent; every vehicle is unique and different. You need to get your hands dirty and understand the specifics of the vehicle and how it performs, so you know where the risks are.
If you’re trying to build a good relationship and all you’re able to do is just ask a lot of questions, that can be counter to building that relationship. What we found helpful was having a good relationship with the partner which allowed us to fundamentally ask good questions from an intimate understanding of the details and the vehicle performance.
Having the ability to dig in and run analysis and do independent verification and validation is important, so that when you do talk with the partner, you’re talking on equal footing. This process involves answering some questions before going to the provider and making everybody smarter so that when you do get to requirements verification, you are confident with the results and you’re not blindly accepting verification.
This approach is applicable to future NASA programs and projects as younger engineers on a program will be able to dig in and improve their understanding. Getting hands-on experience quickly and efficiently is useful. Otherwise, the younger engineers may be scratching their heads about what are the details they’re trying to evaluate. Having the ability to do some of this independent analysis gets simple questions answered and makes the NASA side smarter.
Creating a flight software ourselves helps our understanding of how their system works and builds that implicit understanding that you get from being hands-on. So, part of the partner’s design for working with NASA should be to independently build everything up from scratch. Having an independent software model helps to focus where the most risk is and to have this risk-based insight process to manage the biggest risks. This method is a good tool for getting the knowledge out to people in an efficient manner. Anytime you can do something hands-on, you’re going to learn a lot more than by just reading a document. And then in order to smartly allocate resources, we’re going to have to put a whole bunch of risks on the books. When we get smart enough though, we might find a particular risk wasn’t actually necessary to put on the books. There’s something to be said for trying to be smarter before you start identifying and calling out risks.
The simulation was a very useful tool to have. There will be some level of discomfort in verifying vehicles meet requirements, but when it came to the DEDL requirements verification that we were looking at with this IV&V simulation, the program found it much easier to reach acceptance. The simulation helped everyone to trust what SpaceX was saying to get the vehicle certified.
Related Resources
About Human Landing Systems Development
Spotlight on Lessons Learned is a monthly series of articles featuring a valuable lesson along with perspective from a NASA technical expert on why the lesson is important. The full lessons are publicly available in NASA’s Lessons Learned Information System (LLIS).
If you have a favorite NASA lesson learned that belongs in the spotlight, please contact us and be sure to include the LLIS Lesson Number.
