Mistake-proofing techniques built into system processes can aid in preventing human error when engineers and test operators make time-critical decisions in unique circumstances such as dangerous weather or limited supplies.
Following June 2022 testing at the E-1 High Pressure Component/Engine Test Facility at NASA’s Stennis Space Center, test operations personnel noted during post-test inspections that the E-1 Cell 2 Triethyl Aluminum-Triethyl Borane (TEA-TEB) injection supply tubing was warm to the touch. This indicated there could have been a TEA-TEB reaction occurring inside the tubing, and personnel determined that air was coming from FireX Master Facility Panel (MFP) Station 6.
TEA-TEB is a pyrophoric liquid which ignites when exposed to air and water and is often used as an ignition method for rocket testing. It is stable when stored and operated with dry, inert purges until ready to be injected into rocket components for ignition.
FireX MFP Station 6 was originally designed and installed to service valve actuation of all stand FireX systems and is therefore branched and routed throughout the stand and all three E-1 test cell locations. The TEA-TEB storage and run systems had been installed for a previous test project in Cell 1 in 2016. The design tapped into a nearby gaseous nitrogen system downstream from MFP Station 6 for the required gaseous nitrogen purge and pressurization, but the MFP drawing and panel labeling was not changed to reflect the additional purpose of Station 6 for TEA-TEB purging.
In preparation for Hurricane Ida in September 2021 the FireX MFP supply was swapped from gaseous nitrogen to high pressure air (HPA) to conserve nitrogen usage at E-1 and had remained in this configuration after hurricane season to conserve nitrogen usage. E-1 FireX MFP Station. 6 is used to purge TEA-TEB, and by using this purge/press system with the HPA supply, air was introduced into the TEA-TEB system in the June 2022 test. No personal injury or facility damage was sustained.
As part of the corrective actions, the air source was removed, and the MFP drawing was relabeled for clarity. The labels were also physically added to the TEA-TEB cart door and to the MFP Station 6 panel for operators in the field.
Lesson Number: 31401
Lesson Date: June 28, 2022
Submitting Organization: Stennis Space Center
HIGHLIGHTS
LESSONS LEARNED
- Fabricating and installing a separate, dedicated gaseous nitrogen-only station exclusively for the TEA-TEB system would be ideal.
- Infrared cameras could be used to detect thermal heating in TEA-TEB system tubing, enabling the test conductor to monitor system conditions prior to releasing personnel back to the test stand facility.
RECOMMENDATIONS
- Ensure proper labeling of system source panels, and especially pay attention to restrictions of commodity sources to avoid unintentional contamination.
- Always use processes that establish constraints or required verifications prior to reactivating systems which have been inactive for a period of time.
Consult the lesson learned for complete lists.
Christina Zeringue
Credit: NASA
NASA Stennis Space Center Chief Safety and Mission Assurance Officer Christina Zeringue on the importance of this lesson learned:
Temporary configuration changes are a normal part of an operational facility. It is also common to have small ‘minor’ changes within a facility that is operational for a long period of time and is needed to support many different functions simultaneously, as is the case with a rocket test stand serving multiple test projects in different design/construction phases and using many different propellants and pressurants. Each change should be carefully managed and considered, and thoughtfully documented.
This lesson is a reminder that no one sets out to make a mistake on purpose. If it had been obvious in the drawings and panel configuration that the gaseous nitrogen system was feeding the Triethyl Aluminum-Triethyl Borane (TEA-TEB) system, the engineers would have made different choices. There is also no substitute for a full system walkdown, ensuring you have considered all downstream branches and pathways before making a substantial change to the supply system.
Managing changes and making good design decisions is always easier when resources and schedule time is plentiful, but quickly becomes a very different challenge when facing an emerging difficulty such as adverse weather and supply shortages — often combined with limited personnel resources to perform the needed work. Mistake-proofing systems and designs at the very beginning is our best defense and protection against these kinds of human errors. Strive for the ideal engineering solution whenever possible, rather than settling for a ‘make-do’ solution.
Related Resources
E-1 TEA-TEB System Contamination Lessons Learned
NASA’s Stennis Space Center Completes Year of Testing and Site Activity
Spotlight on Lessons Learned is a monthly series of articles featuring a valuable lesson along with perspective from a NASA technical expert on why the lesson is important. The full lessons are publicly available in NASA’s Lessons Learned Information System (LLIS).
If you have a favorite NASA lesson learned that belongs in the spotlight, please contact us and be sure to include the LLIS Lesson Number.
