EPISODE 96: AIRWORTHINESS

Transcript

CJ Bixby: First flights are exciting. There’s so much work to get your system ready, so much preparation to be ready as a team to execute. When you finally get through it, you kind of exhale.

In the case of X-59, we’re going to take that airplane eventually and fly it in places that we don’t usually fly research airplanes, so that communities that don’t usually get sonic booms can hear X-59’s version of sonic thump.

There’s sort of the everyday-ness of achieving airworthiness, of keeping your system records and your configuration in shape and doing all your inspections and stuff. But to achieve it for the first time, or for a new thing that we’re trying to do, we do have a process that we work through.

Deana Nunley (Host): Welcome to Small Steps, Giant Leaps, a NASA APPEL Knowledge Services podcast where we tap into project experiences to share best practices, lessons learned and novel ideas.

I’m Deana Nunley.

Excitement is building as NASA gets ready to begin X-57 and X-59 test flights in the coming months. There’s a lot of buzz about the initial tests to prove airworthiness of the X-57 Maxwell all-electric aircraft and the X-59 quiet supersonic experimental aircraft.

Today on the podcast we’re unpacking processes required to achieve “airworthiness” with Armstrong Flight Research Center Chief Engineer CJ Bixby. CJ, thank you for joining us.

Bixby: Oh, my pleasure.

Host: How would you define airworthiness?

Bixby: So, there’s sort of a general definition of airworthiness that even the FAA uses, where your system is in a condition for safe operation, but also kind of all the paperwork is done, so your configuration is baselined, and your records are maintained. And then all the people around the system are also sort of qualified and trained and experienced and proficient, right? So, people doing the inspections and the maintenance and the modifications, people who operate the system, and even people who participate in assessing the airworthiness need to all be appropriately trained, experienced, etc.

So, your system can be sitting on the ramp, engines running, perfect operating condition, but it isn’t airworthy unless everything else is in order. And so, we want all of that stuff to be in order, and the system and its components are in a condition for safe operations at an acceptable level of risk.

Host: What’s the process for achieving airworthiness?

Bixby: So, there’s sort of the everyday-ness of achieving airworthiness, of keeping your system records and your configuration in shape and doing all your inspections and stuff, and that’s sort of the everyday-ness of just maintaining airworthiness. But to achieve it for the first time, or for a new thing that we’re trying to do, we do have a process that we work through. At Armstrong, we call that the Airworthiness and Flight Safety Review process, and it consists of sort of three main reviews.

So first we’ll do a peer review, and that’s a really comprehensive review by a group of independent subject matter experts. So, they’re independent of the project, but they have subject matter expertise in disciplines that are critical to the project. And they are chartered to go look at the project and look at the analyses that the project has and their operational procedures and documentation, the hazards that they’ve developed, and their qualification test results, and anything to do with flight, ground, or range safety. That independent board takes a really good look at that, and then they come up with some findings and conclusions.

And then we have a management review. So, we have Armstrong senior leaders on this management panel, and there’s 200 or 300 years of flight test experience sitting around the table when the managers are there. And so, this independent group comes to this group of managers and says, ‘Here’s the project and here’s what they’re trying to do, and here’s how they’re trying to do it. And here’s what sort of our findings and conclusions about those plans.’ And at the same time, the project also is in that briefing, and they say, ‘We heard that briefing and we understand their findings. And here’s our plan to close out the remaining work prior to flight.’

And so that senior group of managers, leaders, takes all that in from both the project and this independent review board, and they formulate a recommendation that goes to the center director. The center director considers that and either approves the project to proceed toward flight or asks the project to do some additional risk mitigation. So, at this point, you’re not cleared to fly. You’re cleared to keep going toward flight.

There’s one more briefing to do before you can fly, so the project goes back and finishes up the work that they need to do to go to flight and finishes up any closure of action items and stuff like that. And they come back to this one final review. Now the board at this last review is senior management again, but there’s also a lot of peers represented at this briefing.

So, these briefings at Armstrong are typically very open, and we encourage people to come and participate in these briefings. So, project team gets up, presents the results of what they’ve been doing since the management review, and then they explain their goals and their plans for this flight or this block of flights that they’re asking permission to go do. And they talk about what they’ve done to close their open items, and then they give sort of this is the last look at the current assessment of the project hazards and risks. And if that is all acceptable, then the senior management signs off. The project team actually presents a form, a flight request form, and the senior management team signs that flight request, and that is approval to go fly. There’s a two-day waiting period, just to make sure everybody’s had a chance to think about what they’re about to do, and then they are approved to go fly after they’re through the waiting period.

Host: Throughout this process, how much of the emphasis is on risk assessment and mitigation?

Bixby: Well, this process is a risk-based process, so most of the FAA processes, they’re standards-based. And so, in a standards-based process, the theory of the case is if you build it like this, the risk will be acceptable. In our process, it’s risk-based. And so, our theory of the case is since it is built like this, if we operate it and inspect it and monitor it in this way, then the risk will be acceptable. So, the risk hazard identification, categorization, and mitigation is really at the center of this process.

Host: What are some hazard analysis best practices and procedures that NASA is developing or following?

Bixby: So, a couple things come to mind. At Armstrong, each of our projects has what we call a system safety working group, and that’s made up of the project manager, chief engineer, in our case, an operations engineer who holds that Tech Authority around airworthiness and the responsibility for the day-to-day airworthiness of the system. And the discipline leads, these are all project people, and they meet regularly as part of their project duties. And we really rely on that system safety working group to create that organizational safety that people need to be able to feel like it’s OK to speak up, or to say, ‘I don’t understand that,’ or ‘do you think that’? To have sort of healthy conflict, and to really think through how the system is working, not just how it’s designed to work, but also how it is working.

And that leads to the other thing that came to mind, which is at Armstrong, we ask each of our engineers to engage in what we’ve been calling system-level thinking. And that’s where you understand the subsystem for which you are responsible in the context of the system. So you understand your subsystem and how it works and how it is likely to fail, but you also need to understand how your piece fits into the system, and what its roles are in the system, and what happens to the system if your piece doesn’t perform as expected, and also how your piece interfaces with other pieces of the system, and what effects will other pieces have on your piece and vice versa. So, you’re in this constant state of kind of zooming in and looking closely at your subsystem and really knowing that subsystem backwards and forwards, but zooming out and looking at your system in the context of the system and how it interfaces with all the other pieces, and zooming out even further to see that entire system and how it operates.

And so, I encourage people to engage in that system-level thinking. We think that everybody is responsible for that level of system-level thinking because it really teases out some of the unexpected and undesigned for behaviors and helps you find those hazards, and sometimes even helps you find the mitigations for some of the hazards because you can leverage aspects of systems that weren’t necessarily designed in.

Host: CJ, is there an airworthiness approach that missions tend to adopt as a success strategy to fly?

Bixby: Yes. There’s a couple things. One is build up, so in aeronautics flight tests, we have the ability to fly a little and fix a little, and fly a little. We take off, we fly around, we land, we try some things, and then we land. And we can tweak things in between flights. And so one thing that allows us to do is that we can fly some relatively low TRL systems that way because everything doesn’t have to work right the first time, certainly the safety critical stuff does, but a lot of the mission systems that are necessary for mission success, they can come along slowly. And it’s sort of a flight test 101 technique to use that ability to fly and fix and fly to sort of plan our flight tests in a way that we kind of sneak up on parts of the envelope that we think might be tricky so that we don’t—if you think getting to 20,000 feet is for some reason maybe tricky for your airplane, you don’t go take off and zoom to 20, right? You creep up a little bit and you try things at different altitudes before you go to 20, say. So, we have that ability to sort of build up to the riskier conditions.

The other thing that we are able to do here, particularly at Armstrong, because we are on Edwards Air Force Base and we are sort of located in this big, restricted airspace, is isolate. So, we are able to operate our airplanes sort of away from people and buildings, and also other traffic. So, for instance, we have a small uninhabited aircraft system workspace here that is dedicated to small UAS. And so, we can fly a relatively unreliable small UAS in this workspace because it can’t do a lot of damage, it can’t hurt somebody or hurt something on the ground. And there’s no traffic. We can sort of sterilize the airspace and make sure that we’re isolated from anything that we could do harm to.

Host: How does achieving airworthiness for aeronautics missions compare with achieving flight worthiness for space missions?

Bixby: Good question. The processes are similar. I have very limited experience with space missions, actually, but I did go through the process with the Orion Pad Abort test. And as I talk to space folks about our process, what I find is that they’re a little more focused on requirements than we are. So, I would say the space folks definitely focus on their hazards, but they sort of try, because they’re trying to move all that hazard mitigation into the design process, it ends up in their requirements, where we are able because of the nature of aeronautics flight tests, we’re able to take a lot of that risk into the operational. And so, we end up very focused on hazards and less focused on requirements.

Host: Let’s talk a bit about your background. You’ve been a flight test engineer, right?

Bixby: Yeah. I spent about 13 years in the industry before I came to NASA. And I was a flight test engineer and a test conductor, first on high performance aircraft, and then on prototype aircraft. So, the job was working with pilots and discipline engineers, and getting an understanding of what the system under test is supposed to do, and figuring out how to measure it, and writing procedures, and ultimately executing those procedures and reviewing the data and writing the reports. And on the prototypes, I also did a lot of the integration and tests leading up to first flight. And I really loved that job, it was kind of perfect for me in a lot of ways. And I loved working with all the different disciplines and the pilots, and especially the mechanics and technicians, who work hands on with these machines, and just know them intimately in a way that designers don’t always get to know the machinery.

I just loved working with all those different people who knew all the different little aspects and trying to integrate all of that knowledge together to come up with the best plan. The other thing that I loved watching how the system came together and you sort of add subsystems onto subsystems, and things start to interact and hopefully work together. And you start to see that sort of emergence that happens when a system becomes greater than the sum of its parts. And the other thing that I loved was the issues that kind of come out of left field. Nobody saw that coming and we don’t understand why it’s happening. And working with lots of different people to try to troubleshoot that and sort of attack it systematically, and it was really like working on a really interesting puzzle with a whole bunch of really smart people every day, and being paid for it. It was awesome.

Host: How does your experience as a flight test engineer influence your perspective as Armstrong chief engineer, especially with the upcoming X-plane missions?

Bixby: Well, as the Armstrong chief engineer, I chair that Airworthiness and Flight Safety Review process that we talked about earlier. And having that experience as a flight test engineer gives me a whole lot of patience and some perspective that I think I wouldn’t have otherwise. I definitely know what our project teams are going through both intellectually and kind of emotionally. I mean, there’s a great deal of uncertainty and anxiety inherent in just getting all these complex systems to work together and become a bigger and more complex system. And then there’s uncertainty and anxiety just going through this process and sort of getting a wire brush from people that you respect, usually your peers and your management. And so I understand that and I understand the effort and the intelligence that it takes to even get to the point where they’re ready to start the review process.

I know the level of teamwork it takes to make it through the process, and I’m thrilled to get to help teams work toward that level of teamwork and work through this process. If you talked to me a couple years ago before I had this job, I would’ve never told you that it seems like my whole career was preparing me for this role, but here and now, that is exactly what it feels like.

Host: That is fascinating. What excites you most about these new X-plane missions?

Bixby: First flights are exciting. There’s so much work to get your system ready, so much preparation to be ready as a team to execute. When you finally get through it, you kind of exhale. It’s a relief actually to get through it, but you’re also so excited to look at the data and to understand how well the system matched your predictions. And I’m really excited and maybe a little bit jealous of the teams who get to go through that. But I’m also really excited about the technology that the X-planes are exploring.

And we’ve already learned so much about electrified power trains just in getting X-57 Mod II ready to fly. And I imagine that actually putting it in the flight environment will yield lots of really interesting lessons and lots of really interesting data. And it’s exciting to take those sort of first steps toward electrification of aviation.

The promise that X-59 represents about being able to someday get across the country in a couple hours, that’s pretty exciting to me as somebody who lives on the West Coast and has to go to the East Coast a lot. And it’s almost as exciting as getting the opportunity to understand how well the system matches predictions, and ultimately being able to, in the case of X-59, we’re going to take that airplane eventually and fly it in places that we don’t usually fly research airplanes, so that communities that don’t usually get sonic booms can hear X-59’s version of sonic thump, and judge whether that’s disturbing to them or not. And so, it’s really exciting to be able to bring the research out of our restricted airspace and into communities who don’t usually get to participate in flight research at this level.

Host: Thanks to CJ for being our guest on the podcast. You’ll find her bio and links to topics discussed during our conversation along with a show transcript at appel.nasa.gov/podcast.

We’d love to hear your suggestions for future guests or topics on the podcast. If you have a suggestion, please share your idea with us on Twitter at NASA APPEL – that’s app-el– and use the hashtag Small Steps, Giant Leaps.

As always, thanks for listening.