Ask OCE — December 21, 2005 — Vol. 1, Issue 1
The advent of email and PowerPoint has, in some respects, eroded our culture of engineering communication. Neither is an adequate substitute for engineering documentation.If you have people working in a technical area, at times they have to capture the engineering that has gone on. You do that with an engineering memo, or a workbook, or a technical report — whatever you want to call it. You need to do that to provide an audit trail of decisions that can be reviewed and challenged by peers.
For the record
A technical memorandum is what we used to call it. We used to have a form for the engineers; they would put their summary at the top, list their assumptions and the boundary conditions next, and then go through the analysis — sometimes even including a summary of some of the equations involved, plus the pros and cons they had considered. All of that preceded their recommendations and/or summary of actions taken.
That became a part of the engineering file. Anybody could go back and review that or challenge it. You could say, “Okay here is the analysis.” You could give it to another person or to another group and have them validate it or critique it. It also stood as a good way of handing off information about work that had been done to a newcomer on the project.
Over the last ten or fifteen years there has been a gradual erosion of the discipline of that kind of engineering documentation. A lot of it has to do with the introduction of email and PowerPoint, which are tremendous tools for communicating but not for engineering documentation.
But does it really matter?
I was on the Failure Review Board for the Mars ’98 failures. The Mars Climate Orbiter failure was ostensibly caused by a metrics conversion error that led to a navigation failure. We were getting the navigation data by tracking the spacecraft to calculate the trajectory. The data that we got from the spacecraft augmented the data from the ground — but there had been some inconsistency noticed during the mission well before the failure at Mars orbit insertion.
When we observe an inconsistency during operations, our practice is to use an engineering reporting system, called an Incident Surprise Anomaly (ISA) report, to record the discrepancy. It’s only one page long, but it’s a formal report. Once it’s submitted, it gets tracked. During the course of a mission there may be 100 or even hundreds of ISAs. Once you write an ISA it becomes a permanent record in the system. It gets reviewed. Its ongoing status gets reviewed. Its closure gets reviewed. People have to buy off and say, “Yes, this Incident Surprise Anomaly, whatever it was, is understood now. We’ve taken the following steps to prevent it from happening again and we’ve corrected whatever it is.”
In the case of the Orbiter, the person who noticed this problem didn’t use the ISA form. He wrote an email message to the person that he thought could solve the problem. That person got the email message, and he looked at it and worked on it for a while. Then his boss gave him something else to do that seemed to be of higher priority than working on the problem outlined in the email.
The guy who noticed the problem thought he was doing the right thing. He wanted the problem taken care of quickly. He sent the email out. The email went to the right place. The guy who received the email probably could have solved the problem. But he didn’t. It got forgotten.
If that message had been generated as an ISA, it could not have been forgotten. It would have become a permanent part of the engineering documentation. In our failure review report, we described the problem something like this: “Communication channels among project engineering groups were too informal.”
No one would argue that open communication is critical to project success. But communication comes in many shapes and sizes. We need to reinforce the distinction between the need for rapid communication and the need for engineering documentation, which creates products that can be peer-reviewed and leaves an audit trail for engineering follow-up and close-out.
John Casani worked at JPL for more than 48 years.
