By Haley Stephenson
Using a wiki platform, the NASA Software Engineering Working Group has set a new precedent for collaboratively authoring, reviewing, and enabling interactivity for handbooks at NASA.
The 135 software engineering requirements for NASA projects are listed in a small, blue booklet, seventy pages long, called NASA Procedural Requirement (NPR) 7150.2. It is one of many NPRs at NASA for disciplines including finance, launch services, legal, human resources, and program/project management. They are the “how to and must do” for work at NASA.
An inherent challenge of writing NPRs is stating what is required to readers without detailing how those requirements should be implemented. Make it too prescriptive and they might discount better-suited processes for their project. Too brief and they are left uncertain. In the case of 7150.2, almost every requirement is one sentence long. For instance, requirement 2.4.1 on software verification reads, “The project shall plan software verification activities, methods, environments, and criteria for the project.” This single line of text is followed by a brief note and then requirement 2.4.2, software validation.
“We [kept] it pretty lean,” said John Kelly, NASA’s program executive for software engineering, who was involved in devising the requirements, “but people were asking us for more information.”
The solution: write a handbook. A sort of hitchhiker’s guide to the NPR, a handbook doesn’t impose additional requirements; it’s meant to be an assist. One prominent and highly regarded example is the NASA Systems Engineering Handbook (NASA/SP-2007-6105), which sits on the bookshelves and desks of systems engineers at NASA and beyond. The NASA software engineering community voiced their need for a similar resource. Their leadership responded with an electronic, browser-based handbook run on a wiki platform—a dramatically different approach for collaborative authorship and review that has sparked interest across NASA.
“Somebody described it almost as a knowledge capture type of activity,” remarked Kelly on the process of creating the handbook. “It’s as if all of a sudden NASA really knows what it knows.”
Not Wikipedia
In 2011, on a detail to the Office of the Chief Engineer at NASA Headquarters, Glenn Research Center’s Kevin Carmichael was assigned by Kelly to lead the development of NASA’s first-ever Software Engineering Handbook (SWEHB).
Typically, creating and updating a handbook is a complicated and time-intensive process involving a plethora of e-mails, edits, version control, and spreadsheets called comment-resolution matrices. It can take years.
But the SWEHB wasn’t to be the usual handbook. Inspired by mobile apps and e-magazines, the software engineering working group leadership wanted the handbook to be electronic. To help refine their options, a member of the group brought in a web-savvy young professional from Goddard Space Flight Center, software engineer Jon Verville.
Verville took the initial concept for the handbook and built on it to arrive at a robust and flexible solution. “It needed to be something that was broadly accessible, irrespective of platform—you know, mobile, desktop, laptop,” he explained. If you have the Internet, you can access the handbook, whereas selecting a proprietary solution or singling out a particular type of hardware like an iPad would limit that accessibility. Instead, Verville recommended a browser-based solution built on a wiki platform.
“For a lot of us, ‘wiki’ meant Wikipedia,” said Carmichael. “We all had some familiarity with that, but we didn’t see how it translated into what we were trying to do.”
The proposed wiki platform wouldn’t look like Wikipedia but it would have similar functionality. It would enable collaborative authorship of the handbook by a defined group. Furthermore, the platform provided simple, yet powerful features such as commenting, revision tracking, database capability, and hyperlinking. For example, publications often reference outside materials that are useful, but perhaps not easily found. “Then it’s an exercise for the reader to hunt that down, and it can be time consuming,” Verville said. “It’s just one more of these little hurdles that people encounter while trying to find the information they need.” Providing direct hyperlinks to online resources significantly reduces this barrier.
If a hyperlink or a reference changes, that’s okay, explained Verville. In paperback, a reference would have to be located everywhere it appears in the handbook, updated page by page, and reprinted. The wiki’s database feature could accomplish the same goal with a few keystrokes and the click of a button.
Additionally, the platform would allow users to select a requirement from the directory or search for it. They could get what they need at the required level of detail and get back to work, or serendipitously discover other useful information that might help them further. Inside the wiki-based handbook Verville proposed, requirements like the aforementioned 2.4.1 on software verification would have a richer, more accessible story to support them.
Convinced, Carmichael and the SWEHB team dubbed Verville the handbook’s architect and committed to using a particular wiki platform called Confluence.
The Beta Handbook
Each section of the SWEHB provides six areas of information per entry: the requirement, its rationale, guidance for implementation, notes for small projects, associated resources, and related lessons learned.
The authoring team consisted of seven members, amounting to three and a half full-time employees, distributed across Tennessee, West Virginia, Pennsylvania, and Washington, D.C. They met in person only three times over the two and a half years they worked together and primarily coordinated through weekly teleconferences. “This was the first time that I managed a distributed team like that,” said Carmichael. “It worked exceedingly well.”
The team came up with a six-step process for authoring and releasing each part of the handbook for review. Throughout each step, a built-in work-tracking system monitored their progress so they could all see who was working on what and if the work was under way, completed, or not yet started. The process steps were the following: author a section, send it to a technical writer for review, send to Carmichael for review, send to Kelly for review, then a final review by the section’s original author, and then post the content to the wiki online.
This didn’t make the content final, however. The final version of the handbook would have to undergo a technical working group review, an agency-wide review, an Engineering Management Board review, and then receive final approval from NASA Chief Engineer Mike Ryschkewitsch. If the SWEHB were printed, this would have meant months to years before practitioners would be able to see any part of it. But the SWEHB wasn’t printed, so Carmichael and his team had another idea.
“We had a lot of people who wanted help; they wanted guidance immediately. They didn’t want to wait two and a half years,” said Carmichael. “So we put stuff out there and we just called it ‘beta.’” This meant that anyone at NASA could see the SWEHB being built from the ground up, section by section.
The beta handbook’s rigorous six-step authorship and collaborative review process provided its contents an acceptable level of pedigree. Making the beta version available online also enabled anyone in the agency to review it and provide input in the form of comments on any of the published pages. Approved edits were made quickly and a team member would e-mail the individual who suggested the change to make sure the revision met their initial intention. “That was a different way of doing things,” said Carmichael. “Whenever we made edits … people could see them immediately.”
“In an old process, you couldn’t do that. It would have been so labor intensive that all this interaction would have been impossible,” said Verville. “For instance, in one month, we received over one hundred comments from software experts across NASA, our team made over three hundred approved online edits, and we had over two thousand visits to the site from our review team. There is no way this could be replicated through anything but the web.”
Posts and Threads
When the team pushed out the call to review the handbook, reviewers were given the option to put their comments into a spreadsheet and e-mail them back or post their comments directly to the bottom of the appropriate wiki page. Carmichael estimated that less than 3 percent of the comments were delivered by spreadsheet.
“The vast majority of people found it to be much easier to put comments directly into the wiki, and people fed off other people’s comments, so it became a good discussion,” said Carmichael.
Throughout the review process, members from all ten centers used the commenting space available at the bottom of every page of the handbook to provide their input. In total, nearly eight hundred comments were collected.
“There’s a little bit of threading here,” said Verville, pointing at his computer screen while clicking through a section of the handbook’s comments. “See, this person at Johnson responded to this person from Dryden …. It’s very contextual. You’re leaving a comment right on the page where you’re reviewing the information, and so when other people went to review it, it wasn’t about the whole handbook. It was on this particular section of 135 sections where they were putting their comments.”
“The inputs we got were fabulous,” Kelly said about the commenting feature. “That made it so much richer than just one person pounding away [on revisions] and not being able to reap some of the inputs from various people who contributed to the wiki environment.”
A Paradigm Shift
The SWEHB is the first of its kind at NASA. Approved by Ryschkewitsch on February 28, 2013, the handbook serves as a successful test case for authoring and reviewing handbooks in a digital environment. Throughout the process, the wiki approach was met with some skepticism and caution, as it did not follow NASA’s traditional print-publication process. As a result, the team worked diligently to gain stakeholder trust and buy-in to accommodate their electronic process, while ensuring the SWEHB would meet NASA’s requirements without compromising its standards.
Interest in capturing organizational or community knowledge using a wiki platform is growing among groups internal and external to NASA. Within NASA, the SWEHB team has been approached by a number of groups who are interested in learning from their process and implementing it in their own organizations. Outside NASA, the Department of Defense sponsored a global collaboration among members from dozens of organizations to create the Systems Engineering Body of Knowledge using a similar platform. They released their final version in late 2012.
The handbook is also representative of how the next generation of employees at NASA will work, explained Carmichael. Like the introduction of e-mail or social media into the workplace, implementing a new or unfamiliar paradigm is often met with some resistance. “Younger people in the agency will readily adopt stuff like this. The software engineering community will readily adopt electronic media like this,” said Carmichael. However, because the SWEHB is not a book that sits on a shelf or a physical document, Carmichael anticipates the agency will see somewhat of a transition period for current employees—especially those who are more comfortable with traditional handbooks—to acclimate to this particular electronic resource and the others that are likely to follow. “It’ll take time to overcome that,” Carmichael said.
In June 2013, a version of NASA’s SWEHB will be made publicly available online at swehb.nasa.gov. This will be beneficial to NASA and its international, industry, and academic partners who build components integrated into NASA missions. If those components have software, they must meet 7150.2. “They have a big interest in knowing what’s in 7150.2 and the reasons behind the different requirements,” explained Verville. “So there’s a big [potential] for people who are our partners to get something out of this as well, maybe have feedback or have a stake in it being relevant to them, and for them to be able to comment on it, too.”
“It’s a nice canned resource for people to pull information from,” said Kelly. Typically, if someone had a question about a requirement or topic, they’d have to track someone down to find what they needed to know. The handbook offers an alternative. “You don’t have to know somebody to ask something and get
something in a real piecemeal fashion,” explained Kelly. “You can go to one place and boom, it’s all there. The knowledge of how you do things—how to successfully do things.
“It’s a lot easier than me fumbling through my files in my office,” laughed Kelly.
 |
Explore the Software Engineering Handbook online at swehb.nasa.gov, or by scanning this code. Access is already available at NASA facilities through this URL. The site will be made publicly available in late June. |
More Articles by Haley Stephenson
- The Toothbrush Hack (ASK 49)
- Something to Shout About: Bloodhound Supersonic Car (ASK 45)
- Weathering Ike (ASK 44)
- Factoring in Humans (ASK 41)
- NASA Knowledge Forum 2: Knowledge in Projects (ASK 40)
