Best Case for Case Studies: An Interview with Jennifer Stevens, Chief Knowledge Integrator

Jennifer Stevens, Chief Knowledge Integrator at Marshall Space Flight Center (MSFC), supports Center and Agency Chief Knowledge Officers and interfaces with the NASA knowledge community.

She develops policy and procedures at MSFC to define and implement Systems Engineering (SE) policy and practices. Specifically working in the Lessons Learned Committee, she has been involved in developing MSFC knowledge strategy and planning, coordinating and integrating grass roots knowledge-fostering initiatives, and creating and teaching courses designed to cultivate a knowledge asking and sharing culture. Stevens developed the Marshall Distilling Process, a cross-center team that works to operationalize lessons learned that the Lessons Learned Committee has approved for implementation.

Susan Snyder, Managing Editor of NASA Knowledge Journal, sat down with Stevens to talk about her most recent work: her case study efforts that she has been leading at MSFC and the story of the cross-center collaborations that helped launch this rich learning offering at her center.

What originally sparked your interest in bringing the case study learning approach to Marshall?

A couple of years ago, the Marshall CKO team, led by Dale Thomas, the former Associate Center Director, Technical and Chief Knowledge Officer at MSFC, participated in the Return to Mission Success (RTMS) forum at Goddard Space Flight Center and saw how successful it was. Dr. Ed Rogers, Goddard’s CKO, leads RTMS, a multi-day learning event that has enormous impact on employees’ understanding of NASA, the organizational culture at GSFC, and the way things work. RTMS uses the case study approach to many of the learning modules. My team and I experienced first-hand the engaging, deep learning through case studies. It really enriches the classroom experience. Ed Rogers is also a renowned expert in developing and facilitating case studies for NASA, and he was willing to support other organizations in running case study-based learning sessions.

Was this your first experience being in a case study discussion?

In my Masters and Doctorate programs in Engineering, we had to write case studies, but we had no guidance. We would be given a topic like “management style” and had to look at two companies and write about their management style. We would write about it in a scenario and then analyze it. Unfortunately, we were so focused on the course topic when developing the scenarios that we would write the case study to support the analysis results. We didn’t use case studies as a knowledge sharing tool. And they could be pretty boring too.

How did the case study learning approach develop at your center?

Well, when we left Goddard’s RTMS program, and we unanimously decided to make Marshall an organization that learns through cases. Our challenge was that none of us were experienced in teaching people how to develop a case, and we needed writers to be skilled in that ability. So, Dale Thomas and I asked Ed Rogers to help develop and present a course on developing case studies. Ed and I developed the learning plan and course agenda, and Ed provided guidance, suggestions, material, and content. Ed taught the 1 ½ day pilot course at MSFC in June 2014. He’s a wonderful teacher who also modeled good teaching practice for me. I recruited interested people from different organizations to take the pilot. We videotaped Ed’s presentation for future reference. I then worked with 3 writers over the next 6 months to write 2 case studies. After that, it was only a matter of time and preparation before we did our first full workshop in April 2015, with 6 teams starting the case study writers experience, and 5 teams finishing in August of that year.

How did you approach your first case study pilot session design?

The first thing I did was take what Ed had written on “How to Write a Case study”, and evaluate how to teach that to a class. I put structure around teaching those steps. I incorporated the key things he taught us in the pilot, such as concept mapping and interviewing skills. Those are two things I didn’t have a lot of experience with, so having Ed teach us and then having the video to review later, helped a great deal. We adopted Ed’s teaching design. We kept in mind that we are teaching a specific kind of case study, so we made sure that we illustrated different kinds, and explained how they are used and how they differ.

These teaching or decision-oriented case studies are set up so readers can do the thinking. You give the reader information and background, but you don’t lead them to a specific answer. Case study-based learning is basically a simulation of a situation without having to go through it for real. Instead of learning principles from a text book or learning in the job environment while everything is going on, you can go through the simulation and not have to pay attention to all the noise. The richness of context helps people think through the situation and come to conclusions. That’s where the learning happens.

When we say we are doing “decisional” case studies at Marshall, I have to be careful because it doesn’t always have to be about a decision. It can just be about a situation. They might better be described as “instructional” or “teaching” case studies. We have to keep the length and focus so that we can use them in about a 1 hour segment in our other courses.

Explaining our end goal for using the case studies lets the writers understand why this particular type of writing. It explains why we talk about what the story should look like and how they themselves might be teaching it. Students learn about all kinds of case studies, and how those other forms can be used in teaching, storytelling, or knowledge capture. They learn how to deconstruct them, so they learn how to look for information and how to organize it. Organizing a narrative is a skill that people need to learn, so they can construct how they put evidence together in a case.

How did you market your course to get trainees to come?

For the pilot I advertised through normal training channels, plus I let the students who attended RTMS at GSFC know. I also invited people from the History Office and the Engineering Directorate that I thought would be interested. One person who had taken RTMS at Goddard and fell in love with the idea of writing case studies came prepared with several ideas she got from talking with her management months before the class was announced. I was pleasantly surprised by the interest in writing given that we have so many engineers here. In fact, Jim Turner, the Technical Assistant in the Engineering Directorate, which employs more than 50% of the people working at the center, took part and is a real advocate now. You just never know where the interest will come from.

For the full course I again used normal training announcement channels, but also invited specific senior people to ensure I had experienced senior partners. Dale Thomas is an example. He took the full course where I paired him with two younger students. I found that senior partners really have to be committed personally to producing the case study. Dale made time to work with his junior partners. He invested his time in them. I’m dedicated to making sure I have advocacy and senior people to pair up with junior people.

What surprised me was that people came out of the woodwork. They told me, “I like to learn that way” and “I have ideas for several case studies I’d like to see written”. In one instance, one of our senior managers wanted to present at a case study at our Real World Marshall Mission Success course to help structure his discussion. A pair of writers, one from his organization, crafted a case study originated from a detailed outline he had made. He has since asked us to help him write another one. We will pair someone in our next course with the senior writer on the first case to develop this second case with this manager’s attentive guidance, knowledge, and invested commitment.

If I signed-up for the course, what would I be committing to?

You would attend the two-day Case Study Writers Workshop for training in understanding the different types of case studies, the purpose and use of decisional case studies, the basic process and schedule, researching and interviewing techniques, and crafting and editing for effectiveness. Then you would participate in the Case Study Writer’s Experience, which is actually optional, but highly encouraged. The Writer’s Experience is a team-based, hands-on case study writing effort with coaching throughout the development.

Did you have a formal approach to working with the Writer’s Experience teams?

I try to keep it to 16 to 18 weeks. I meet with them first every week, and then space it out to every two weeks, and then to every three weeks. I try to be flexible to the business need and the writers’ work loads. That can slow things down. Work gets in the way. The process gets bogged down. It gets hard, but if the teams don’t meet, they don’t get the work done. Major project events affected our progress this year. There were two teams I coached that got stalled by the senior partner going on a detail. On one I ended up partnering with the junior member. The other writer I had to encourage independently. Fortunately his senior partner was able to do enough before he left so that the junior partner was able to finish with an excellent case study.

What are the key parts of a case study?

First there is the opening in which you set the stage. Then, you set the context- the project, location, who is doing what, and the history, issues and conditions. It funnels the context from the broader picture down to the specific situation. Next, you illustrate the decision point or conundrum. This is where writers, having written the opening and scenario details, illustrate a decision point or conundrum. At a certain point, the writers may include a Stop and Think or a You Make the Call question to prompt the reader to pause and reflect on the situation. The “stop and think” is good, but sometimes, it creates too many breaks in the story. It is sometimes better to stop and end. Or not ask a question. It depends on their goal with the story’s use and scope.

The trick isn’t how to fill up the space but how to write a concise package. I help the writers refine their narrative. We’ll still adjusting the process. In editing, you can’t love something so much you can’t leave on the table. And our writers get a massive amount of information after interviewing key people. It takes the partners some in depth discussions between them to craft a story they’re both happy with.

How long are your typical case studies?

Typically about three to six pages with pictures and references. However, there’s no right or wrong. For instance, the “Columbia Return to Flight” case study is 17 pages with three sections- there were two Return to Flights. The length of the story can be short or long, depending on the case and what you are trying to teach with the case.

Are there skills or competencies that help someone be successful in writing case studies?

There are so many skills that go into it. I never realized how many. Ed Rogers developed his process for writing a case study as an individual, and he isn’t an engineer. He’s more of an Organizational Development (OD) person. He’s mentally not as constrained by process as us technical-types might be when trying to write a targeted case study on a schedule in teams. Ed is highly skilled in writing case studies, so the tangential skills or competencies that we teach in this course weren’t obvious to us before we started. He just does them naturally.

The first thing you have to do when you write a case study is know a little about the area you are writing about. We need the writing team to be conversant in an area that they may not have a lot knowledge of, so we pair more experienced practitioners with less experienced practitioners to enable better knowledge flow. At least one partner should be conversant about the subject.

The second important skill is interviewing skills. We teach this to some extent. Interviewers have to listen for understanding, not judgment. But, they also have to manage their time to get the facts needed for the story. There is a balance between letting a person talk, which may result in a keen insight, or asking guiding questions to stay focused.

Sometimes the best story isn’t the one you thought you were going to write. In some interview situations, more case studies topics can emerge. The compelling one that emerges might become the case study Then the team has to go back and interview again to build out the new case study. Interviewers have to listen for the things that correlate with this different focus – then they have to go back and reconcile all of the data gathered to tell the story. They have to be flexible. They might really have wanted to tell a specific story. We are not natively used to giving up our biases. It’s a matter of listening for the right information, not the bias-confirming information. People just have to restrain themselves, be aware and listen. It’s a skill, and it’s not a cultural norm.

The writing team interviews people who were intimately involved in the situation or case. It can be surprising that emotions can be so strong even after a decade. Interviewers have to learn to be sensitive to repressed trauma, but not be afraid to ask about the emotional experience. The emotional experience is a huge part of the real world experience, and perhaps harder to appreciate than the technical details.

Note taking skills help. Ed Rogers taught us a unique type of interviewing/note taking using concept mapping, where the interviewer draws a map of the different concepts that are being gathered in an interview. The end product is a map of related thoughts. It is much more visual and memorable than a long word document. We tried to use the concept mapping, with mixed success because it is a different way of thinking. Some of our younger people like it; linking things helps to remember them. I tried it and it really does help.

Another skill is the ability to pull all the information down to a coherent point so people can make decisions on different options– It’s presenting a conundrum without giving an answer. I do let the writers write questions, but I encourage them to avoid leading the reader to the writer’s conclusion. They need to write to a point, to bring the learner to reflect on what is happening. I have a lot of people who want to give “the answer”. In reality, there usually is no single answer. That is why it is a case study and not a mishap investigation. What worked or didn’t work might not be the only answer to what could have worked or would not have worked.

The final skill is so hard. It’s editing. Editing out great information to tell a clear story. It’s ”crafting” a narrative. It’s more like “Case Study 201.” Editing is a refining skill. This is what we struggle with – and what our writers are struggling with. At the end of the writers experience we may or may not have mastered story crafting. I sometimes tweak the case myself and ask them to review it. I’m okay with the students getting to a certain level of polishing that is good enough. Good enough is when other people can read the case, maybe come to a classroom discussion, and learn from what the writers have developed, even if it is an imperfect final draft.

When the case development is in final draft, does it go through a review process?

Yes, and the review process can be a challenge. We fact check rigorously, and ensure that graphics are not copyrighted anywhere. We coordinate with Legal, Export Control, and other management to ensure that the case is consistent with policy and guidance.

The draft always goes to key players, the interviewees. It can go through any number of iterations. We really strive to get balance in the case. People’s perceptions of the same event can differ widely. Sometimes that is the whole point of the case study: two sides see something completely differently, and that’s why things went awry. We try to represent both sides without blaming. In the end, the case study is not a news report or historical document of record; it’s an narrative of recollections and factual evidence that reflects our history and common experience.

As a side note, one of our cases is about two project leaders with two very different perspectives. During the review process both parties finally learned what the other person had been thinking, how they saw the situation. Pieces of the puzzle started falling together, leading to greater understanding on both their parts about what was really going on at the time.

Did you have any help from others at Marshall or outside of Marshall – to review the case writings, to help develop the facilitation guide, or working group questions?

Ed Rogers was our go-to coach. He was always a big help—he would tie in on some of our meetings. His guidance in setting up the pilot class and providing feedback was essential. He provided encouragement and advice on next steps. Ed came down for the pilot, and he tied in on the next year’s class by Adobe Connect. He especially helped with how to edit, how to wrap-up the case, and performing the interview. It was great to have him support that; he is a great resource for learning how to engage in the finer details of case study writing.

Does anyone not succeed in writing a case study?

Some. It helps to have some background in what you are writing about. They have to learn a lot if they don’t know the topic or understand the context at all. They get fire-hosed with so much information, so it helps to have some kernel of knowledge to put structure on the case. Without some knowledge they might never figure out when to end the questions and focus on a topic. The deeper they go, the deeper it gets.

Some people come into it wanting to do a case study that is something other than what we teach (instructional case studies). The course never gels with their idea of what it was going to be. Also, I had one student who added the class the day before we started. I didn’t have time to examine his knowledge areas and find a suitable partner to pair with him. He had specific ideas about what he wanted to do, which didn’t align with anything on the docket at the time. This, combined with a limited amount of time he was allowed to give to the course resulted in his not finishing.

The course also takes time: time to interview, time to think about what you have been told, time to write. Work can overtake the time they can spend. They can’t complete the development, but at least they learn the principles. Even if you just take the workshop, the exposure to the skills, methods, and the cases we review are enriching and applicable to everyday work.

What are the types of learning that you are currently focusing on?

One we are crafting now is on highlighting the different perspectives of a Principle Investigator running an innovative, low budget project that the Center wants to manage at a programmatic level with the systems engineering tools. So, how do you marry small, agile project approaches with big program management techniques when you aren’t used to doing that? The project needs the programmatic approach, because when you are in the middle of the project, you aren’t always looking at the future. By working at a programmatic level, you will have the tools and process you need to plan for the future because there is a process. But putting effort into conforming to PM and SE requirements can be seen as overly burdensome to small projects with very little budget. Flexibility is essential to be a good project manager, but there are reasons why we have a structure for managing work.

There is also a case being developed on Teaming, concerning a young, inexperienced team that is given a hands-on project. It illustrates the challenges they have coming together as a team. That one needs more work, more focus. We have one more about a project that suffered communication problems because of a lack of role definition and adaptation to a changing budgetary environment. It is more of a conundrum story than a specific decision point narrative.

Do you choose topics and POCs before the class?

I make sure I have them. If students come in with some topic or idea, that is great. In our last class we had one student come in with an idea for a case on the Return to Flight after Columbia. It is fascinating. I shared that with Patrick Johnson, Human Exploration’s CKO. The gentleman who came in with the idea was one of our almost-senior people who had gone through RTF. He took the course because he enjoyed learning from case studies – reading them really engaged his curiosity. To foster idea generation, we seed the class with experienced people, who usually propose ideas in class. Also, management brings us topics. I love it when people are already engaged with learning from case studies and want to share the knowledge they consider important through storytelling.

What I would really like to do is get more cases in procurement and other institutional support areas. I know they infuse some lessons learned into some courses, but I think we could really amplify our business process courses with more case studies about the real world of NASA. I’m working on developing more ideas and POCs for those institutional organizations.

How many times have you run this course and what are your plans to run it in the future?

We ran the pilot in 2014, the first full course in 2015, and this will be the second year of the full course. I am revamping the design to improve it based on lessons learned these past two years. I am trying to get familiar with Adobe Connect because other Centers have expressed an interest in having this there, so I am toying with the idea of tying them in virtually, maybe next year. I can teach the course, but to run the program there, the post-class Writer’s Experience—where case studies are actually developed–has to be done at the host center because it is a high-touch process. There would need to be someone who can mentor students through the process without me being there. As for Marshall, we plan on continuing to offer the course every year. My hope is that we can enable the other Centers to adopt and adapt the course to augment their knowledge capture and sharing also.

Are the cases that were developed in the course offering being presented at Marshall?

Yes, at the Marshall version of Goddard’s RTMS. We call it Real World Marshall Mission Success (RWMMS), which we first offered in August 2015. Five of the 9 case studies presented last year were from the Case Study Writers Experience. They were very successful. We will continue a yearly offering of RWMMS using our Marshall case studies.

The writers themselves get to present their cases at RWMMS. This has had a great motivating effect on them to polish their cases to a level that others can learn from them. It has been great for our writers. They figure out how much they really do know about the subject, and it also brings up questions or comments that they hadn’t thought of or known about. They may decide to revise their case study to bring in a little more information or present a point in a slightly different way. One of our junior writers took his own initiative to dry run his case in his staff meeting. The branch engaged in a great discussion and brought up some points he then clarified in his case before when he presented it at RWMMS. Overall, it has been a privilege to watch their growth and the satisfaction at having produced something that is used immediately, and appreciated by their peers who are learning about the real world of Marshall as we strive for mission success.

What are the unexpected benefits of the case study approach?

Five things:

• The incredible amount of information each student gains about a subject, even if they were involved in the situation themselves. The learning can be incredible.
• The struggle to wrangle and distill all that information into a balanced narrative based on the evidence at hand. Isn’t that what we have to do daily at NASA?
• Cross-generational knowledge sharing, including senior members learning new skills from junior members.
• Contextual learning; it makes knowledge stick.
• Exposure to experienced and senior managers in a positive and engaging, outside-of-the-workday forum. They find out who you are and that you’re proactively pursuing relevant knowledge.

What does your leadership say about these accomplishments?

Our former Associate Center Director, Technical and CKO Dale Thomas is very pleased with it. He plan on using cases we produce the Systems Engineering courses he teaches at University of Alabama in Huntsville. As I said earlier, one senior manager requested we prepare case studies for him to present at Real World Marshall Mission Success. As my students have produced really good, relevant case studies I am seeing management being more impressed with the clear benefits of this kind of learning. I am working with some to identify cases they would like to see done, and encourage their staff to participate in the course. They seem to have no shortage of ideas for potential case studies. You just have to ask the question.

What’s the greatest lesson learned about this journey?

Be brave. Because nobody had ever done this before, and even Ed Rogers who wrote numerous cases, hasn’t done this before, we weren’t sure you could teach case study writing and successfully produce any that could be used. And this is case study writing with engineers. The cliché is that we engineers became engineers to avoid writing. The reality is you find wonderful people, with wonderful insights and a wondrous array of talents and interests. NASA people are so diverse. Just because it’s outside the box doesn’t mean that nobody is going to enjoy it. You can’t assume that just because you don’t see it happening, you shouldn’t do it. There’s such wonderful diversity here.

Jennifer Stevens is Chief Knowledge Integrator at Marshall Space Flight Center.

Susan Snyder is the Managing Editor of the NASA Knowledge Journal.