Distilling and Infusing Lessons at Marshall

With her diverse systems engineering background, Jennifer Stevens is a natural to lead the recently established Marshall Distilling Team.

No, not because she understands how to make sour mash in a backyard still, but because she understands both how parts fits into the whole and how the efforts of individuals impact customers and suppliers, both upstream and downstream.

For 15 years, Jennifer worked with Thiokol, which taught her project engineering and program management.  She developed materials and processing tests for solid rocket motors, primarily on the Reusable Solid Rocket Motors in the Space Shuttle program, touching nearly every system and material for the motors.  She was working on a Shuttle nozzle problem when her boss asked her to do a lunch and learn on the problem material.  She likened the material (and much of her work) to silly putty – if you pull it slowly, it stretches.  If you pull it too fast, it breaks.

We recently sat down with Jennifer to find out more about why the Marshall Distilling Team was established and how their decisions impact practitioners across Marshall and the agency.

Question:  Jennifer, tell me more about why the distilling team was established?

Answer:  In 2012, there was a push at NASA on knowledge management.  One of the requirements of this push was that centers had to establish a Chief Knowledge Officer role and appoint someone into the role. Dale Thomas, Associate Center Director, Technical, was appointed for Marshall.   One of the first things Dale did as CKO was to set-up a Lessons Learned Committee (LLC) manned by senior Marshall Executives from across the center.  This team meets regularly to decide which project and program lessons learned should be approved in the Lessons Learned Information System (LLIS).  It became quickly apparent to Dale that some of the lessons learned could be actively infused into our work processes to improve the way we do work.   For instance, changes might be made to center policy and processes (Marshall Policy Requirements (MPRs)), training programs, communications, or program reviews as applicable.)  The LLC members already sat on a newly established Program/Project Management Advisory Council (PPMAC) that met biweekly (now monthly), to advise projects from set-up through development to ensure they are following the appropriate project protocols and standards. They decided that another team should meet more often and assess the lessons in more detail, make recommendations, and then ensure that the “infusion” got implemented.  That became the mission of the Distilling Team – to distill the Center’s lessons learned into something actionable so that the learning gets infused into the workplace.  The chart to the right reflects the process as it currently works.

This is an image of the stages of distilling and infusing lessons learned into work processes at Marshall Space Flight Center.
Image Credit: Jennifer Stevens / Marshall Space Flight Center

Question:You were asked to lead the distilling team.  What was your priority the first few months you were in your role?

Answer:  We kicked-off the team in October of 2012, and at that time I was the Center Data Manager.  I worked with Dale and the LL Committee to identify the individuals who would sit on the Distilling Team.  We ended-up with participants from every major discipline in our center – Engineering, Project Offices, Human Resources- discipline, programmatic, and institutional representatives.  We spent the first couple of months developing the processes, charter, and the team norms.  We met weekly, and early-on we piloted a couple of different ways of distilling different forms of lessons learned.  For example, we started with an LLIS lessons learned drawn from an individual’s submission.  The second form was lessons captured in presentations rather than written, formatted lessons.  We had people who developed a LL present their findings and document or presentation to the Distilling Team at our regular meeting. Finally, we tweaked the process to distill lessons learned from LL reports.  Each of these different forms has slightly different levels of information.

Question:  What way worked better?

Answer:  What we learned from that was that it is important for the person creating the LL to come to the team and discuss the background and the nuances of the LL.  This made for much better distilling because we could ask questions and get clarification.  We also could ask the presenter what they thought should change to make sure the learning was infused in the right places in the Center. People that come in can help you understand how significant one lesson is over another.  You have to understand what is important to your center and understand what is actionable.  In reports you may not get all the content, so you may not understand it.  However, there is a challenge for us with in-person presentations too – If people on the team have experience with the project, they might have a “knot” to work out, something that they felt conflicted about on the project that never got resolved.  So, we make sure that we document that knot during the meeting and move it over to the side, so that the person feels heard, but the emotion doesn’t derail the story.

Question:  Where else do lessons learned come from at Marshall?

Answer:  Well, we found that LLs come in lots of formats and various ways that they are captured.  For instance, many of the LLs being presented to leadership wouldn’t trickle down in the organizations, but now that the Distilling Team is in place, leaders will send presenters to us – and as a result, much more recent LLs are coming into the work place.  We also spend time looking at LLs that are in reports.  We have to know our priorities because a report might have 20-30 lessons and/or recommendations – there is a lot to weed through.

In an example of having someone close to a lesson come to the DT to help us distill, we had the Chief Engineer come in to talk about ISERV and we processed a lot of LLs in one session.(ISERV is aremote-controlled Earth-observing camera system that was launched last July to the International Space Station (ISS) aboard the Japan Aerospace Exploration Agency’s third H-II Transfer Vehicle (HTV-3).)   The system is directed by researchers on the ground to acquire imagery of specific areas of the globe for disaster analysis and environmental studies.) There were interesting lessons on integrating Class D projects on Space Station and we’ve learned a lot about COTS – commercial off-the -shelf software.  We decided to focus on infusion actions that could help small, inexperienced project teams be more successful and on informing designers and project leaders about the nuances of using COTS equipment. 

Question:  So, it sounds like the Distilling Team got to be known as a place that got things done with respect to infusing the lessons learned.  What were some of the challenges the team faced?

Answer:  Well, after about a year, we had many, many lessons learned and a lot of them seemed to be related or repeated.  I wanted to help the team see the relationships.  So, I took all of the issues and recommendations, and I printed them up from our tracking spreadsheet.  I cut the lessons, recommendations, and infusion actions into separate pieces of paper, put big sheets of paper on the wall, and together we posted the small pieces of paper into “like” buckets or topics.  We ended up with several areas on the paper (or map) that we were able to name like “training” or “communication” or “program management”.  This helped the team see the relationship of the LLs – and to understand that often-times, they don’t fit into a perfect category because they are inter-related.

Question:  Can you give me an example?

Answer:  A lot of times, you see that managers make decisions that differ from what the policy says they should do in that instance.  People don’t read policy practices and procedures and follow them to letter, and you can never assume that they will. It goes back tohow we train supervisors, branch chiefs, and leaders, to do their jobs.  Often times, when you put a person in a leadership job, they aren’t experienced in the job or job area.  Also, we don’t always take the time to train the technical person on how to lead teams, manage people, coach and develop talent.  Maybe we are missing teaching opportunities to pass on knowledge due to busy schedules.   But sometimes we experienced people think things are just “duhs”, and we think people already know or they should know.  But they don’t because we haven’t taken the time to train and mentor them.  So, is that the “management” bucket or the “training” bucket or  an engineering discipline issue ?

But getting back to buckets, we often find that an issue isn’t just PM or SE or S&MA or training or whatever.  A lot of times lessons are about interactions, and solutions cover multiple areas.  We have processed hundreds of lessons in the past 2 years.  Fortunately (tongue in cheek), we keep making the same mistakes over and over, so the buckets are actually not overflowing… You have to decide what is a priority to tackle.

 

This is picture of Jennifer Stevens, the subject of the interview.
Photo Credit: Jennifer Stevens

Question:  So, what are some of the Distilling Team lessons learned so far?

Answer:  We make recommendations, but they must be infusible and actionable.  We just have to identify the things that are infusible.  The team is made up of HR/Training, projects and programs, institutional orgs and an engineering representative.  They are free to take the LL back to their orgs and infuse them as they see fit.  But you can’t just come up with some nebulous wish list.  You have to have actionable recommendations.

HR takes lessons back that are useful for training courses that are coming up or in development.  Lessons illustrate the principles being taught.    The end effect is that Training knows to ask us for help in identifying lessons to support specific courses instead of us just pushing lessons to them.

Also, regarding the PPMAC LLC leadership – we told them, “If you don’t appoint someone to be on the Distilling Team and empower them to take the actions back to their organizations, then “you” are it, and you need to take the actions back to your organization.” In other words, people nominated to the Distilling Team have to be empowered to be on the team and to make change without a lot of permissions.

We are beginning to see that projects are doing more with lessons learned.  For example, the SLS groups will proactively go to LLIS or look for lessons in other places so they can go to design and review activities prepared to be as smart as they can be.

Something I learned from going through this process – there was a LL that was sitting in a queue – I thought it was just a materials and process lessons learned.  It turned out to be about what happens if there is a long lag time between building big hardware.  I wouldn’t have guessed the lesson was really about the lag time, or that it was really applicable to big projects.  But the SLS representative said ‘we are talking about that right now.’  She realized they needed to be aware of how to keep process skills and knowledge retained when they were going to be building really expensive parts with long lag times between builds.  It was fortuitously timely that we distilled that lesson right then, but even if it were just a little earlier or a little later, she would have been aware of the lesson, and been able to take it straight back to her organization. The other lesson learned for me is that one person can’t predict where any single lesson will really apply- it takes a diverse team.

Question:  What was a really big one for the Distilling Team?

Answer:  What you find is that you see things showing up over and over again.  There were some that were small projects with inexperienced project team leads.  These small teams – they get funding to go do something, so they go.  Then they realize that there are problems with the contract, team issues, they didn’t know what steps they needed to go through, and they don’t have funds for getting more Safety and Mission Assurance involvement up-front, things like that.  And, there are no handbooks for the small projects.

We also realized that these teams don’t work well not because of PM and SE guidance, but because the team leads come in and they aren’t trained or mentored properly.  To be successful, they need a couple of things – more previous hands-on experience with small projects, and better definitions for leading small teams.  The problem is that we don’t have enough small projects at Marshall to give people previous experience.  We have one massive flagship project –SLS- which takes half of our people.  So folks don’t get to develop capability to lead all the facets of a project when they are that big.

Question:  How did the team resolve that? 

Answer:  So, we talked to the training people – the training program Systems Engineering Leadership Development Program (SELDP) does a good job, but we don’t have a Project Management Leadership Development Program (PMLDP) – we don’t have a lot of small project teams, and some of the ones we do have need better mentoring.

In addition to the lack of leadership experience and mentoring, we have the flip-side of that.  Some of our early career people come in with no team skills at all.  Sometimes, they don’t know that they need to be teachable.  For example, on a HOPE project, we had someone that needed to do a schedule with a Work Breakdown Structure (WBS).  This individual argued with the mentor for three weeks that he didn’t need to do it.  We had a trainer who was working the interventions for some of the team issues, and we were wearing out the couple of mentors we had.

In the end, the Distilling Team figured out we didn’t need a big mentoring program; we needed to teach mentors how to coach small project teams to be successful.  They needed the right tools and materials to facilitate HOPE-type projects.  Mentoring had to be tailored to the specific needs of those small teams and inexperienced team members, and the mentors themselves need to take training.  Broad mentoring program practices just weren’t the right fix.

In this example, the Distilling Team determined that we didn’t have the scope or time to figure out the detailed answers.  And, the big lesson learned was that we didn’t have to have the answers.  This was a leadership issue, so we took the recommendation back to the LLC. Dale looked to the two PM Office representatives that have the small teams, and he tasked them with developing a plan for how to create the mentoring skills for leading small teams.  They developed the plan and they were able to address this issue.  We helped by working with them to define what the needed skills and competencies would be and giving them the background from the lessons.  The lessons gave them the “why” of task.

The solutions came from the team that was tasked by Dale to come up with the plan, not the Distilling Team.  The Distilling Team identified the problem – the small project leads don’t know what they need to know to avoid team issues down the line – and recommended a specific, actionable infusion solution that the DT supported, but was beyond our scope to fulfill.

Question:  Very interesting lessons from leadership and the Distilling Team!  What’s next for the team? 

Answer:  Because we have so many lessons learned that we’ve gathered, and because the Distilling Team and Dale wanted some metrics, reports and a database to gather the LLs, we are developing a MSFC LL Distilling Database.   Marjie Davis, our other MSFC CDM, and I are currently populating the data and developing metrics report for our CKO.

We got this idea from going to our annual NASA CKO knowledge community forum at JSC in April.  We heard from Goddard’s CKO, Ed Rogers and his former NASA CKO colleague, Marisa Connel, now with Northrop Grumman Aerospace Systems, and they talked about a case where she developed a similar tracking database.   We thought a database would help us get our arms around the volume of LLs we were working with as a team, and help us better articulate and track our successes.

The database features include the ability to add a lesson project, org code, lesson source, the team’s recommendation – which is the distilling/infusion action, impact, results, comments, etc. We plan to use it real time during our team meetings.  We can open it up during Distilling Team meetings, add the lesson, discuss the issues, and discuss what needs to be changed, document these steps in the database, and add actionable items (infusion actions). We can add follow-ups that detail the impact of the actions.  We can run reports in the meeting too.

Closing:  Marshall’s Lessons Learned Committee and Distilling Team functions are a leading practice at NASA for taking not just the lessons learned and ensuring they get put into the LLIS for future project reference, but also for making real change to numerous processes that ultimately contribute to satisfied teams and mission success.

Learn more about the Marshall Space Flight Center’s Chief Knowledge Office knowledge strategy and the Lessons Learned Distilling Team:

Marshal Space Flight Center’s Knowledge Strategy (PDF)
The Marshall Star (PDF)