A Pause and Learn (PaL) session is a team conversation. On the surface, PaL is deceptively simple, just conversation among members of a team, typically conducted after a significant event or project milestone. But it is different in many ways from other conversations—from a staff meeting, for example. A PaL is facilitated by a knowledgeable outsider who helps the team identify and capture knowledge gained at key project stages. Essential elements of the conversation are documented in the form of Knowledge Maps (KMAPs), which can then be shared to benefit others. A Knowledge Map is a visual representation of a conversation, highlighting valuable insights and comments within the context of that conversation.
The PaL practice was introduced at Goddard in 2005-2006 by Ed Rogers, the Center’s Chief Knowledge Officer. As of December 2014, more than 90 Pause and Learn sessions have been facilitated at Goddard, 24 of which took place in 2014 (a record year.) Carrying out PaLs for almost ten years has taught us a lot about what works and what doesn’t. We have learned which core elements are essential and where useful changes are possible.
The fundamental elements that make a PaL effective are:
- A facilitated conversation
- Key questions
- Team ownership of the process
- Clear and immediate benefits.
Outcomes can be less than optimal when any of these core elements is disregarded. At the same time, it is important to keep learning from the process and be flexible. In a 2012 ASK Magazine article, I described an adaptation of the process to allow one team to learn directly from another by observing a PaL. Since then, we have tried other innovations, some with more success than others.
A FACILITATED CONVERSATION
A PaL session is best facilitated by someone outside the project or organization participating in the PaL, someone who doesn’t have a personal stake in the conversation but whose primary goal is to act as a facilitator and guide to maximize the benefits participants get from the conversation. At the same time, the facilitator must be sufficiently knowledgeable about the general operations and processes of the organization to understand its issues and goals. This facilitator function was initially provided by the Chief Knowledge Officer and now continues within the Flight Projects Directorate by a Knowledge Management lead.
This facilitated approach works well, with the corollary that all participants must understand that they are coming to the conversation as equals. The organizational hierarchy is temporarily set aside and everyone in the room has an equal opportunity to speak up. To ensure an open and honest conversation, expectations need to be set ahead of time with the team leaders. In practice, this means that it is best for the team leaders to set the tone by sending the PaL invitation to the participants and explaining the intent, but allowing others to take the lead when the conversation starts.
Open conversation is less likely when management layers above that team are invited to the process. In some cases, it is also valuable for the team to meet and talk without their immediate leadership/management in the room.
Getting the right participants in the room is critical. When a mission is being built out-of-house and Goddard’s role is limited, it may be difficult or even not advisable to suggest a Goddard PaL. Since the primary objective is for Goddard to continuously learn from its missions, a focus on Goddard’s lesson can be achieved by focusing on the Goddard team’s lessons rather than those of the entire mission team.
We recently sat down with a mission manager to talk about what he learned as mission manager for a PI-led out-of-house mission. There is value in doing that even when a traditional PaL with all the key stakeholders is not possible, but the benefits of PaLs usually come from group discussion.
One PaL last year failed to achieve its intended objectives when key members of the team were not able to attend. We decided to gather insights from the rest of the team by conducting one-on-one interviews, but the process quickly fell apart. The failure demonstrated that a PaL conversation with all key team members in the room is different from and more than a collection of individual perspectives. It allows individual perspectives to be expressed, but also gives the team an opportunity to correct misunderstandings and come to a common understanding of each other’s perspectives. What can be achieved in ninety minutes of conversation is not achieved by compiling independent insights from team members gathered in one-on-one interviews.
That experience reminded us to stick to the basics of the PaL and watch out for the risks of diverging from the original approach. While it’s still a good idea to collect insights from a core member of the team who isn’t able to attend the PaL, it is not advisable to try to build a PaL out of individual interviews. If it is not a team conversation, it is not a PaL.
STARTING WITH KEY QUESTIONS
Another fundamental element of a successful PaL is a focus on key questions. The facilitator meets with the team leader (ideally face-to-face) to explain what a PaL is, how it is conducted, its benefits, expected outputs, and to discuss who should be invited as well as the dynamics of the conversation. The facilitator also asks a few questions to get a sense of the top issues that should be addressed. While the key questions are always the same—“What happened? What went well? What didn’t go well? What could we have done differently?”—there can be variations. For example, a project moving from Phase B to Phase C might want to focus on “What are we going to do differently moving forward?”—a question likely to provide immediate benefits.
In the past year, we have worked with one office to make the PaL a complement to their existing lessons learned process which had one individual write up lessons learned in a pre-formatted presentation which was then given to management. The process was primarily meant to capture lessons at the level of the technical workforce and worked well to identify technical issues and strategies to remedy them. It did not sufficiently capture the project challenges that originated at higher levels in the interactions between key stakeholders, however. When a Pause and Learn is conducted, these higher level stakeholder dynamics are discussed (and ideally all the stakeholders are represented in the room). Sometimes teams are reluctant to talk about issues that are externally imposed on them and, they think, not open to change. Why bother talking about it when we know there’s nothing we can do about it? But it is precisely those issues that need to be brought to the attention of management through as many channels as possible. A Knowledge Map is tool, and one way to convey important messages.
As the PaL process has become better known at Goddard, team leads have sometimes asked for a PaL when they needed a facilitator to help with a group discussion. When the key questions are significantly different from those of a traditional PaL though, it’s probably not a PaL and should be called something else.
A TEAM OR PROJECT-OWNED PROCESS
Initially, PaL sessions focused almost entirely on supporting the team’s learning. The conversation maps or knowledge maps developed were owned by the team and were not disseminated outside the team, in part because the lessons might not be fully understood without their local context. The preferred method for sharing lessons across projects was a face-to-face workshop where participants could have open discussions about their experience.
This internal PaL format is still used occasionally, but most project-specific PaL sessions are now accompanied by conversation maps meant for dissemination across projects. The web of maps that has evolved over the past two years is accessible to everyone at Goddard and by extension everyone at NASA.
For the Flight Projects Directorate, the PaL sessions and associated maps are the preferred approach for identifying and sharing lessons learned. It has therefore become important to document the sessions and lessons in a way that makes them accessible to other projects. “Accessible” means available for review and constructed in such a way that they make sense to someone outside the project.
Making the maps accessible to other projects can have disadvantages. Project team members can become concerned about who is going to see the maps and how the information might be interpreted or—more to the point—misinterpreted. The map review and validation process is designed to address these concerns. Team leaders review the draft knowledge maps before they are integrated into the aggregated map system known as the KMAP web. The review is meant to identify key elements that may have been missed, correct errors, adjust the wording to avoid misinterpretations. The key is to keep the maps as close to the essence of the conversation as possible.
Once they have been approved by the team’s leadership, the maps go through a second review by management, giving them the opportunity to ask for clarifications and add comments. Management does not redact the maps but has the ability to provide additional guidance when a lesson or insight from a project could be misread or misinterpreted.
CLEAR AND IMMEDIATE BENEFITS
There were 24 Pause and Learn sessions in 2014. One office has fully institutionalized the process and conducts a PaL after each key event. One project conducted two PaLs in one year, both follow-ups to key reviews. This is a relatively small, very fast-paced project whose leadership has found value in the PaL conversation within the team and in the careful articulation of key insights on the knowledge maps, which they have posted on the wall in the project’s central office hub.
In one case this past year, a different PaL format was deployed to gather lessons learned. This effort was carried out by an external consultant, with the support of the Office of the Chief Knowledge Officer, within the Management Operations Directorate. A much more structured approach was used and a more structured, linear report was generated, with specific recommendations for action. The process required two full days of meetings. While this format has value under certain conditions, it is not realistic to ask project teams to set aside two days for a Pause and Learn activity. A typical PaL session is scheduled for ninety minutes with the entire team. The PaL Map review/validation process requires another hour or so of the project leaders’ time.
The return on investment of a traditional ninety-minute PaL is quite substantial, in part because the team’s investment of time—a scarce project resource—is reasonable.
Reflection time tends to be undervalued, especially where keeping to the schedule is critical to success—a fact of life for many projects. But just as project managers regularly point out that a key lesson learned is to know how to spend some reserves early to address issues that would cost much more to deal with later on, time invested in reflection early on and throughout the project life cycle can save time and money later. And of course identifying the most effective way to spend reserves requires some reflection.
In the end, a project increases its chances of success when art meets science, when people are able to work well together to address tough technical challenges. Working well together means communicating well. That is why the Pause and Learn is so important. It develops or reinforces a team’s ability to engage in open, honest conversations; it maintains or opens up ways for team members to talk to each other and address issues that may not necessarily come up in traditional staff meetings or in one-on-one conversations.
CONTINUED LEARNING AND IMPROVEMENT
While keeping the fundamental PaL principles in mind, we continue to look for ways to improve and expand this valuable practice. Here are a few potential improvements we are working on or considering:
- Approaching the large projects at a lower level. Large, complex projects cannot easily do PaLs at the mission level. There would be some value in conducting PaLs on a more regular basis with instrument teams or sub-systems, especially for in-house instruments.
- Demonstrating the value of aggregating lessons and making them accessible to other projects. The return on a 90-minute investment of time is clear. The ROI of time and effort spent making lessons widely accessible needs to be demonstrated. There is more work to be done with the project teams to demonstrate the value of looking through and discussing insights and lessons from other projects and more to be done to ensure that knowledge maps present lessons and insights in a usable way.
- Analyzing and using lessons better. There is a need for more in-depth analysis of aggregated lessons, more use of lessons and insights in workshops, and more follow-up by management, which could benefit from reviewing and discussing the Knowledge Maps on a regular basis.
- Conducting timely workshops based on the project lifecycles. For example, if multiple projects will be going through Integration and Testing in the next six months, a workshop bringing together projects about to enter that phase and projects that have recently completed that phase would enable an exchange of valuable knowledge when it is needed.
- Increasing integration of the PaL Process/Lessons Learned process with other knowledge-management-related efforts within the Flight Projects Directorate. Integrating the PaL process with professional development, best practices and associated knowledge networks and other activities can provide new benefits. This has already started to happen and will likely increase.
Barbara Fillip is the Knowledge Management Lead at NASA Goddard Space Flight Center.
RELATED RESOURCES / REFERENCES
- Richard Day and Ed Rogers, “Enhancing NASA’s Performance as a Learning Organization.” ASK Magazine, Issue 22, January 1, 2006. /2006/01/01/enhancing-nasas-performance-as-a-learning-organization/ & Edward W. Rogers, “The Knowledge Management Journey”, August 11, 2013, ASK Magazine, Issue 51. /2013/08/11/the-knowledge-management-journey/
- Barbara Fillip, “Project Knowledge in the Moment.” ASK Magazine, Issue 48, November 1, 2012. /2012/11/01/project-knowledge-in-the-moment/
- Daniel Patrick Forrester, Consider: Harnessing the Power of Reflective Thinking in Your Organization, 2011.
- APQC, “Assessing the Impact of the Lessons Learned Process,” 2010.
