Posts Tagged ‘align’

25th Anniversary ButterflyAt RK&A, we think a lot about intentional practice and we encourage our clients to do the same. In planning meetings and reflection workshops, we ask clients to think about which elements of their work align with their institutional mission and vision (check out Randi’s blog post for more about the challenges of alignment). We push them to consider who might be the right audience for their program or exhibition, and we ask them to talk about the intended outcomes for their projects. Posing these kinds of questions is much easier for an “outsider” to do because we don’t have institutional baggage or a personal connection to a problem project. As consultants, we aren’t beholden to the way things have always been done. I get it – it can be hard to let go; but seeing clients seek information to make informed decisions is a powerful, exciting process. These clients want more information. They are willing to try new things, to change old (and sometimes new) programs to see if they can improve upon the results. These are museum professionals who want the very best experiences for their visitors.

We recently completed a project with a history museum and the results were, well, not as rosy as one might hope. Change is HardAfter explaining the challenges of changing students’ perspectives in a short, one-time museum visit, we started talking about what could be done to increase the effectiveness of the program. One of our suggestions was to increase the time allotted for the program and rather than spending that extra time in the exhibition, use that time to facilitate a discussion with students so they can process and reflect on what they had seen. Changing a program’s format and duration is a difficult task for the museum to undertake – it may require extra staff and certainly a different schedule – but it could make a difference. A few days later, our client asked us if there are any studies that show that longer programs are more effective. After failing to come up with any examples (if you know of any such studies, please leave a comment), the client asked for another study to see if a longer program leads to a different outcome.

As an evaluator, I want to support museums as they change the way they do their work. Evaluation can provide the necessary information to see if new ideas work. It can give clients the data-based push they need to let go of the way things have always been done and to try something new. If nothing else, the evaluation process can be a forum to remind people that even when you are changing course, there is a place for you on the Cycle of Intentional Practice: Plan, Align, Evaluate, Reflect.

Read Full Post »

We have been thinking about intentional practice a lot lately.  The article below, written by Randi, appeared in ASTC Dimensions May/June 2008 issue.  If you would like to read more of Randi’s thoughts on intentional practice, be sure to read her 2007 Curator article, “The Case for Holistic Intentionality.”

At museum conferences these days, people are talking about accountability, public impact, and relevance. These ideas are not new. A decade ago, in a 1997 keynote address for the Mid-Atlantic Association of Museums’ 50th anniversary, the late Smithsonian scholar Stephen Weil spoke of the “in-your-face, bottom-line, hard-nosed questions”—the ones that museums often hope to keep under wraps: “Do museums really matter? Can and do museums make a difference?”In arguing that some museums do make a difference, and that all should strive to do so, Weil supported the notion that “the very things that make a museum good are its intent to make a ‘positive difference in the quality of people’s lives.’” He borrowed this last phrase from the United Way of America, which was then challenging its grantees to document the benefits a given program had made in their lives.

Today, museums face accountability questions from many directions. In response to the Government Performance and Results Act of 1993, U.S. federal agencies began to articulate the kinds of outcomes they expected grantees to document. Private foundations followed suit, reexamining their own evaluation practices, as well as those of grantees. The effort continues. The National Science Foundation recently published its Framework for Evaluating Impacts of Informal Science Education Projects, outlining five categories of impact it expects grantees to assess. And for those who object that “you can’t measure mission-centered work,” current United Way CEO Brian Gallagher, as reported in the Wall Street Journal, has a succinct reply: “You most certainly can. The question is, ‘Are you committed to do it?’ And then, ‘Are you committed to report on it?’”

As museums begin to grapple with their intent to make a positive difference, they can start by reexamining their museum’s mission. Weil believed, as many do still, that a mission is key to an institution’s success. A museum’s mission should be a declaration of its core purpose—clarifying what the museum values, reflecting what the museum embodies, and describing its intent to affect its public and community. Establishing a clear institutional purpose, Weil believed, is the first step to being able to assess effectiveness in achieving public impact.

From my own experience as an evaluator, I would add this observation: Museums do not, in and of themselves, value, reflect, or intend. People do.

An institution’s mission will not be within reach unless everyone who works in that institution is mission-focused and mission-driven. Before museums can assess their impact, staff must collectively clarify their intent. Public impact, relevance, and value grow from what I have called “intentional practice”—the willingness of everyone in the museum to examine all operational activities through three mission-based filters: clarity of intent, alignment of practice and resources, and reflective inquiry.

  • Clarity of intent. Opportunities for all staff to come together to discuss the core values of their museum are vital. Colleagues should both encourage others to explore their passions and also challenge others’ thinking as a way of clarifying what is truly of importance.  In the spirit of thoughtful inquiry, why not ask a colleague to defend his or her position? Most people appreciate being asked to explain why they think the way they do. This kind of exploration allows practitioners to voice the passion behind their ideas and learn what they, as a group, really care about. Reexamining the essence of the museum together can reinvigorate the collaborative spirit, enabling staff to further their practice with intent.
  • Alignment of practices and resources. Unless the work of the museum is aligned with its intent, staff may spend time and resources on activities that are good in themselves but may not support the museum’s intent. Perhaps staff should determine—through evaluation—which programs yield the highest impact, keep those programs, and either improve or discontinue those that do not deliver impact. Aligning practice—the activities a museum does and how it does them—and resources so they support the museum’s intent requires thinking about what you should be doing and what you need not do any more. Conversations about realignment will deepen staff members’ understanding of the museum’s intent and the ways in which their work supports it.
  • Reflective inquiry. As an evaluator, I frequently see front-end and formative evaluation being used effectively to shape a final visitor experience. The same cannot be said of summative evaluation. By the time a mandated final report is done, practitioners may have little time or motivation to review it. This is unfortunate because much can be learned through reflecting on past work.

I see a strong relationship between taking the time to think about the work you have done and learning from the work you have done. Practitioners who want to be intentional in their practice can use summative evaluation as a way to gain insight and knowledge about visitors’ perspectives and experiences. The outcome of such reflective inquiry is learning about the ways in which their museum is achieving impact. I would encourage all museums to routinely set aside time for staff to use inquiry as a reflection strategy and to discuss their practice in the context of the institution’s intent.

In conclusion, accountability questions are not likely to disappear, but even if they did, museum practitioners would still need to respond to the “To what end?” question. The sustainable health of the museum depends on it.

Most of the workers I encounter in museums are passionate about their work and want to make a positive difference in people’s lives. If practitioners begin collaborating with colleagues to clarify their museum’s intent, realign their practices and resources to support that intent, and engage in reflective inquiry to learn how they can improve their efforts, they will be on their way to achieving that goal.

Read Full Post »

Welcome to our new Throwback Thursday series, where we take a moment to look back at projects from our archives.  Today we’ll be sharing a case study about our planning and evaluation work with the Science Museum of Virginia and their Sphere Corps Program.  You might recall this particular Science On a Sphere program from one of our prior posts, Learning to Embrace Failure, and today we’ll share a bit more about how we approached the study, what we learned, and the implications of those findings.

Sphere Corps Program [2012]

For this planning and evaluation project with The Science Museum of Virginia (SMV), RK&A evaluated Sphere Corps, a Science on a Sphere program about climate change developed by SMV with funding from the National Oceanic and Atmospheric Administration (NOAA).    

How did we approach this study?  

The study was designed around RK&A’s belief that organizations must be intentional in their practice by continually clarifying purpose, aligning practices and resources to achieve purpose, measuring outcomes, and learning from practice to strengthen ongoing planning and actions.  To this end, the Sphere Corps project included five phases of work—a literature review, a workshop to define intended program outcomes, two rounds of formative evaluation, and two reflection workshops.  Formative evaluation data were collected using naturalistic observations and in-depth interviews.  Each phase of work allowed staff to explore their vision for the Sphere Corps program and how it changed over time as they learned from and reflected on evaluation findings.

What did we learn?SOS

SMV staff’s goal was to create a facilitated, inquiry-based Science on a Sphere program about climate change.  RK&A first completed a literature review that revealed a facilitated Sphere experience was in keeping with best practices and that using inquiry methods in a 20-minute program would be challenging but worth exploring further.  Staff then brainstormed and honed the outcomes they hoped to achieve in Sphere Corps, which guided planning and script development.  The first round of formative evaluation identified implementation barriers and an overabundance of iClicker questions, all of which created a challenging environment for educators to effectively use inquiry.  Upon reflection, staff reduced the number of iClicker questions and added visualizations and questions that required close observation of the Sphere.  Following a second round of formative evaluation, staff made additional changes to the program script and began to reflect on the reality of using inquiry in a single 20-minute program.  Since the script covered a range of topics related to climate change, staff wondered if they should instead go deeper with one topic while encouraging more visitor observation and interpretation of Sphere data.  Out of this discussion arose the idea of “mini-programs”—a series of programs that would focus on communicating one key idea about climate change, such as helping people understand the difference between weather and climate.

What are the implications of the findings?

Central to the idea of the “mini-program” is the idea of doing less to achieve more.  Impact and outcomes are incredibly difficult to achieve and trying to achieve too much often results in accomplishing very little.  Through a reflection workshop and staff discussion, the SMV team was able to prioritize and streamline the outcomes and indicators originally written for the Sphere Corps program.  Staff also recognized that their primary goal with the Sphere Corps program is to encourage visitors to think more critically about the science behind climate change.  By scaling down the number of topics covered in the presentation, each program could intentionally focus on: (1) one key idea or question related to climate change; (2) achievement of only a few intended outcomes; and (3) implementation of specific facilitation strategies to achieve those outcomes.  Intentionally covering less content also opens up opportunities to more effectively use inquiry methods.

Read Full Post »

25th Anniversary ButterflySometimes when learning surfaces slowly, it is barely visible, until one day the world looks different.  Responding to that difference is the first layer of that complex process often labeled as learning.  The Cycle of Intentional Practice was a long time coming—emerging from many years of conducting evaluations, where I worked closely with museum staff and leadership as well as with visitors.  The Cycle of Intentional Practice is an illustration of an ideal work cycle that started to form when I was writing “A Case for Holistic Intentionality”.  I am visually oriented and I often have to draw my ideas before I write about them; in this case, I was writing about my ideas and then I felt the need to create a visualization to depict what I was thinking—in part to help me understand what I was thinking, but also to help others.  I included the first iteration of the cycle in the manuscript to Curator, but the editor said the Journal does not usually publish that kind of illustration, so I put it aside.

That original cycle differs from the one I use today—it was simpler (it included “Plan,” “Act,” and “Evaluate”), and while I didn’t know it at the time, it was a draft.  There have been several more iterations over time (one was “Plan,” “Act,” and “Evaluate & Reflect,” for example); as I continue to learn and improve my practice, I change the cycle accordingly.  Most stunning to me was that the first draft of the cycle showed nothing in the center—nothing!  I feel a little embarrassed by my omission and I am not entirely sure what I was thinking at the time, but I hope my oversight was short-lived.  At some point I placed the word “Intentions” in the center, and as I clarified my ideas, with the hope of applying the cycle to our evaluation and planning work, I eventually replaced “intentions” with “impact.”  I recall how difficult it was to explain the concept of “intentions” so I eventually needed to remove the word from the center (as much as I loved having it there).  If my goal was to have museums apply the cycle to their daily and strategic work, the cycle needed to represent an idea people found comfortable and doable.  Soon I realized that intentionality was the larger concept of the cycle and what needed to be placed in the center was the result of a museum’s work on its publics–impact.  So was born our intentionality work with museums.  Then I realized the true power of intentionality—mission could go in the center as well as outcomes, or anything for that matter.  The artist’s rendition below demonstrates the versatility of intentionality as a concept.

Cycle of Intentional Practice

An artistic rendering of the Cycle of Intentional Practice by artist Andrea Herrick

What I find most amazing is that two crucial ideas—reflection and impact—were not present in the first iterations of the cycle, although they were discussed when I talked about intentionality.  Our intentional planning work (which we refer to as impact planning) would be rudderless without the presence of impact and our ability to learn from our work would be weakened without reflection.  And that brings me to another realization, which I am reminded of daily—the never-ending pursuit of achieving clarity of thought, followed by writing a clear expression of that thought.

Today I talk about the Cycle of Intentional Practice as a draft—it will always be on the verge of becoming, but these days I am more comfortable with the idea of the Cycle being a draft—an idea in process—than I was a decade ago; in fact, I have come to realize that all work is a draft and that if one is serious about learning and applying new ideas to work and life, then all ideas, all products, all knowledge are mere drafts because learning is continuous, right?

Humbling?  Yes indeed.

Read Full Post »

Welcome to the Intentional Museum Blog, an all-staff endeavor of Randi Korn & Associates.  We intend to share our thoughts and questions twice monthly about compelling ideas we come across in our work and readings.  The platform for our practice and the inspiration for our postings is the Cycle of Intentional Practice.

Cycle of Intentional Practice

This cycle has emerged slowly, evolving over the last 10 to 15 years, as I started to think about all I had learned as an evaluator and researcher while pursuing my passion of studying people’s experiences in informal learning environments.  In our work as planners and evaluators in museums, we have witnessed how difficult it can be to achieve results that reflect a museum’s original intentions.  Likewise, we have witnessed that results surface when organizations focus their energy, decisions, actions, and dollars towards well-articulated ends.  Thus, increasingly, intentionality is becoming a driving force in our work with clients, as we recognize how vital it is as an idea, action, theory, and practice.

Intentionality is not a new concept, as several well-known authors have written about it.  For example, business author Jim Collins notes in Good to Great that the work of a great organization must “attract and channel resources directed solely . . .” to their intentions and “reject resources that drive them away from” their passion and unique value.  Collins’ concept applies to museums, too, as described by museum scholar Stephen Weil in Making Museums Matter.  He wrote, “The only activities in which the museum can legitimately engage are those intended to further its institutional purpose.”  Collins’ and Weil’s concepts embody the essence of intentionality, where all actions are purposeful, deliberate, and focused on achieving the intended impact or results.  With funders requiring evidence that their dollars are being used in the way they intended, intentionality rings of relevance.  Likewise, with fewer dollars available to museums, how museums use those dollars is increasingly important.

The Cycle of Intentional Practice places “impact” in the center of the cycle and assumes museums want to make a positive difference in people’s lives, which is how I define “impact.”  Intentional practice requires that the museum articulates the impact it would like to achieve (by writing an impact statement), align a museum’s practices and resources to support the impact it would like to achieve, measure the ways in which the museum is achieving impact, and reflect on the results to learn from them for the purpose of improvement.  Intentional practice may seem insular, but it acknowledges a museum’s responsibility to its external community through evaluation and other efforts.  The tension between a museum and its public is real and the museum may need to work hard to balance its internal aspirations and resources with its community’s needs.  Balancing potentially conflicting ideals, though challenging, demonstrates that the organization is striving to be true to itself and its audience and community.   I have learned in my 30 or so years of experience that when a museum applies laser focus to the center of the cycle, it creates an opportunity to achieve meaningful, measurable results that an evaluator can detect.

We invite you to share your thoughts—agreements or disagreements—in the spirit of our collective learning, as learning has always been the motivational force behind our work.

Read Full Post »