Much of Intentional Practice work is about process, and a significant part of process work requires that we talk with each other.  I realize that humans exchange ideas verbally all the time, although given we live in the screen age (computers, phones, and pads), perhaps people are conversing face-to-face less and less.  And perhaps, as well, we need practice convening and having productive conversations.  This third principle of the seven principles of Intentional Practice is staff using inquiry and active listening to understand and appreciate varying viewpoints; it is about having productive conversations that support the work of your museum.  It is a principle because open, productive conversations among staff are necessary if a museum is to achieve impact.

#3: Staff use inquiry and active listening to understand and appreciate varying viewpoints.

RK&A uses inquiry (e.g., ask open-ended questions) for several reasons:  1) inquiry, with a few ground rules, creates a neutralizing and democratizing atmosphere that invites and welcomes all viewpoints; 2) inquiry promotes others to ask questions, and it is through conversation and dialogue that social, professional, and personal learning emerges; and 3) inquiry allows staff to come to their own understanding about an issue or topic; and 4) asking well-articulated and purposeful questions allows everyone to explore their thoughts, come to know their perspectives and the perspectives of others in their group, and reach a collective appreciation for all ways of knowing.  I realize that my view of inquiry and active listening may sound Pollyannaish, but because I have seen such conversations reach useful and invigorating ends, I believe fully in the principle.  I realize, as well, that the four points above are complicated and deserve their own individual blog post, if not a chapter in a book (which is forthcoming by the way—in about 18 months if all goes as planned), but, for today’s post, I have chosen to focus on the ground rules, as without rules of engagement, conversations easily can go awry. 

Ground rule #1: Participate with authenticity
First, in workshops we invite and encourage all workshop participants to contribute with authenticity.  We ask all participants to respect all others in the gathering as well as the purpose of the gathering; authentic and genuine participation is essential to a successful planning process.  We want to hear everyone’s genuine thoughts about the ideas under discussion.

Ground rule #2:  Listen to first understand, then respond
Second, while we encourage everyone to be themselves, we also request that everyone be respectful as the conversation ensues.  To that end, we encourage listening to understand—not to respond, at least not right away.  We recommend allowing ample time to process and understand before responding, so as to avoid knee-jerk reactions to potentially unpopular positions.  We respect what all individuals bring to a situation, and we recognize that all bring a unique and valuable intellect, commitment, passion, and experience to the group conversation.  If someone does not fully understand what someone is saying, we support asking additional questions to clarify a point that may not have been clearly expressed initially.

Ground rule #3: Realize process work is an art and science

Finally, the Cycle of Intentional Practice work is iterative and process-oriented.  Process work can be messy, and it also can be uncomfortable for some.  While most of the gatherings (e.g., workshops) may have a defined agenda, framework, and theory to support the work, we cannot anticipate exactly what will unfold throughout the course of the workshop; if we knew, we would be dismissing the uniqueness of the institution and individuals’ contributions.  The art emerges as we all respond to each other’s thoughts and we end up in an unexpected place of understanding; the art also emerges, with a little bit of science, in how we ask the questions, ensuring that we and others are asking balanced, non-leading, and unbiased questions.

 

Coffee Break Icon

Recently at RK&A, we’ve started gathering as an office for a bimonthly “Coffee Break” to discuss current issues and recent developments in the field.  For each Coffee Break, we select a current article, blog post, or other publication for everyone to read and reflect on.  So far, topics have ranged from data visualization to the role of museums as social innovators, and it has been fascinating to reflect on the museum field and our place in it with my colleagues.  We would like to invite you to learn along with us as we explore current topics and trends in the field in our “Coffee Break” blog series.

Last week, we discussed John Wetenhall’s article “Why Not to Run Your Museum ‘More Like a Business’” in AAM’s recent Museum issue (May/June 2017).  In the article, Wetenhall considers the increasing pressure some museums feel to run “more like a business.”  Business models value growth and often operate with the perspective that “bigger is better.”  When applied to museums, “bigger” takes many forms—growing attendance, membership, collections, and buildings—and as we discussed the article, we wondered along with Wetenhall, “But where does this lead?”  When a museum’s annual report presents its achievements through numbers, square footage, and the like, they do not reflect the museum’s deeper mission and purpose.

What resonated most with my work, as an evaluator, was Wetenhall’s call to action to “temper measures of quantity with the matrices of quality and impact.”  The “endless quest for more…weakens our institutions in the long run” because museums lose sight of their true mission—to make a difference in the lives of their visitors.  We work with museums and other informal learning institutions to help them articulate and achieve their intended impact on visitors, whatever that may be—whether to inspire, to enrich, to explore, or to teach.   The quantitative and qualitative data generated through evaluation can provide a metric of value that speaks more directly to a museum’s mission and impact on visitors than, for example, program attendance.  We hope our work helps museums focus on impact, improve the quality of visitor experiences, and demonstrate their public value in meaningful ways.

As you may recall, for the next several months I will be highlighting the seven principles of Intentional Practice.  Last month I wrote about the principle #1: The organization wants to achieve something greater than itself (e.g., impact) among the audiences it serves, and this month I discuss the importance of collaboration to Intentional Practice.

#2: Staff work collaboratively across the organization.

Not too long ago in museum history, exhibitions were developed by a lone curator.  The idea to use interdisciplinary teams to develop exhibitions was instigated at the Field Museum in the early 1980s, and the rest, as they say, is history.  Although some museums still struggle to implement and maintain what has become known as “the team approach” to exhibition development, significant progress has been made in the last 30 years.  Initially, this shift in practice was in response to the national trend that was taking hold whereby museums were starting to embrace the public dimension of their work, recognizing that successful communication of complex ideas might require more than a subject matter specialist.

My belief in the necessity of collaboration grew from my experience conducting hundreds of evaluations over the last 30-plus years.  For an exhibition to be successful, for example, all the following stars would need to be aligned: the exhibition team would have a thesis (e.g., a one-sentence big idea), identify a few messages that support the thesis, have coherent interpretive text that communicates the thesis and messages, have tested interpretive elements (e.g., interactives, text) to ensure they communicate the thesis and messages, have adequate resources (time and dollars) to actualize the exhibition as intended, and have the right combination of expertise and skills to execute the exhibition as intended.  Hard, complicated, and messy work, indeed.

Puzzle graphic created by Davo Sime from the Noun Project

Applying my evaluation experience to intentionality, I realized that if a museum was to achieve a discernible impact among audiences, the entire museum would need to participate.  Achieving impact, which in the case of Intentional Practice means making a positive difference in the quality of people’s lives, might not happen if one person or one department bears all the responsibility; instead, achieving impact should be the collective effort of all staff working together towards an agreed-upon end.  To capitalize on the benefits of collaboration, we request our clients create working groups comprised of individuals from different departments when we facilitate Intentional Practice workshops; we seek interdisciplinary collaboration because we want people to problem solve with those whom they might not normally choose to work.  What is the result of people thinking alongside colleagues from other departments?  Increased appreciation for the unique perspectives and skills their colleagues offer!  They slowly recognize that their collective brain power and actions are stronger than any one individual’s work.

Cover of Forces for Good, by Leslie R. Crutchfield

Leslie Crutchfield and Heather McLeod Grant demonstrate the benefits of collaboration in their well-researched book Forces for Good.  They studied 12 high-impact nonprofits to understand what made these nonprofits successful, and yes, high-impact nonprofits collaborate with other nonprofits that have shared goals.  Their updated research appears in the 2012 edition of the book and in this article called “Local Forces for Good.”  They make the point that no one can or should do the work alone, and I agree.

As anyone who has experienced the team approach to exhibition development will know, process work can feel disorganized.  Museum leaders can mitigate confusion by clearly communicating who does what and modeling patience and trust.  From an Intentional Practice perspective, all staff are apt to benefit from working collaboratively, as collaboration increases the chance for an individual’s professional and personal learning.

In keeping with this year’s blog series about how my Intentional Practice has evolved over the last 10 years, I will be using the next seven months to present the seven principles of Intentional Practice.  The emergence of these principles was organic; I did not set out to identify these principles prior to embarking on this work—the list just came to me one day last summer.  In fact, I had forgotten that I had even written the list until I was cleaning up my Intentional Practice folder on my computer last week.  To my surprise and delight, there it was!  Suffice it to say, over the next seven months I will mull over the principles, which may shift or change as I clarify my thinking.  For that reason, I will share one per month.

#1: The organization wants to achieve something greater than itself (e.g., impact) among the audiences it serves.

 

The first principle is a prerequisite for Intentional Planning; and a museum cannot move forward in Intentional Practice if it isn’t interested in working for the common good.  Clarifying intended impact isn’t about the museum benefiting; it is about the public—the recipient of the museum’s work—benefiting.  Even the statement, “People become life-long museum visitors” doesn’t place the benefit solely on the museum visitor, as repeated visitation is a means to a greater end—for the visitor.  Achieving impact is about making a difference in people’s lives, which requires the full force of the museum behind it.  A museum that is insular, self-serving, or arrogant may not be able to pursue Intentional Planning.  Likewise, a museum with a relentless focus on the bottom line may thwart Intentional Practice work, not because it wants to but rather, persistent attention on the bottom line has a funny way of interfering with integrity and ingenuity.  People may inadvertently revert to traditional ways, which for some museums may mean looking inward rather than outward.  Fear might overtake confidence, risk-taking might disappear, and working on behalf of the bottom line might seem like the only survival strategy available on the horizon.  While organizations can balance bottom-line concerns with achieving something greater than themselves, more times than not, organizations create an either/or situation rather than an “and” situation.

rk-blog-post-1

The New Museum by John Cotton Dana

The idea of a museum thinking outside of itself for the common good is an age-old idea in museums that holds value and importance today.  A century ago, John Cotton Dana said, “A museum is good only insofar it is of use”—a statement that is often quoted today by museum staff who want their museums to be viewed as convening places where people can gather to have important conversations about contemporary issues.  Dana’s many important writings are compiled in a book called The New Museum (1999) published by the Newark Museum, and they are worth reading.

Stephen Weil

Stephen Weil

And, in Making Museums Matter (2002), noted scholar and museum director Stephen Weil writes in the chapter “Can and do they make a difference” that: “If our museums are not being operated with the ultimate goal of improving people’s lives, on what alternative basis might we possibly ask for public support?”  In this piece and several others, Weil makes a case for museums to do their work to “make a positive difference in the quality of people’s lives,” which is how all of us at RK&A define impact.

In 1996, Harold Skramstad, former director of Henry Ford and Greenfield Village in Dearborn, MI, in a presentation during the 150th celebration of the Smithsonian, noted that mission statements, which museums like to use to demonstrate their purpose, do not answer the “so what?” question.  Museums spend a lot of time agonizing over their mission and visions statements (both of which are about the museum), when it might make more sense to use some of that time thinking about the impact they want to achieve on audiences.

The “so what” question is a running theme, at least implicitly, in Emlyn Koster’s writings; Emlyn, Executive Director of the North Carolina Museum of Natural Sciences, writes about “relevance” as the necessary element that museums in today’s world must boldly embrace.  For me, relevance is connected to the concept of achieving impact, as audiences will benefit from a museum that is relevant to their lives.  I suggest reading these two pieces by Koster, neither of which are available digitally for free: “In search of relevance: Science centers as innovators in the evolution of museums” in Daedalus, 1999; and “The Relevant Museum: A reflection on Sustainability” in Museum News, 2006. Both make a case for relevance as a necessary requirement for today’s museums. Emlyn also makes the point that sustainability of our planet is the relevant topic for science museums.  I believe he is right.

Relevance also is a viable approach to organizational sustainability for any museum, as maintaining the relevance of what your organization does for its audiences will keep your museum fresh, contemporary, and most important—purposeful and meaningful to your audiences.

Ten years have passed since “The Case for Holistic Intentionality” appeared in Curator.  On the one hand, 10 years isn’t that long ago, but on the other hand, a lot has changed in how I think about intentionality.  The article (actually written 12 years ago) presents a concept about the characteristics of an intentional museum and makes a case for such organizations.  What the article had not benefited from—since it was only a concept rather than proof of concept—was my experience helping museums move towards intentional practice.

My colleague, Stephanie Downey, suggested I write 12 blog posts this year—one each month—to share the Intentional Practice strategies we developed and continue to hone and implement with museums.  She thought that this year of reflection and sharing would support the work I already will be doing as I spend this next year writing a book on Intentional Practice.  This undertaking has been in my mind for a while, and I’m excited that I have finally committed myself to this task.

Honestly, what is difficult about applying words to ideas is that the very nature of Intentional Practice presumes nothing is stationary.  Ideas are fluid, strategies are ever changing, the external environment is in constant flux, and learning is continuous.  Much like the law of physics that says everything is in constant motion, my ideas about intentionality and Intentional Practice are forever changing—not in big discernable ways (I might be the only one who notices), but in little ways.  My thinking changes almost daily, which isn’t a bad thing, except if I want to write about it!  At some point I will have to say the acronym, ELMO, “Enough, Let’s Move On”—something I learned from a museum professional who was in one of the first Intentional Practice workshops. ELMO comes in very handy, as you can imagine!

The Cycle of Intentional Practice, presented in a blog posted on January 2, 2013, has changed considerably, at least to me.  Ten years ago, Curator didn’t want to include the graphic in the article, and it is only now that I am grateful.  This is what it looked like in 2013:

Cycle of Intentional Practice

And this is what it looks like today:

 

The most significant shift (aside from its cleaner look) (thank you Amanda Krantz and Cathy Sigmond) is that there are quadrants. I always described the cycle as having quadrants but only recently does the cycle have them.  The order of the quadrants is also different, align now follows reflect instead of plan.  How odd it seems to me now that this wasn’t the original order.  Alignment makes sense after reflection—when you ask how you can align your actions to achieve impact—after reflecting on evaluation results.  Three concepts are unchanged: impact remains the centerpiece of the cycle; one can start anywhere on the cycle; and none of the quadrants are mutually exclusive, as one can reflect when planning, evaluating, and aligning.

While those ideas seem stationary today; what will I think tomorrow?  ELMO!

I look forward to the coming months when I will be sharing my thoughts about how my intentional practice work has evolved.

While waiting to get my hands on Nina Simon’s newest book, The Art of Relevance, I enjoyed working my way through her blog, Museums 2.0.  I was especially touched by a pair of posts I’d read just before Labor Day weekend: (1) a “sneak peek” of The Art of Relevance; and (2) an honest, reflective post about her vacation from 2008, in which she examines the field-wide conflict between elitism and inclusivity in the context of her experience at Yellowstone National Park.  I thought about these posts while biking on the Mt. Vernon Trail that weekend because the trail’s ultra-accessibility (parking lots, paved walkways, picnic tables, etc.) makes for a crowded ride. I almost wished that everyone else would just go away so I could enjoy zipping along the trail’s curves at top speed.  When I felt annoyed after navigating around joggers, walkers, other cyclists, and picnickers of all ages sprawled along the trail, I found myself reexamining my mindset in the context of Simon’s reflection:

Yellowstone…was an access dream—and my nightmare. You could drive right up to the geysers…I hated it…On this trip, for the first time, I truly understood the position of people who disagree with me, those who feel that eating and boisterous talking in museums is not only undesirable but violating and painful…I get it now. I felt it at Yellowstone.”

So how can I, as an advocate for accessibility and relevance in museums and parks, reconcile my advocacy with my attitude?  For parks and museums, there is value in hosting a range of people who fall at different points on a spectrum of museum literacy.  In some of our studies, RK&A helps museums identify different “clusters” of visitors, understood by their ranges of prior knowledge, conceptions, and attitudes.  Audience segmentation allows the museum to meet visitors “where they are,” welcoming people with all levels of museum experience; one segment is not necessarily more ideal than the other.   According to theorists in educational psychology, in classrooms and beyond, peers with different levels of mastery of the same subject can help each other learn.  For example, Lev Vygotsky’s concept of the Zone of Proximal Development (ZPD) refers to the difference between what someone can do on their own, and what they can do with help.  So, those more comfortable with using the museum or park can actually help less experienced users who want to engage with museums to learn more than what they could learn on their own.  (Not to mention, teaching someone something new causes the more experienced person to understand the subject more deeply, too.)  While museums and parks are considering how to best serve different audiences, what opportunities can they create for visitors to “scaffold” for each other, stretching the value of visitors’ museum experiences beyond themselves?

As museums can facilitate opportunities for visitors to grow and learn together, evaluators can scaffold for museums’ development in understanding visitors, too.  Research shows that experiential learning, much like the learning opportunities offered by many parks and museums, is a powerful tool for enhancing our empathetic abilities.  After experiencing a “Yellowstone” moment, I have a better appreciation for the impulse to preserving the authenticity of place or experience I hold dear or sacred, and why we might feel reluctant to welcome people who might not only have varying amounts of experience visiting an art museum, but also have different ways of using a museum or park than you or I do.  Gretchen Jennings reminds us to build our capacity for empathy by remembering “when we have felt like part of an ‘out-group,’ to savor those experiences…that show that our institutions can empathize with the concerns of their audiences.”  Only two years ago, I was brand new at navigating multi-use trails; I remember what it’s like to feel like I didn’t belong on a trail that I now know well and use with ease.

Evaluators move fluidly between empathizing with multiple audiences, sharing the visitor experience with the museum in ways their staff find understandable, meaningful, and useful—stretching them just beyond the realm of what they already know about their visitors.  As a new team member at RK&A, I’ve been observing my colleagues as conduits who transmit information about a museum’s audience to the museum staff who are responsible for enhancing visitors’ experiences.  Evaluators fall in between the visitor and the museum, facilitating a relationship between two entities that seek to better understand the other party.  That understanding can come from walking in either the museum’s or the visitor’s shoes—or, in the case of the evaluator, by wearing both.

img_6510

A calmer moment on the Mount Vernon Trail.

RKA Blog_Evaluation Design Series_Logo

This spring, RK&A undertook an ambitious project with the Smithsonian National Museum of Natural History (NMNH) to conduct a meta-analysis of reports from the last 10 years of evaluation completed for the museum.  In this context, “meta-analysis” essentially means reanalyzing past analyses with the goal of identifying larger trends or gaps in research.  This project was both challenging and rewarding, and so I wanted to share our experience on the blog.

The specific goals for this project were to:

  • Understand consistencies or inconsistencies in findings across reports;
  • Identify areas of interest for further study;
  • Help the museum build on its existing knowledge base; and
  • Create a more standardized framework for future evaluations that would help the museum continue building its knowledge base by connecting past studies to present and future evaluations.

The first step of the meta-analysis process was to perform an initial review of the reports and determine criteria for inclusion in the analysis.  One of the underlying goals for the project was to demonstrate to the institution at large (not just the Exhibits and Education departments) that evaluation is a useful, scientific, and rigorous tool that can inform future work at the museum.  Therefore, we wanted to make sure that the evaluation reports included in the study adhered to these high standards.

For this reason, we omitted a few reports that we considered casual “explorations” of an exhibition or component type, rather than a systematic study using acceptable evaluation and research protocols.  For example, an “exploration” might consist of a random short observation of an exhibition and casual conversations with a small number of visitors about their experiences.  While these types of studies can be useful and informative on small-scale projects, they were not rigorous enough to support the larger goals of this project.

We also omitted reports in which the sampling and data collection methods were not clearly stated, because this left us unsure of exactly who was recruited, how they were recruited, and how that data were collected (e.g., Were the observations cued or uncued?  What instrument was used?).  Although these studies may have been rigorous, there is no way for us to know without a clear statement of the methodology in the report.

Next, we needed to develop a framework to use for analyzing and comparing evaluations.  Over the course of several meetings with NMNH, we discussed and clarified the ideas and outcomes that were most important to the museum.  Based on these discussions and a review of NMNH’s existing evaluation framework for public education offerings and the institution’s core messages, we developed a new evaluation framework which would be our analytic lens.  The new framework centered on four main categories, with the most emphasis placed on the Influence category:Metaanalysis

Within the Influence category, we looked at a number of specific outcomes that were important to NMNH, such as whether visitors are “awe-inspired” by what they encounter in the museum or whether visitors report becoming more involved in “preserving and sustaining” the natural world.  To show some of the challenges we faced in making comparisons across reports, I’ll highlight an example from one outcome—“Visitors are curious about the information and ideas presented in the exhibition.”

Understanding whether visitors are “curious” about the information and ideas presented in an exhibition was difficult because many evaluations did not explore visitors curiosity.  Instead, we had to think about what types of questions, visitor responses, and visitor behaviors might serve as proxy indicators that visitors were curious about what they had seen or experienced.  For example, audience research studies conducted between 2010 and 2014 at NMNH asked entering visitors “Which of these experiences are you especially looking forward to in the National Museum of Natural History today” and exiting visitors “Which of these experiences were especially satisfying to you in the National Museum of Natural History today.”  We decided that visitors who indicated they were especially looking forward to (entering) or satisfied by (exiting) “enriching my understanding” may be considered “curious” to learn more about the content and ideas presented by the museum.  For other evaluations that didn’t explore elements associated with curiosity, we looked for indicators such as asking questions or seeking clarifications of staff and volunteers about something a visitor has seen in an exhibition.

However, we also acknowledge that visitors’ desire to “enrich” their “understanding” or “gain more information” about a topic does not always directly relate to curiosity.  For example, one evaluation that asked about both “curiosity” and “gaining information” found that an exhibition exceeded visitors’ expectations about having their “curiosity sparked” but fell short in “enriching understanding” or “gaining information.”  We learned from this that if curiosity is an important measure of NMNH’s influence on visitors, future evaluations should be clear in how they explore curiosity in their instruments and how they discuss it in their findings.

In light of the results of the meta-analysis, we are excited to see how NMNH uses the reporting tool we created from this work.  The tool standardizes the categories that evaluators and museum staff use to collect information and measure impact so the museum can build on its knowledge of the visitor experience and apply it to future exhibition and education practices.