25th Anniversary ButterflyFor me, intentionality, a concept I view as essential to museum planning, emerged from two core experiences: results from hundreds of exhibition and program evaluations; and observing museum staff wanting to put too many concepts into an exhibition. Intuitively I knew there was a connection between exhibitions that didn’t fare too well (at least according to the evaluations) and staff not letting go of ideas that are near and dear to their hearts—regardless of whether those ideas supported the thesis of the exhibition.

When I have the good fortune to attend planning meetings, I always find myself thinking critically about what should be included in the exhibition under discussion and what could be saved for another time. My consideration always includes the big idea of the exhibition, what the museum would like to achieve with the exhibition vis-à-vis the public, humans’ capacity to process new ideas when in unfamiliar environments (like that of an exhibition hall), evaluation results from other projects that show what leads to quality visitor experiences and what might move visitors away from having quality experiences, and my utmost respect for scholars’ knowledge and passions. While passionate individuals love their subject matter (and really, I love their subject matter, too), one’s willingness to recognize that not all good ideas (or even great ones) belong in an exhibition and then exercising follow through are traits of intentional practice.

Embedded in intentional practice is the concept of alignment—ensuring that project concepts, components, and elements are present because they support the impact the team wants to achieve. If there are concepts, components, elements that do not contribute to the Intentionalitycore idea of the exhibition and its potential impact on audiences, they need to be omitted. I certainly don’t mean to sound ruthless, but I am acutely aware of how easy it is to keep putting more and more into an exhibition plan and how painfully difficult it is to take anything away. I am also aware of how challenging it is to stay focused on the exhibition’s big idea and have the discipline to say no to ideas because they do not support the intended impact of the exhibition. Learning to say “no” is a necessary survival skill and saying “no” is deeply connected to intentional practice. When practitioners are intentional, they are focused on the impact they want to achieve; they exercise discipline and restraint when determining how to best move forward; and their decision making is egoless and for the sake of achieving the results the team envisions.

Intentional practice represents the culmination of my experiences to date, and my passion for it is directly is tied to my evaluation experience. Over the years I started to realize that when exhibitions tried to do too much, visitors’ experiences didn’t amount to much—from their perspective; their heads were full but descriptions of their experiences were nebulous. Sense-making seemed futile. Nudging me was my memory of exhibition development discussions and tensions about what to put in (everything) and what to take out (nothing). Clarifying the intent of the exhibition and then staying focused on the intent of the exhibition is hard work—not likely to end soon, which is okay. Seeking clarity—whether in thought or action—is a never-ending pursuit.

We have been thinking about intentional practice a lot lately.  The article below, written by Randi, appeared in ASTC Dimensions May/June 2008 issue.  If you would like to read more of Randi’s thoughts on intentional practice, be sure to read her 2007 Curator article, “The Case for Holistic Intentionality.”

At museum conferences these days, people are talking about accountability, public impact, and relevance. These ideas are not new. A decade ago, in a 1997 keynote address for the Mid-Atlantic Association of Museums’ 50th anniversary, the late Smithsonian scholar Stephen Weil spoke of the “in-your-face, bottom-line, hard-nosed questions”—the ones that museums often hope to keep under wraps: “Do museums really matter? Can and do museums make a difference?”In arguing that some museums do make a difference, and that all should strive to do so, Weil supported the notion that “the very things that make a museum good are its intent to make a ‘positive difference in the quality of people’s lives.’” He borrowed this last phrase from the United Way of America, which was then challenging its grantees to document the benefits a given program had made in their lives.

Today, museums face accountability questions from many directions. In response to the Government Performance and Results Act of 1993, U.S. federal agencies began to articulate the kinds of outcomes they expected grantees to document. Private foundations followed suit, reexamining their own evaluation practices, as well as those of grantees. The effort continues. The National Science Foundation recently published its Framework for Evaluating Impacts of Informal Science Education Projects, outlining five categories of impact it expects grantees to assess. And for those who object that “you can’t measure mission-centered work,” current United Way CEO Brian Gallagher, as reported in the Wall Street Journal, has a succinct reply: “You most certainly can. The question is, ‘Are you committed to do it?’ And then, ‘Are you committed to report on it?’”

As museums begin to grapple with their intent to make a positive difference, they can start by reexamining their museum’s mission. Weil believed, as many do still, that a mission is key to an institution’s success. A museum’s mission should be a declaration of its core purpose—clarifying what the museum values, reflecting what the museum embodies, and describing its intent to affect its public and community. Establishing a clear institutional purpose, Weil believed, is the first step to being able to assess effectiveness in achieving public impact.

From my own experience as an evaluator, I would add this observation: Museums do not, in and of themselves, value, reflect, or intend. People do.

An institution’s mission will not be within reach unless everyone who works in that institution is mission-focused and mission-driven. Before museums can assess their impact, staff must collectively clarify their intent. Public impact, relevance, and value grow from what I have called “intentional practice”—the willingness of everyone in the museum to examine all operational activities through three mission-based filters: clarity of intent, alignment of practice and resources, and reflective inquiry.

  • Clarity of intent. Opportunities for all staff to come together to discuss the core values of their museum are vital. Colleagues should both encourage others to explore their passions and also challenge others’ thinking as a way of clarifying what is truly of importance.  In the spirit of thoughtful inquiry, why not ask a colleague to defend his or her position? Most people appreciate being asked to explain why they think the way they do. This kind of exploration allows practitioners to voice the passion behind their ideas and learn what they, as a group, really care about. Reexamining the essence of the museum together can reinvigorate the collaborative spirit, enabling staff to further their practice with intent.
  • Alignment of practices and resources. Unless the work of the museum is aligned with its intent, staff may spend time and resources on activities that are good in themselves but may not support the museum’s intent. Perhaps staff should determine—through evaluation—which programs yield the highest impact, keep those programs, and either improve or discontinue those that do not deliver impact. Aligning practice—the activities a museum does and how it does them—and resources so they support the museum’s intent requires thinking about what you should be doing and what you need not do any more. Conversations about realignment will deepen staff members’ understanding of the museum’s intent and the ways in which their work supports it.
  • Reflective inquiry. As an evaluator, I frequently see front-end and formative evaluation being used effectively to shape a final visitor experience. The same cannot be said of summative evaluation. By the time a mandated final report is done, practitioners may have little time or motivation to review it. This is unfortunate because much can be learned through reflecting on past work.

I see a strong relationship between taking the time to think about the work you have done and learning from the work you have done. Practitioners who want to be intentional in their practice can use summative evaluation as a way to gain insight and knowledge about visitors’ perspectives and experiences. The outcome of such reflective inquiry is learning about the ways in which their museum is achieving impact. I would encourage all museums to routinely set aside time for staff to use inquiry as a reflection strategy and to discuss their practice in the context of the institution’s intent.

In conclusion, accountability questions are not likely to disappear, but even if they did, museum practitioners would still need to respond to the “To what end?” question. The sustainable health of the museum depends on it.

Most of the workers I encounter in museums are passionate about their work and want to make a positive difference in people’s lives. If practitioners begin collaborating with colleagues to clarify their museum’s intent, realign their practices and resources to support that intent, and engage in reflective inquiry to learn how they can improve their efforts, they will be on their way to achieving that goal.

25th Anniversary ButterflyI’ve always loved solving puzzles. And to me, people are the most fascinating puzzle of all. Perhaps that’s why I studied two people-focused topics as an undergraduate- biological anthropology and history. Not only was I curious to learn about how our evolutionary past has shaped human behavior, but I also wanted to understand how our cultural experiences affect those behaviors.

To this point, one of the great things about museums is that they bring people together to share ideas and learn from one another. I’ve always thought of museums as some of the most innovative and fun places to spend my time. They’re also fascinating places to explore people. Where else can you encounter a blend of different age groups, cultural histories, educational backgrounds, learning styles, hands-on experiences, fascinating artifacts and ideas?

I like to think of evaluation as having a continuous set of “people puzzles” to solve. How do people behave in and use museums? What motivates their behaviors? What does learning look like in a museum? How can a museum craft meaningful experiences for the people that walk through its doors? What do those experiences look like?

The solutions to these puzzles are plentiful and ever-changing and that’s what I love about them. As someone who’s relatively new to the field, I’m excited to have the chance to observe and interpret peoples’ often mysterious behaviors to help museums continually create experiences that are truly responsive to their visitors’ needs.

But perhaps most importantly, as a new evaluator, I’m excited by the opportunity to help those that work in museums learn to continually ask this ever-evolving set of questions themselves. Because underlying our field of people watching and data analyzing is the idea that it’s perfectly alright to not have the solutions, as long as you’re willing to ask questions in order to uncover them and learn from mistakes along the way.

So, although as an evaluator I may never be able to definitively solve these people puzzles, I can help museums and their visitors understand and relate to each other in new and unanticipated ways.

25th Anniversary ButterflyOne of the things I love about my job is the part where I need to find order in chaos, such as the trends in a large sample of individual in-depth interviews, and refining our processes for collecting and analyzing data in order to find that order more efficiently. I’d say that I have always had this tendency towards order and process. As a little girl, I could spend hours lining up figurines in a single line and moving them one by one like they are marching in a parade. I have learned immensely from and with my colleagues at RK&A particularly in regard to process. And while I can think of several examples about how we have refined process, my favorite example is Johanna Jones’s work with timing and tracking. Johanna was with RK&A for 13 years and over that span of time she honed RK&A’s timing and tracking processes to a truly distinct approach, learning a great deal from others like Beverly Serrell, who Randi considers the queen of timing and tracking.

photo

Timing and tracking was something that hooked me into the museum evaluation field. I remember being completely fascinated by Arthur Melton’s timing and tracking studies when I learned about them in grad school. As I was formulating my thesis research topic, I knew that I wanted to incorporate timing and tracking just to have the opportunity to try it. For my thesis research, I timed and tracked art museum visitors to determine whether security guards affected visitors’ movement in the gallery. I had meticulously drawn maps full of dots and lines to track a visitor’s and security guard’s behaviors, plus I wrote open-ended notes about what the security guard was doing in the space. The data was wholly exhausting to collect and time consuming to analyze, but I felt like it gave me a handle for what timing and tracking was. . .

 

. . . Then, Johanna shared with me a timing and tracking form for one of my earliest projects at RK&A. I was completely blown away by it. It was neat, clean, concise, and focused. Every component in the exhibition was clearly identified with columns to indicate when a visitor started and stopped using it. There was also a short list of behaviors that we looked for at each exhibit. It was perfect! Elegant even. I still marvel at how refined the timing and tracking process became and strive to apply that same clarity and creativity to thinking about the various other processes that make up the work we do.

25th Anniversary ButterflyYou can’t escape technology in museums. Visitors use smartphones to take pictures. Exhibits use touch screens and high-tech interactives to share stories and information. Programs use technology to help visitors engage. Everywhere you look there is a screen . . . until you encounter an evaluator armed with a clipboard and a pencil. I don’t think this is because evaluators are luddites. I think this is because, much like exhibit and program designers, we want our use of technology to make sense. Technology has to make data collection easier not only for us, but for the visitor, too.

I have been using technology for data collection since I was in grad school. We had grant money to spend, so iPads were purchased and students were encouraged to try new things. We approached the task with gusto, certain that this would make things easier; after all, if we enter visitor data as we collect it, we will have eliminated the need to enter data later! It sounded like the perfect plan, until we realized we were collecting data outside—in the elements. In Seattle. In November. We learned a few lessons that day: iPads don’t do well in the cold and neither do the cold, bare fingers we needed for the touch screens. Perhaps in this case, using iPads wasn’t the best plan, given our data collection environment.

Since joining RK&A two years ago, we have experimented with technology as well. Each time we elect to use technology, we think about how it will affect the project. In some cases, it is simple – if we take interview notes on a laptop or tablet while talking to visitors, we can record more of what the visitor is saying and eliminate some work on the back end. This is a low-risk decision that we frequently make. In other cases, we rely much more heavily on technology by using tablets to collect survey data at museums. This is a higher-risk decision because while there are many positive aspects to non-paper data collection, there are also challenges.

Tablet data collection requires me to think about survey presentation in a different way. The survey is designed mostlytechnology for the data collector since we often administer surveys verbally, so it has to be easy to manipulate. Some question formats, such as the scales that RK&A often creates, which are non-traditional scales, don’t always translate well from paper to digital. I have to think about how the question is presented for the data collector, and what, if any, information they have to present to the visitor (e.g., a visual representation of the scale), then find a way to balance the two. Also, we ask the visitor to enter their own demographic information, formerly a single page of questions. But when collecting data on a tablet, we need to balance how much the visitor has to scroll with the number of pages they have to click through. This can be tricky when skip logic directs visitors to the appropriate questions. Regardless of the demographic information the visitor inputs, the survey has to be easy for them to complete. Each time, I learn something I need to change in future surveys to make it easier for visitors.

There are huge advances taking place in digital data collection as new software and platforms are created, and we researchers and evaluators develop best practices for their use. For me, every project that uses technology to help with data collection teaches me something new and makes me a better practitioner. What experiences have you had and what have you learned? The field is changing and I can’t wait to see what we think of next.

In March, Intentional Museum announced it’s first blog competition, asking students to reflect on the following question: Through your intentional practice, how do you help enrich the lives of others?  Below you will find the winning post from Faithe Miller McCreery.  Faithe is a graduate student at the University of Washington in Seattle where she is a member of the Museology class of 2015. 

faithe_collections2I’m a collections girl. At this point in my career, most of my professional and volunteer experience has centered on digitizing, cataloging, rehousing, and otherwise maintaining artifacts and archives. This nearly always involves entering the museum through a back door, taking a staff-only stairwell down to the basement, and cozying up to a collection for some quality one-on-one time.

Part of me loves this. I appreciate having a private little space where I can hunker down and concentrate on projects with few distractions. But getting into the collections basement routine can also be very isolating. At times I find myself developing a sort of artifact-based myopia: I am very attentive to the work at hand, but I begin to forget why I’ve chosen to work with collections in the first place. When I’m alone in a lab with a roll of film negatives, I have to remind myself that I’m not in this career for the negatives themselves: I chose museums because I want to inspire, educate, and empower people.

As an intern for the Klondike Gold Rush National Historical Park – Seattle Unit, my responsibilities include occasionally manning the Visitor Services desk at the front of the museum. While I’m there, I greet incoming visitors, direct them to specific exhibits, and just generally chat about any number of subjects that may arise. Technically, staffing the front desk does not fall under the auspices of either the Interpretation or Curatorial departments toward which my internship is tailored. My time there arose purely to fulfill a staffing gap at the museum, and it was a task that I initially dreaded as taking attention away from my “actual” duties. The truth is, though, interacting directly with visitors has become something of a respite from the quiet and isolated collections environment where I spend the majority of my time. Visitor Services has become a part of my work that I quite look forward to.

When I see the unbridled enthusiasm of children who just can’t wait to explore local history, listen to older people share stories of their own relatives’ experiences panning for gold, or welcome a group of tourists into their first cultural institution in Seattle, I develop a far greater appreciation for the work that I do with collections. Interacting with visitors serves as a much-needed reminder for me that ultimately, collections lose their purpose if they become unbridled from the institutional mission. The work that I do with collections occurs not in spite of visitors, but for them –and having an eye on the front desk helps me understand the needs, questions, and concerns that I should be addressing while I toil away in Curatorial. It may seem paradoxical, but I’m so glad to have learned that the more time I spend outside of the basement, the more I appreciate the significance of the work that I do while I’m there. Great things happen in the basement –but let’s not discount the ground level, too.

 

Effective and cle25th Anniversary Butterflyar communication is a skill that all evaluators must master but sometimes those of us in the evaluation field forget that we may be speaking a foreign language to our clients. We become comfortable with acronyms like IRB or throw around names for data collection methods like surveys, focus groups, naturalistic observations, and ethnography to name a few—not realizing that those words can have different meanings to different people. Precision of language is vitally important, and it goes hand-in-hand with another skill that, over time, I have come to respect—that of active listening. As evaluators, we need to listen to ourselves, visitors, our colleagues, and our clients. To me, active listening is listening first to understand and then responding, and I have found it difficult to do especially when I feel that I have something important to say. We’ve all been there I think; those moments when you start to tune out someone because you are searching for the right opening to say what’s been on your mind for the last several minutes. It’s fairly easy to spot when someone is not actively listening because their comments result in non sequiturs.

One of the challenges with active listening is that it is mentally exhausting. It’s much easier to let your brain relax at regular intelistening2rvals rather than to be constantly aware of what every other person is saying. When conducting in-depth interviews, asking visitors open-ended questions about an exhibition or program, which are intended to result in visitor-centered conversations, we are “on” the entire time and do very little talking ourselves. We are very careful to train our interviewers to actively listen to visitors’ responses so they can discern whether visitors are responding to the questions or whether their responses require additional questions; simultaneously they also have to make sure they understand what the visitor means by the words he or she uses—which is the essence of understanding. At the end of a day of interviewing, I warn my data collectors that they will feel mentally exhausted, because I have experienced this kind of fatigue so many times before. This one example points to a key tenet of active listening—you, as the listener, really do not say much at all; rather your primary job is to listen to understand and ask questions to seek clarity if you don’t understand.

With our clients, active listening is important, too. It’s one of the many things I’ve learned to do while working at RK&A. I learned to actively listen first when I was an interviewer (which I still like to do when the project calls for it because it is a great reminder of the importance of active listening). Then, I learned the key role active listening plays in client meetings and intentional planning workshops we facilitate. Most of what we do as evaluators is ask questions, listen, and then ask more questions to seek further clarity. The mental exhaustion I feel at the end of the day is a good sign that I have done my job as an active listener. I’d be concerned if that feeling were to ever disappear.

Follow

Get every new post delivered to your Inbox.

Join 121 other followers