25th Anniversary ButterflyLike me, you have probably seen dozens of friends and/or celebrities participating in the ALS Ice Bucket Challenge, may have participated yourself, or at least heard about it on the news. While the challenges have slowed since peaking about a week ago, they have reached broadly through my network. I was particularly surprised to see some of my non-American friends from the United Arab Emirates and Thailand participating just yesterday. An article from The Guardian noted in its headline, “The Ice Bucket Challenge for ALS is a gimmick—and it’s good.” The Challenge was intended to raise dollars for ALS research, and by all accounts, the ALS Challenge rings of success: as of August 26 date, the Challenge brought in 88.5 million dollars! True—raising dollars is an obvious output of the Challenge; however another is to raise awareness of ALS. The evaluator in me asks, “How much awareness has the Challenge actually raised (and how do we know)? Does raising awareness mean that people know the acronym ALS (but do they know what ALS stands for)? Does raising awareness mean that people know one fact about ALS? Do people know that it is a horrible disease that scientists are still trying to understand (and thus, that is why research dollars are needed)?

 

Raising awareness is often something museums desire as an end result for visitors who have seen an exhibition or attended a program. But raising awareness about something is an elusive concept—unless staff articulate what they mean by raising awareness. I have often heard museum professionals say that awareness is something that is hard to measure. But what I think is really difficult about measuring awareness is defining a realistic and appropriate lens through which to articulate awareness. For instance, if an exhibition wants to raise awareness of issues of social justice, is it successful in raising awareness only if everyone comes away with a thorough understanding of social justice issues in their community or strong feelings of advocacy for social justice issues? While I, curators, and exhibition designers may want to see everyone walk away from an exhibition about social justice take immediate action, it would be erroneous to say that is the only measure of success in raising awareness. I believe success is seeing any movement along a continuum from pre- to post-campaign, post-program, or post-exhibition. Certainly it is exciting and to see those big jumps across a continuum of awareness, but what I have learned along the way is that many people making little jumps or baby steps along a continuum are big successes.

My niece and daughter taking their baby steps together.

My niece and daughter taking their baby steps together.

Today’s Throwback Thursday comes from deep in the RK&A vault – a study we did in 2002 for the National Museum of American History (NMAH) in Washington, DC.

For Which It Stands: The American Flag in American Life [2002]

Study Context

The National Museum of American History, Bering Center (NMAH) asked RK&A in 2002 to concept test ideas for For Which It Stands: The American Flag in American Life, a new exhibition that would feature the Star-Spangled Banner, the flag that inspired the national anthem. The study’s objectives were to examine:

  • The range of meanings people attach to the flag
  • How participants think their meaning of the flag has been shaped
  • Participants’ tolerance for different meanings of the flag
  • Whether seeing objects and images selected for the exhibition causes people to see the flag in new ways
  • Whether people understand the changing, complex meaning of the flag.

Approach

NMAH is part of the Smithsonian, and, as such, it tries to accommodate visitors from all backgrounds and of all ages. Like many other museums, NMAH wants to attract teens and appeal to adults. Therefore, we tested the exhibition’s concepts using focus groups. We conducted three focus groups with teens and three focus groups with adults. Sixty individuals participated in the six groups.

Findings

Through discussions about the exhibition panels, participants were exposed to new ideas and stories about the American flag, which broadened their understanding of the flag. Participants freely shared their personal stories about what the American flag means to them and everyone said they enjoyed hearing other people’s stories and ideas and thinking about the American flag in new ways. However, individuals’ personal meanings of the flag were not altered as a result of their experience.

Conclusion

Although exhibition planners initially had wanted the exhibition to change people’s meaning of the American flag, they learned that participants’ beliefs about the flag were shaped by unique experiences deeply rooted in their identities. RK&A’s audience research demonstrated that visitors bring valuable experiences to museums and their stories can add depth to other visitors’ museum experiences.

25th Anniversary ButterflyAs we move further and further into the digital age, museums hold something that is becoming a rare commodity—real objects and artifacts. It may be hard to believe, but one day, many tangible objects may be obsolete, the way that printed photographs and airplane tickets are becoming scarce items. Instead of going on “digs,” future archaeologists may primarily use computer-driven devices to search for clues of our ancestors. In the distant future, I can imagine that museums will be magical places where people can see “the real thing.” …But wait, maybe they already are?

 

This is my third reflection—informed by what I have learned about museum visitors in all my years studying them. I have found that there are many reasons people visit museums, but I believe the primary reason, one that we may take for granted, is that they want to see “the real thing.” A common question heard in museums is, “is it real?” especially in regard to bones and historical objects. I have heard it in our research, but you have probably heard it too, or said it yourself while walking through a museum. Why do people ask this question? What underlies the need to know if something is “real.” As museum goers, can’t observing a replica of a dinosaur skeleton or a 17th century Dutch ice skate tell us just as much as the real thing? Maybe so. But there is just something about being in the presence of authentic artifacts and objects that is thrilling; maybe it has to do with feeling connected to other people, to the past, or to other parts of the world. Professors David Thelen and Roy Rosenzweig say it best in their landmark 1998 national study, The Presence of the Past: “approaching artifacts and sites on their own terms, visitors could cut through all the intervening stories, step around all the agendas that have been advanced in the meantime, and feel that they were experiencing a moment from the past almost as it had originally been experienced—and with none of the overwhelming distortions that they associated with movies and television, the other purveyors of immediacy.”

 

Whenever studying museum visitors, I come face to face with their sense of wonder about and desire to get clDinosaur Sueose to (even touch) real objects. Whether evaluating text panels, interactive exhibits, touch tables, or ideas and concepts, visitors will usually keep coming back to the objects. As museum professionals, it is sometimes too easy to forget the centrality of the object when you are knee-deep in trying to interpret and contextualize something. We can get lost in these various mediums of interpretation, but visitors will usually remind us what they are really there for. For example, I was doing a study for a museum and historic site last year in which we were testing ideas for high-tech touch tables intended to convey information about the historic building the museum is housed in. I had gotten so wrapped up in testing all the information, that I had mentally pushed aside the museum’s biggest asset, the historic building it resides in. But visitors brought me back when they practically skipped over my questions about the touch tables and rather, kept circling back to the building itself—its authentic and tangible sense of history. Of course that is what they wanted to talk about and why they were there. This isn’t to say that interpretation of any kind is futile. But I believe it is important to keep reminding ourselves, as museum professionals, that interpretation should be used primarily to help visitors make sense of the objects and artifacts they are there to see—it’s really that simple.

 

The very reason I work with museums is because of my own sense of wonder and astonishment when it comes to objects and artifacts. Yes, I love studying people and how they learn and make sense of experiences, but I could do that in many different settings. I chose to do it in museums because of my own belief that we can learn so much from studying “the real thing.”

25th Anniversary ButterflyFor me, intentionality, a concept I view as essential to museum planning, emerged from two core experiences: results from hundreds of exhibition and program evaluations; and observing museum staff wanting to put too many concepts into an exhibition. Intuitively I knew there was a connection between exhibitions that didn’t fare too well (at least according to the evaluations) and staff not letting go of ideas that are near and dear to their hearts—regardless of whether those ideas supported the thesis of the exhibition.

When I have the good fortune to attend planning meetings, I always find myself thinking critically about what should be included in the exhibition under discussion and what could be saved for another time. My consideration always includes the big idea of the exhibition, what the museum would like to achieve with the exhibition vis-à-vis the public, humans’ capacity to process new ideas when in unfamiliar environments (like that of an exhibition hall), evaluation results from other projects that show what leads to quality visitor experiences and what might move visitors away from having quality experiences, and my utmost respect for scholars’ knowledge and passions. While passionate individuals love their subject matter (and really, I love their subject matter, too), one’s willingness to recognize that not all good ideas (or even great ones) belong in an exhibition and then exercising follow through are traits of intentional practice.

Embedded in intentional practice is the concept of alignment—ensuring that project concepts, components, and elements are present because they support the impact the team wants to achieve. If there are concepts, components, elements that do not contribute to the Intentionalitycore idea of the exhibition and its potential impact on audiences, they need to be omitted. I certainly don’t mean to sound ruthless, but I am acutely aware of how easy it is to keep putting more and more into an exhibition plan and how painfully difficult it is to take anything away. I am also aware of how challenging it is to stay focused on the exhibition’s big idea and have the discipline to say no to ideas because they do not support the intended impact of the exhibition. Learning to say “no” is a necessary survival skill and saying “no” is deeply connected to intentional practice. When practitioners are intentional, they are focused on the impact they want to achieve; they exercise discipline and restraint when determining how to best move forward; and their decision making is egoless and for the sake of achieving the results the team envisions.

Intentional practice represents the culmination of my experiences to date, and my passion for it is directly is tied to my evaluation experience. Over the years I started to realize that when exhibitions tried to do too much, visitors’ experiences didn’t amount to much—from their perspective; their heads were full but descriptions of their experiences were nebulous. Sense-making seemed futile. Nudging me was my memory of exhibition development discussions and tensions about what to put in (everything) and what to take out (nothing). Clarifying the intent of the exhibition and then staying focused on the intent of the exhibition is hard work—not likely to end soon, which is okay. Seeking clarity—whether in thought or action—is a never-ending pursuit.

We have been thinking about intentional practice a lot lately.  The article below, written by Randi, appeared in ASTC Dimensions May/June 2008 issue.  If you would like to read more of Randi’s thoughts on intentional practice, be sure to read her 2007 Curator article, “The Case for Holistic Intentionality.”

At museum conferences these days, people are talking about accountability, public impact, and relevance. These ideas are not new. A decade ago, in a 1997 keynote address for the Mid-Atlantic Association of Museums’ 50th anniversary, the late Smithsonian scholar Stephen Weil spoke of the “in-your-face, bottom-line, hard-nosed questions”—the ones that museums often hope to keep under wraps: “Do museums really matter? Can and do museums make a difference?”In arguing that some museums do make a difference, and that all should strive to do so, Weil supported the notion that “the very things that make a museum good are its intent to make a ‘positive difference in the quality of people’s lives.’” He borrowed this last phrase from the United Way of America, which was then challenging its grantees to document the benefits a given program had made in their lives.

Today, museums face accountability questions from many directions. In response to the Government Performance and Results Act of 1993, U.S. federal agencies began to articulate the kinds of outcomes they expected grantees to document. Private foundations followed suit, reexamining their own evaluation practices, as well as those of grantees. The effort continues. The National Science Foundation recently published its Framework for Evaluating Impacts of Informal Science Education Projects, outlining five categories of impact it expects grantees to assess. And for those who object that “you can’t measure mission-centered work,” current United Way CEO Brian Gallagher, as reported in the Wall Street Journal, has a succinct reply: “You most certainly can. The question is, ‘Are you committed to do it?’ And then, ‘Are you committed to report on it?’”

As museums begin to grapple with their intent to make a positive difference, they can start by reexamining their museum’s mission. Weil believed, as many do still, that a mission is key to an institution’s success. A museum’s mission should be a declaration of its core purpose—clarifying what the museum values, reflecting what the museum embodies, and describing its intent to affect its public and community. Establishing a clear institutional purpose, Weil believed, is the first step to being able to assess effectiveness in achieving public impact.

From my own experience as an evaluator, I would add this observation: Museums do not, in and of themselves, value, reflect, or intend. People do.

An institution’s mission will not be within reach unless everyone who works in that institution is mission-focused and mission-driven. Before museums can assess their impact, staff must collectively clarify their intent. Public impact, relevance, and value grow from what I have called “intentional practice”—the willingness of everyone in the museum to examine all operational activities through three mission-based filters: clarity of intent, alignment of practice and resources, and reflective inquiry.

  • Clarity of intent. Opportunities for all staff to come together to discuss the core values of their museum are vital. Colleagues should both encourage others to explore their passions and also challenge others’ thinking as a way of clarifying what is truly of importance.  In the spirit of thoughtful inquiry, why not ask a colleague to defend his or her position? Most people appreciate being asked to explain why they think the way they do. This kind of exploration allows practitioners to voice the passion behind their ideas and learn what they, as a group, really care about. Reexamining the essence of the museum together can reinvigorate the collaborative spirit, enabling staff to further their practice with intent.
  • Alignment of practices and resources. Unless the work of the museum is aligned with its intent, staff may spend time and resources on activities that are good in themselves but may not support the museum’s intent. Perhaps staff should determine—through evaluation—which programs yield the highest impact, keep those programs, and either improve or discontinue those that do not deliver impact. Aligning practice—the activities a museum does and how it does them—and resources so they support the museum’s intent requires thinking about what you should be doing and what you need not do any more. Conversations about realignment will deepen staff members’ understanding of the museum’s intent and the ways in which their work supports it.
  • Reflective inquiry. As an evaluator, I frequently see front-end and formative evaluation being used effectively to shape a final visitor experience. The same cannot be said of summative evaluation. By the time a mandated final report is done, practitioners may have little time or motivation to review it. This is unfortunate because much can be learned through reflecting on past work.

I see a strong relationship between taking the time to think about the work you have done and learning from the work you have done. Practitioners who want to be intentional in their practice can use summative evaluation as a way to gain insight and knowledge about visitors’ perspectives and experiences. The outcome of such reflective inquiry is learning about the ways in which their museum is achieving impact. I would encourage all museums to routinely set aside time for staff to use inquiry as a reflection strategy and to discuss their practice in the context of the institution’s intent.

In conclusion, accountability questions are not likely to disappear, but even if they did, museum practitioners would still need to respond to the “To what end?” question. The sustainable health of the museum depends on it.

Most of the workers I encounter in museums are passionate about their work and want to make a positive difference in people’s lives. If practitioners begin collaborating with colleagues to clarify their museum’s intent, realign their practices and resources to support that intent, and engage in reflective inquiry to learn how they can improve their efforts, they will be on their way to achieving that goal.

25th Anniversary ButterflyI’ve always loved solving puzzles. And to me, people are the most fascinating puzzle of all. Perhaps that’s why I studied two people-focused topics as an undergraduate- biological anthropology and history. Not only was I curious to learn about how our evolutionary past has shaped human behavior, but I also wanted to understand how our cultural experiences affect those behaviors.

To this point, one of the great things about museums is that they bring people together to share ideas and learn from one another. I’ve always thought of museums as some of the most innovative and fun places to spend my time. They’re also fascinating places to explore people. Where else can you encounter a blend of different age groups, cultural histories, educational backgrounds, learning styles, hands-on experiences, fascinating artifacts and ideas?

I like to think of evaluation as having a continuous set of “people puzzles” to solve. How do people behave in and use museums? What motivates their behaviors? What does learning look like in a museum? How can a museum craft meaningful experiences for the people that walk through its doors? What do those experiences look like?

The solutions to these puzzles are plentiful and ever-changing and that’s what I love about them. As someone who’s relatively new to the field, I’m excited to have the chance to observe and interpret peoples’ often mysterious behaviors to help museums continually create experiences that are truly responsive to their visitors’ needs.

But perhaps most importantly, as a new evaluator, I’m excited by the opportunity to help those that work in museums learn to continually ask this ever-evolving set of questions themselves. Because underlying our field of people watching and data analyzing is the idea that it’s perfectly alright to not have the solutions, as long as you’re willing to ask questions in order to uncover them and learn from mistakes along the way.

So, although as an evaluator I may never be able to definitively solve these people puzzles, I can help museums and their visitors understand and relate to each other in new and unanticipated ways.

25th Anniversary ButterflyOne of the things I love about my job is the part where I need to find order in chaos, such as the trends in a large sample of individual in-depth interviews, and refining our processes for collecting and analyzing data in order to find that order more efficiently. I’d say that I have always had this tendency towards order and process. As a little girl, I could spend hours lining up figurines in a single line and moving them one by one like they are marching in a parade. I have learned immensely from and with my colleagues at RK&A particularly in regard to process. And while I can think of several examples about how we have refined process, my favorite example is Johanna Jones’s work with timing and tracking. Johanna was with RK&A for 13 years and over that span of time she honed RK&A’s timing and tracking processes to a truly distinct approach, learning a great deal from others like Beverly Serrell, who Randi considers the queen of timing and tracking.

photo

Timing and tracking was something that hooked me into the museum evaluation field. I remember being completely fascinated by Arthur Melton’s timing and tracking studies when I learned about them in grad school. As I was formulating my thesis research topic, I knew that I wanted to incorporate timing and tracking just to have the opportunity to try it. For my thesis research, I timed and tracked art museum visitors to determine whether security guards affected visitors’ movement in the gallery. I had meticulously drawn maps full of dots and lines to track a visitor’s and security guard’s behaviors, plus I wrote open-ended notes about what the security guard was doing in the space. The data was wholly exhausting to collect and time consuming to analyze, but I felt like it gave me a handle for what timing and tracking was. . .

 

. . . Then, Johanna shared with me a timing and tracking form for one of my earliest projects at RK&A. I was completely blown away by it. It was neat, clean, concise, and focused. Every component in the exhibition was clearly identified with columns to indicate when a visitor started and stopped using it. There was also a short list of behaviors that we looked for at each exhibit. It was perfect! Elegant even. I still marvel at how refined the timing and tracking process became and strive to apply that same clarity and creativity to thinking about the various other processes that make up the work we do.

Follow

Get every new post delivered to your Inbox.

Join 126 other followers