25th Anniversary ButterflyI’ve always loved solving puzzles. And to me, people are the most fascinating puzzle of all. Perhaps that’s why I studied two people-focused topics as an undergraduate- biological anthropology and history. Not only was I curious to learn about how our evolutionary past has shaped human behavior, but I also wanted to understand how our cultural experiences affect those behaviors.

To this point, one of the great things about museums is that they bring people together to share ideas and learn from one another. I’ve always thought of museums as some of the most innovative and fun places to spend my time. They’re also fascinating places to explore people. Where else can you encounter a blend of different age groups, cultural histories, educational backgrounds, learning styles, hands-on experiences, fascinating artifacts and ideas?

I like to think of evaluation as having a continuous set of “people puzzles” to solve. How do people behave in and use museums? What motivates their behaviors? What does learning look like in a museum? How can a museum craft meaningful experiences for the people that walk through its doors? What do those experiences look like?

The solutions to these puzzles are plentiful and ever-changing and that’s what I love about them. As someone who’s relatively new to the field, I’m excited to have the chance to observe and interpret peoples’ often mysterious behaviors to help museums continually create experiences that are truly responsive to their visitors’ needs.

But perhaps most importantly, as a new evaluator, I’m excited by the opportunity to help those that work in museums learn to continually ask this ever-evolving set of questions themselves. Because underlying our field of people watching and data analyzing is the idea that it’s perfectly alright to not have the solutions, as long as you’re willing to ask questions in order to uncover them and learn from mistakes along the way.

So, although as an evaluator I may never be able to definitively solve these people puzzles, I can help museums and their visitors understand and relate to each other in new and unanticipated ways.

25th Anniversary ButterflyOne of the things I love about my job is the part where I need to find order in chaos, such as the trends in a large sample of individual in-depth interviews, and refining our processes for collecting and analyzing data in order to find that order more efficiently. I’d say that I have always had this tendency towards order and process. As a little girl, I could spend hours lining up figurines in a single line and moving them one by one like they are marching in a parade. I have learned immensely from and with my colleagues at RK&A particularly in regard to process. And while I can think of several examples about how we have refined process, my favorite example is Johanna Jones’s work with timing and tracking. Johanna was with RK&A for 13 years and over that span of time she honed RK&A’s timing and tracking processes to a truly distinct approach, learning a great deal from others like Beverly Serrell, who Randi considers the queen of timing and tracking.


Timing and tracking was something that hooked me into the museum evaluation field. I remember being completely fascinated by Arthur Melton’s timing and tracking studies when I learned about them in grad school. As I was formulating my thesis research topic, I knew that I wanted to incorporate timing and tracking just to have the opportunity to try it. For my thesis research, I timed and tracked art museum visitors to determine whether security guards affected visitors’ movement in the gallery. I had meticulously drawn maps full of dots and lines to track a visitor’s and security guard’s behaviors, plus I wrote open-ended notes about what the security guard was doing in the space. The data was wholly exhausting to collect and time consuming to analyze, but I felt like it gave me a handle for what timing and tracking was. . .


. . . Then, Johanna shared with me a timing and tracking form for one of my earliest projects at RK&A. I was completely blown away by it. It was neat, clean, concise, and focused. Every component in the exhibition was clearly identified with columns to indicate when a visitor started and stopped using it. There was also a short list of behaviors that we looked for at each exhibit. It was perfect! Elegant even. I still marvel at how refined the timing and tracking process became and strive to apply that same clarity and creativity to thinking about the various other processes that make up the work we do.

25th Anniversary ButterflyYou can’t escape technology in museums. Visitors use smartphones to take pictures. Exhibits use touch screens and high-tech interactives to share stories and information. Programs use technology to help visitors engage. Everywhere you look there is a screen . . . until you encounter an evaluator armed with a clipboard and a pencil. I don’t think this is because evaluators are luddites. I think this is because, much like exhibit and program designers, we want our use of technology to make sense. Technology has to make data collection easier not only for us, but for the visitor, too.

I have been using technology for data collection since I was in grad school. We had grant money to spend, so iPads were purchased and students were encouraged to try new things. We approached the task with gusto, certain that this would make things easier; after all, if we enter visitor data as we collect it, we will have eliminated the need to enter data later! It sounded like the perfect plan, until we realized we were collecting data outside—in the elements. In Seattle. In November. We learned a few lessons that day: iPads don’t do well in the cold and neither do the cold, bare fingers we needed for the touch screens. Perhaps in this case, using iPads wasn’t the best plan, given our data collection environment.

Since joining RK&A two years ago, we have experimented with technology as well. Each time we elect to use technology, we think about how it will affect the project. In some cases, it is simple – if we take interview notes on a laptop or tablet while talking to visitors, we can record more of what the visitor is saying and eliminate some work on the back end. This is a low-risk decision that we frequently make. In other cases, we rely much more heavily on technology by using tablets to collect survey data at museums. This is a higher-risk decision because while there are many positive aspects to non-paper data collection, there are also challenges.

Tablet data collection requires me to think about survey presentation in a different way. The survey is designed mostlytechnology for the data collector since we often administer surveys verbally, so it has to be easy to manipulate. Some question formats, such as the scales that RK&A often creates, which are non-traditional scales, don’t always translate well from paper to digital. I have to think about how the question is presented for the data collector, and what, if any, information they have to present to the visitor (e.g., a visual representation of the scale), then find a way to balance the two. Also, we ask the visitor to enter their own demographic information, formerly a single page of questions. But when collecting data on a tablet, we need to balance how much the visitor has to scroll with the number of pages they have to click through. This can be tricky when skip logic directs visitors to the appropriate questions. Regardless of the demographic information the visitor inputs, the survey has to be easy for them to complete. Each time, I learn something I need to change in future surveys to make it easier for visitors.

There are huge advances taking place in digital data collection as new software and platforms are created, and we researchers and evaluators develop best practices for their use. For me, every project that uses technology to help with data collection teaches me something new and makes me a better practitioner. What experiences have you had and what have you learned? The field is changing and I can’t wait to see what we think of next.

In March, Intentional Museum announced it’s first blog competition, asking students to reflect on the following question: Through your intentional practice, how do you help enrich the lives of others?  Below you will find the winning post from Faithe Miller McCreery.  Faithe is a graduate student at the University of Washington in Seattle where she is a member of the Museology class of 2015. 

faithe_collections2I’m a collections girl. At this point in my career, most of my professional and volunteer experience has centered on digitizing, cataloging, rehousing, and otherwise maintaining artifacts and archives. This nearly always involves entering the museum through a back door, taking a staff-only stairwell down to the basement, and cozying up to a collection for some quality one-on-one time.

Part of me loves this. I appreciate having a private little space where I can hunker down and concentrate on projects with few distractions. But getting into the collections basement routine can also be very isolating. At times I find myself developing a sort of artifact-based myopia: I am very attentive to the work at hand, but I begin to forget why I’ve chosen to work with collections in the first place. When I’m alone in a lab with a roll of film negatives, I have to remind myself that I’m not in this career for the negatives themselves: I chose museums because I want to inspire, educate, and empower people.

As an intern for the Klondike Gold Rush National Historical Park – Seattle Unit, my responsibilities include occasionally manning the Visitor Services desk at the front of the museum. While I’m there, I greet incoming visitors, direct them to specific exhibits, and just generally chat about any number of subjects that may arise. Technically, staffing the front desk does not fall under the auspices of either the Interpretation or Curatorial departments toward which my internship is tailored. My time there arose purely to fulfill a staffing gap at the museum, and it was a task that I initially dreaded as taking attention away from my “actual” duties. The truth is, though, interacting directly with visitors has become something of a respite from the quiet and isolated collections environment where I spend the majority of my time. Visitor Services has become a part of my work that I quite look forward to.

When I see the unbridled enthusiasm of children who just can’t wait to explore local history, listen to older people share stories of their own relatives’ experiences panning for gold, or welcome a group of tourists into their first cultural institution in Seattle, I develop a far greater appreciation for the work that I do with collections. Interacting with visitors serves as a much-needed reminder for me that ultimately, collections lose their purpose if they become unbridled from the institutional mission. The work that I do with collections occurs not in spite of visitors, but for them –and having an eye on the front desk helps me understand the needs, questions, and concerns that I should be addressing while I toil away in Curatorial. It may seem paradoxical, but I’m so glad to have learned that the more time I spend outside of the basement, the more I appreciate the significance of the work that I do while I’m there. Great things happen in the basement –but let’s not discount the ground level, too.


Effective and cle25th Anniversary Butterflyar communication is a skill that all evaluators must master but sometimes those of us in the evaluation field forget that we may be speaking a foreign language to our clients. We become comfortable with acronyms like IRB or throw around names for data collection methods like surveys, focus groups, naturalistic observations, and ethnography to name a few—not realizing that those words can have different meanings to different people. Precision of language is vitally important, and it goes hand-in-hand with another skill that, over time, I have come to respect—that of active listening. As evaluators, we need to listen to ourselves, visitors, our colleagues, and our clients. To me, active listening is listening first to understand and then responding, and I have found it difficult to do especially when I feel that I have something important to say. We’ve all been there I think; those moments when you start to tune out someone because you are searching for the right opening to say what’s been on your mind for the last several minutes. It’s fairly easy to spot when someone is not actively listening because their comments result in non sequiturs.

One of the challenges with active listening is that it is mentally exhausting. It’s much easier to let your brain relax at regular intelistening2rvals rather than to be constantly aware of what every other person is saying. When conducting in-depth interviews, asking visitors open-ended questions about an exhibition or program, which are intended to result in visitor-centered conversations, we are “on” the entire time and do very little talking ourselves. We are very careful to train our interviewers to actively listen to visitors’ responses so they can discern whether visitors are responding to the questions or whether their responses require additional questions; simultaneously they also have to make sure they understand what the visitor means by the words he or she uses—which is the essence of understanding. At the end of a day of interviewing, I warn my data collectors that they will feel mentally exhausted, because I have experienced this kind of fatigue so many times before. This one example points to a key tenet of active listening—you, as the listener, really do not say much at all; rather your primary job is to listen to understand and ask questions to seek clarity if you don’t understand.

With our clients, active listening is important, too. It’s one of the many things I’ve learned to do while working at RK&A. I learned to actively listen first when I was an interviewer (which I still like to do when the project calls for it because it is a great reminder of the importance of active listening). Then, I learned the key role active listening plays in client meetings and intentional planning workshops we facilitate. Most of what we do as evaluators is ask questions, listen, and then ask more questions to seek further clarity. The mental exhaustion I feel at the end of the day is a good sign that I have done my job as an active listener. I’d be concerned if that feeling were to ever disappear.

25th Anniversary ButterflyWe are excited to introduce a guest blogger this week: Johanna Jones, former Managing Director of RK&A’s San Francisco office.  Having been with RK&A for 14 years, Johanna contributed greatly to the company’s learning, and thus, we are happy she agreed to reflect as part our 25 years of learning series.


As I think back on my work at RK&A, like Stephanie, I am struck by an unintended outcome of the evaluation process; namely, that asking visitors questions about their experience can serve an important interpretive role in museums and for visitors. We don’t often think about the direct value of evaluation on visitors—rather we focus on evaluation as an essential step in the institutional cycle of learning (see reflection 2 for the cycle of learning) and in creating an intentional organization which, of course, benefits visitors. I would argue that the evaluation process itself—the very act of interviewing visitors—serves a powerful interpretative function. When visitors are asked open-ended questions about their visit, they are afforded the time and space to reflect on their experiences. The questions become a framework for thinking about their visit—beyond what they liked and didn’t like—that prompts them to consider “What does this mean to me?”openended question

I didn’t always see the direct value of evaluation for visitors. When I first started building my evaluation skills, the educator in me worried about imposing on visitors’ time by asking questions. I had some rocky starts—people have been so saturated with market research that they are wary of someone approaching them for feedback. But I quickly learned that if you ask visitors meaningful open-ended questions, you are usually met with meaningful responses. I remember a Vietnam veteran who cried when telling me what the American flag meant to him and the young children who were pumped to save the Condors (not exactly warm and cuddly creatures). I recall avid art museum-goers who were amazed to realize that they could interpret works of art for themselves and Twenty-somethings who expressed civic pride in a once beleaguered natural history museum. The more I talked to visitors the more I realized that the evaluations were not only fulfilling the institutions’ need to understand visitors but also visitors’ need to process and make meaning from their museum experience.

Now, as a museum visitor, I still use meaty open-ended questions. I ask them of myself and my family members when we visit museums. And, I must say, I find our conversations richer for it.

seattle_kerry_parkWe are gearing up for an adventure in the Pacific Northwest – several members of the RK&A staff will be in Seattle later this month for the AAM annual conference and we are so excited! Be sure to look for us at the conference and check out our panels on Sunday and Monday:


Empathy Doesn’t Count: Measuring What Matters in Exhibitions

Sunday, May 18, 2-3:15pm

This session will provide examples from a number of different institutions that have endeavored to evaluate (i.e., measure) the seemingly unmeasurable aspects of exhibition design and interpretation.  Panelists will share methods and engage participants in thinking creatively about evaluation.

Stephanie Downey, Randi Korn & Associates; Jason Porter, Skirball Cultural Center; Lynne Azarchi, Kidsbridge Tolerance Museum; Sarah Cole, Carnegie Museum of Natural History


Stop, Collaborate and Listen: Making the Most of Evaluation

Monday, May 19, 8:45-10am

Ever use an expensive evaluation report to prop up a table leg?  Join panelists who have created and consumed museum evaluations for a fast-paced, jargon-free talk about getting the most out of evaluation – whether you’re an expert or a novice.  Then bring a sample project and try out what you’ve learned!

Renae Youngs, Minnesota State Arts Board; Emily Craig, Randi Korn & Associates; Betsy O’Brien, St. Louis Science Center


We will also be attending the CARE/PRAM Luncheon on Tuesday, May 20 at the Sheraton Seattle.  We look forward to seeing many of you at the conference and would love to catch up in Seattle!


Get every new post delivered to your Inbox.

Join 119 other followers