Archive for June, 2015

As RK&A’s first Research Fellow and a doctoral candidate in sociology at Northwestern University, I’m delighted to write the first of my Intentional Museum posts exploring the relationship of sociological research and museum evaluation. As it turns out, the timing of this writing is pretty fortuitous: I’m currently preparing a presentation for this year’s Leadership Exchange in Arts and Disability (LEAD) Conference at the Kennedy Center, which has me thinking about how sociological research on museums can benefit practitioners. So I thought I’d start, as I’ll start in that presentation, with the big picture.

First things first: What do sociologists have to say about museums? In general, our research has spoken to three distinct themes. We’ve shown how visitor demographics in art museums reflect broader systems of social inequality by explaining how people’s education and class background shapes their familiarity with art. This helps explain patterns of visitor attendance, while also identifying the societal barriers that may leave some people out of the conversations museums try to foster.  We’ve shown how broader social changes (i.e. funding structures or political conditions) can impact what happens inside museums. This work illustrates how the environments in which organizations operate define what counts as legitimate operations, which in turn influences what museums do. Most recently, we’ve lifted the hood to look inside museums and focus on practice: what people do within these organizations, and how. Because museums offer a particularly apposite case for examining how people interact with objects, some of this work has examined how different objects and environments can structure interpretation and shape organizational goals.

Recently, I’ve grown increasingly curious about why so little has been said about the intersections of sociology and evaluation. In part this is because throughout my doctoral fieldwork on museum education, people regularly confused sociological research with evaluative practice. It was easy at the time to point out differences. Perhaps the most foundational one regards the role of theory. Sociologists study specific things to tell a more general story about the social world, entering museums to answer a theoretically-motivated question (for example, about inequality, legitimacy, or practice) that can speak beyond a single museum, exhibit, or program. In contrast, evaluators concentrate on how to help particular museums articulate their practical objectives (for exhibition development, for program assessment, and so forth), and then develop research designs to assess them. They may even make formal recommendations, which is not typically within the purview of sociology.

However, when reflecting – as I often do, and as I’ve been doing for LEAD – on what people in museums can do with broader sociological ideas, I inevitably find myself asking how, if at all, sociology and evaluation are akin in helping museum practitioners. For one, the best work in sociology and in evaluation rests on carefully prescribed methods. A sociologist’s ability to specify the method by which he or she arrives at a particular theory is what ensures it is sound. Evaluators, similarly, help museum practitioners identify their guiding questions at the outset of a project (just like sociologists must do for themselves) and select the methods (surveys, focus groups, observations, interviews) that will guide that client to results that are both reliable and useful, an important aim our Twitter chat explored on June 9th. In this way, both the sociologist and the evaluator practice with intention.

Perhaps most importantly, both professions can also aid museum practitioners in becoming more intentional. In assuaging museum educators that I was not doing program evaluation, I often explained that my research could instead give practitioners the language to talk about and reflect upon how they interpret the broader social conditions and institutional environments in which museums operate. Or, paraphrasing Max Weber – one of sociology’s founders – scientists can through their work promote clarity about choices by showing people the results of their actions. But now it seems false to me to distinguish this guiding philosophy too starkly from the aims of evaluation. High quality evaluations present museum staff with systematic research in efforts to help them make more informed choices about what they do, how they do it, and with what impact. This objective, I’ve come to understand, is central to how RK&A understands the role of evaluation in museum settings, and what we here call “intentional” practice.

Read Full Post »

Last week, Stephanie wrote a thoughtful piece about the recent upswing in museum professionals who are conducting evaluation and the importance of thinking critically about evaluation. Among other things, she asked “how can the field be sure the results produced are reliable and useful?” You can read the full post here. In an effort to open up discussion and conversation on thinking critically about evaluation, we’re excited to announce RK&A’s first Twitter chat. We hope you will join us!

Join the Conversation

From 2-3pm EDT on Tuesday, June 9th, RK&A’s Stephanie Downey, Amanda Krantz, and Cathy Sigmond will host RK&A’s first Twitter chat on thinking critically about evaluation, using the hashtag #RKAchat.  To join the conversation make sure your tweets include this hashtag. 

Why this topic?

Many museum professionals are enthusiastic advocates for evaluation and view it as essential to their work.  As evaluators, we’re ecstatic about this! But for an evaluation to be truly useful, museum professionals need to think evaluatively about evaluation.  In other words, museum professionals must think critically about how evaluations are planned and conducted to fully make sense of evaluation results within the context of the reliability and validity of the study.

To that end, we’re hosting a discussion on thinking critically about evaluation.  We’ll pose some questions so we can hear your thoughts and experiences, and we will share our thoughts on how museum professionals can position themselves to be critical consumers of evaluation.

Twitter Chat Questions

During the Twitter chat, @IntentionalMuse will tweet numbered questions (for example, “Q1: What do you evaluate, and how? #RKAchat”).  Your response tweet should reference the question (for example, “A1: We talk to visitors about what they took away from an exhibition to evaluate the exhibition’s learning goals #RKAchat”).

Q1: What do you evaluate, and how?

Q2: What do you think characterizes a “good” quality evaluation?

Q3: What characterizes a “bad” quality evaluation?

Q4: What are the challenges in conducting high quality evaluation that will provide meaningful results?

Q5: With quality in mind, what is one way you might think about or approach evaluation differently in the future?

How to Participate

If you do not already have one, create a Twitter account.  On Tuesday, June 9th from 2-3pm EDT, tweet using the hashtag #RKAchat.  You can monitor the tweets related to the chat by searching for #RKAchat on Twitter.

Read Full Post »