Posts Tagged ‘engagement’

My life as a museum evaluator preceded my life as a parent, and over the past 15 plus years, I have learned a great deal about child development, family learning, and parent-child interactions in informal learning settings.  I know what I know from academia (Piaget and Vygotsky), from classic studies on family learning, and from hours and hours spent pouring over data collected from observing and interviewing parents and children in all kinds of museums.  I have even written an article about parent-child interactions, published in Museums and Social Issues, based on research we did at the Please Touch Museum in Philadelphia.  I know that many museum professionals would describe ideal parent-child interaction as parents guiding and scaffolding their child through a museum experience that is primarily child-directed.  The image of a parent crouched down at a child’s level, together actively engaged in observing, asking questions, and speculating on the meaning of a particular experience or object comes to mind.  So, as you can imagine I entered motherhood with all kind of fantasies about what it would be like to take my own children to museums.  While pregnant I imagined me and my unborn boy/girl twins (I’ll call them F and E) at The Met looking closely at and wondering about hieroglyphs in an Egyptian tomb or at the American Museum of Natural History hypothesizing about what we could see in all those amazing dioramas.  I can happily say that almost nine years later, I have had both of those experiences with F and E, but for many years, my experience with them in museums was anything but ideal.

Instead of the idyllic scene I describe above, for many years, taking F and E to museums was actually kind of tortuous.  As the researcher pre-motherhood, I knew nothing of the reality of dirty diapers, snack containers, winter coats and hats, tantrums, or corralling two toddlers with sticky hands though a crowded gallery.  Let’s just say in those early years, had “researcher me” observed “mommy me” in a museum with F and E, “researcher me” would have been horrified.  “Researcher me” would have seen “mommy me” slumped over on a bench, covered in dried cheerios, texting, yawning, or maybe having a conversation with another mom, while F and E bounced around like pin-balls from exhibit to exhibit or laid on the floor having a melt-down because they wanted to go to the gift shop or get candy out of the vending machine. That’s not to say that it was all bad in those early years—there were moments of wonder in the Brooklyn Children’s Museum, the Newark Museum, the New York Hall of Science, and others. But those moments of wonder were outnumbered by moments of chaos.  So, when F and E were about four years old, I threw in the towel and pretty much stopped taking them to museums.

E shooting a cannon aboard the USS Constellation in BaltimoreThen, I’m not sure what happened, maybe it was because they doubled in age, but last March I took F and E on an Amtrak trip to the Mid-Atlantic States and decided to take them to a few museums.  And, to my utter amazement, we had a blast.  Over the course of three days, we went to the Constitution Center, the National Aquarium, the Sports Legends Museum, and took a self-guided tour of the USS Constellation (a naval fighting ship built at the end of the 18th Century).  I let F and E lead the way in every case and was there to gently guide them in observing and experiencing everything.  They looked at everything, asking a ton of questions and speculating on what it might have been like to sleep in the bowels of a wooden ship, to play on an NFL football team, F in the huddle at the Sports Legend Museum in Baltimoreand what the heck kind of fish is that?  Since our train trip, we have visited many other museums in the New York City area (and they visit a lot of museums on school field trips, too).  In each case, I have been amazed and so proud to realize that we are experiencing museums in exactly the way I imagined all those years ago.  It took awhile, but we finally got there (although, I have fully accepted that vending machines and gift shops still rule…).

Read Full Post »

One of the amazing benefits of working as an evaluator with a variety of institutions is the opportunity for personal learning.  Having an art-history background, I find myself learning the most when I’m placed in non-art environments—reading about fault lines and earthquakes at the California Academy of Science, or getting my hands dirty while exploring decomposers at the New York Botanical Garden.  Granted, as an evaluator, my job is to understand how visitors experience these exhibits and programs, but as a museum-loving individual, I can’t help but want to engage with the content myself.

In the past month, some of my RK&A colleagues and I have had the pleasure of evaluating exhibits and programs about a range of topics, including tropical animal and plant life, of which I, as an Ohio native, have great appreciation for and very little personal knowledge.  Recently we conducted a formative evaluation at the Miami Science Museum of the aquarium component planned for the new Phillip and Patricia Frost Museum of Science.  The exhibits (a mix of zero edge tankinteractives and live-animal tanks) were designed specifically to encourage visitors to look closely, discuss their observations, ask questions, and explain what their observations might mean.  As I observed from visitors’ experiences, these sorts of exhibits and behaviors prompted visitors to engage as active participants and informal learners, having fun exploring an exhibit while employing scientific skills (sometimes even unknowingly).

It was a few weeks later on a work trip to Puerto Rico when I first realized how much I had unknowingly absorbed from our recent environmentally-focused work and how often I was using these newly found scientific skills.  My colleague Emily Craig had surprised me with a visit to a local beach between our data collection sessions, and as we walked up and down the beach, I noticed that we were doing the same behaviors we had monitored weeks earlier when conducting the formative evaluation at the Miami Science Sea UrchinMuseum.  We pointed out bright green vegetation and abandoned white shells once home to small creatures adhered to the driftwood.  We looked closely at the patterns of snail shells, which reminded me of patterns from blue and white china.  We tested the suction-based strength of a sea urchin (one of the live animals we had learned about at the Miami Science Museum) when we attempted to carefully move it to safer grounds. Octopus We speculated about the type of rocks that made up the shore based on the way the rocks seemed to cement fossils and sea glass in their cracks.  We observed a baby octopus that had been washed ashore before scooping it up and returning it to sea.  We made claims about the small, squishy spheres we found on the shoreline, hypothesizing that they were eggs and guessing which creatures had laid them.  In short, we were bringing our museum-honed scientific skills and sense of investigative science to the Puerto Rican shoreline.

Emily and Ros at the beachBeing on the shore for that moment gave me an even greater appreciation of the work that museums and cultural institutions do and their importance in our lives.  Though I often visit institutions wearing my evaluator hat, focusing on other people’s experiences rather than my own, the information and knowledge institutions offer still seep into my subconscious interactions with the world—prompting me to wonder just how much we take away from our museum experiences that we may never even recognize.

Read Full Post »

This week, I’d like to begin to hone in on the idea of measuring impact that Randi raised in our first blog post.  We define impact as the difference museums can make in the quality of people’s lives, and measuring it can be both exciting and intimidating.  Exciting because just about every museum professional I’ve ever met believes museums have the potential to affect people in deeply powerful ways.  Stories abound from people who have distinct and palpable memories of museum visits from childhood—memories that became etched in their being and identity (for many, it is the giant heart at the Franklin Institute, for others it may be a beautiful Monet water lily, and for the nine-year old me, it was a historic house, My Old Kentucky Home).  It’s these kinds of experiences that draw museum professionals to their field.  On the other hand, the idea of measuring impact can be intimidating because some think it is impossible to evaluate, measure, or assess something as intangible as a personal connection, engagement, identity growth, a lasting memory, an aesthetic experience, or an “ah-ha” moment.  When these fears emerge, we try our best to allay them and try to move them towards an important first step in measuring impact—describing what impact looks like or sounds like.  Evaluators are accustomed to figuring out how to measure something—once the impact is described.

The Giant Heart at The Franklin Institute

The Giant Heart at The Franklin Institute

I, as a researcher and evaluator, become excited at the thought of being tasked with measuring impacts, such as “engagement” or “creativity.”  I relish the idea of studying something so I can explain the unexplainable, of drawing meaning from and describing unique human experiences.  As long as I can remember, I have been interested in the complexities of the human experience, especially in how it plays out in specific contexts, within the social realm, and in relationship to material culture, such as art, artifacts, and natural history specimens.  These interests led me to the field of anthropology and to work in social science research and museum evaluation, where I have the pleasure of spending my days exploring the ways people make meaning in museums and other similar institutions.

Sometimes when museums cite their impact, they fall back on the common practice of reporting visitation numbers.  While not unimportant, numbers indicate only that people came—they do not indicate the quality of visitors’ experiences.  Imagine hearing that a museum attracted a million visitors and then hearing about the qualitative difference a museum has made in people’s lives—wouldn’t that sound more meaningful? This brings me to discussing an often overlooked methodology in museum evaluation—case study research.  Its low rate of use is interesting in light of museums’ desire for evidence of impact, as a case study can provide rich details of a person’s or entity’s (e.g., a school) experience.  Case study research is “an in-depth description and analysis of a bounded system”—that “system” could be any number of things: an individual museum visitor, a school partner, or a community.  It provides a focused, in-depth study of one particular person or entity.  Practically speaking, what we do is follow several participants (or “cases”) over time (during and after a program for example), by interviewing them repeatedly, observing them in the program, and interviewing others within their sphere of influence (such as a parent, spouse,  museum professional, student, or community member) who can comment about their experience. The outcome is a concrete, contextualized, nuanced understanding of a particular phenomenon (for example,  a person’s growth over time, a relationship between a museum and school, or a museum’s affect on a community) that can explain not only what happened as a result of the program, but how it happened.   Knowing the “what” and the “how” are invaluable to museums; the “what” can offer indictors of impact and the “how” tells you what the museum might have been doing to create the “case” experience.

An example of case study research comes from an evaluation we did for an art museum that was launching a new multi-visit program in middle schools.  We began our work by helping the museum define its intended impact, which is articulated as: “Students are empowered to think and act creatively in their lives, their learning, and their community.”  We then worked with staff to operationalize the impact statement by developing a series of concrete, measureable outcome statements.  We identified our “cases” as three middle schools.  Each “case” study included a series of interviews with students, classroom teachers, a few parents, and program staff, as well as an observation of program activities in the school and in the museum—all over the course of several months.  The data were rich, specific descriptions of what happened to participants and how the program functioned in each school—all in relationship to the impact statement.  Not surprisingly, each school had a slightly different experience, with one school more closely meeting the impact statement than the others.  The case study approach unveiled the complex interplay of variables at each school to help explain why one school was more successful than the others.  Both the successes and challenges provided great insight as the museum considered its second year of program implementation.

I know what you are thinking.  How can we measure impact by focusing on a few individuals or one or two schools?  What about generalizability?  While these questions are reasonable, they miss the point of case studies, which I believe strongly aligns with what we know about museum experiences—case studies account for differences in people’s unique dispositions, life experiences, and knowledge; they value distinctiveness; and they recognize the complexities of life and situations and do not try to simplify them.  If a museum really has trouble accepting the essential value of what a case study can afford, plenty of museum programs are small enough to warrant conducting a case study without worrying about generalizability.  The example above is a relatively small program serving six schools, three of which we examined through our study.  In another example, we used case study research to assess the impact of a museum-based summer camp serving 20 or so teens.  Conducting case studies to demonstrate the impact of museums’ small programs might be just the perfect baby step towards museums’ beginning to measure impact.

I sense urgency in museums’ need to have evidence of the value of museums in the American landscape.  I think it is time we stopped worrying about what might be immeasurable and instead begin describing what success looks like.

Read Full Post »

A few weeks ago, Randi blogged about the lack of emphasis grantors place on professional learning as a valuable outcome of projects they have funded.  The fear of failure I sense from practitioners when planning an evaluation is often palpable, as practitioners often think about evaluation as a judgment tool and fear the possibility of failure (especially in the eyes of the funder).  The innovation-obsessed culture of the non-profit sector exacerbates the situation: be the best; make a discernible difference in people’s lives; be innovative; don’t make a mistake; and if you do err, certainly don’t tell anyone about it.  Understandably, the possibility of failure creates a stress level that can override people’s professional sensibilities of what is really important.  Yet, I personally feel refreshed when I hear museum practitioners reflect on their failures during a conference presentation; not because I want to see people fail but because mistakes often lead to learning.  And, as an evaluator, it is my job to help museum practitioners wade through evaluation results and reflect on what did not work and why in the spirit of learning.  My job is to help people value and use evaluation as a learning tool.

Failure CartoonI recently had the pleasure of working on a project with the Science Museum of Virginia (SMV) in Richmond.  The Museum, like many others, received funding from the National Oceanic and Atmospheric Administration (NOAA) to develop programming for Science on a Sphere® (SoS).  And, the Museum, like many others, had high hopes of creating a compelling program—one that uses inquiry to engage visitors in the science behind the timely issue of climate change.  Inquiry can be elegant in its simplicity but it is also incredibly difficult to master under even the best of circumstances.  Staff quickly realized that creating and implementing such a program was a challenging endeavor for a whole host of reasons—some of which were unique to the Museum’s particular installation of SoS.  The challenges staff faced are well documented in the evaluation reports they have shared on NOAA’s web site (http://www.oesd.noaa.gov/network/sos_evals.html) as well as informalscience.org (http://informalscience.org/evaluation/show/654).  Yet, the specific challenges are not important; what is important is that they reflected on and grappled with their challenges throughout the project in the spirit of furthering everyone’s professional learning.  They discussed what worked well and addressed elements that did not work as well.  They invited colleagues from a partner institution to reflect on their struggles with them—something we all might find a bit scary and uncomfortable but, for them, proved invaluable.  In the end, they emerged from the process with a clearer idea of what to do next, and they realized how far they had come.

SMV staff recognized that their program may not be unique and that other museums may have done or may be doing something similar.  But each and every time staff members (from any museum) reflect on the lessons learned from a project, their experience is unique because learning always emerges, even if it is subtle and nuanced.  The notion that every museum program has to be innovative, groundbreaking, or unique is an inappropriate standard, and, frankly, unrealistic.  In fact, when museums embrace innovation as a goal, they too, must embrace and feel comfortable with the idea of failure, especially if they want to affect the audiences they serve.  Grantmakers for Effective Organizations share this sentiment (http://www.geofunders.org/geo-priorities) when defining practices that support non-profit success.  The organization states that “[embracing] failure” is one way we will know that “grantmakers have embraced evaluation as a learning and improvement mechanism.”  An ideal first step would be for all of us—institutions, evaluators, and funders—to proudly share our failures and lessons learned with others.

Read Full Post »

RecFLOWently, I and my RK&A colleagues finished a project for the Indianapolis Museum of Art looking at the effects of Mary Miss’ public art installation FLOW: Can You See the River?FLOW was conceptualized around the idea that the White River is underappreciated and even ignored by Indianapolis residents who do not fully understand the importance of the White River to the city (see http://flowcanyouseetheriver.org/ for more information on the project).  I was thrilled to work on such an interesting project, and the evaluation results were mostly positive (see http://informalscience.org/reports/0000/0688/2012_RKA_IMA_FLOW_Summ_dist.pdf for evaluation results).  It is rare for me to have internal struggles with projects we work on, so when I felt a tension with this project, I thought it worthy of exploration.

As part of our work we examined people’s engagement with the installation.  But if I were to honestly answer the question, “Would I engage with the FLOW installation?” (imagining that I lived in the Indianapolis area), my answer would be no.  In fact, when I consider the amount of public art that I have engaged with in the last year, it is fairly limited, and not for lack of exposure.  My realization is confounding because I am what some may consider the expected audience for public art installations.  I have degrees in art history and art education and I am absolutely an “art person.”  I love seeing museum exhibitions and collections of late 19th and early 20th century painting and enjoy attending biennials and triennials to keep up with what new things are happening in the art world.  So why don’t I engage with more public art?  In reflecting on that question, I have tried to think about the few public art pieces that have given me pause in the last year and the incidents surrounding them.  Here are the experiences that stand out to me in order from what I would deem the least profound experience to the most profound experience:

Minneapolis Sculpture Garden.  I stopped here this past spring during an excursion to the Walker Art Center during the AAM conference.  As such, it was a planned trip in a time slot I had already allocated for art viewing.  I was also on a pilgrimage to see Claes Oldenburg & Coosje van Bruggen’s Spoonbridge and Cherry—an iconic image of public art.  The James Turrell that I happened upon was a bonus (thank you sculpture garden brochure for pointing it out!).  While technically my experience was a public art experience, I would say that it was more aligned with a traditional museum experience than a public art experience.  I was on a scheduled trip to a museum in a city I was visiting that allowed me to see several artworks in a single location, including one on my must-see list.

Kat Healey’s Coming Home. My interest in public art in airports is hit or miss, but this work at the Philly airport caught my eye one day, and I have stopped at it multiple times since (of course, never when I am on my way home).  I wouldn’t consider this piece to be something that would naturally capture my attention, but the title “Coming Home” caught my eye.  Something about the poignant contrast of the words “coming home” when I was getting ready to leave home for a trip struck me enough to slow me down.  I can’t remember if I read the identification label first or took in the whole work, which is fairly large and detailed, but either way, the image of the work has stayed with me.  Who knows if I would have paid it any attention had the artwork been titled otherwise (see http://www.phl.org/arts/current/Pages/KayHealy.aspx for more on the installation).

Charles Ray, Boy with a Frog

Charles Ray, Boy with a Frog

Charles Ray’s Boy with a Frog.  I was immediately enamored with this piece when I came across it in Venice.  I had spent three months in Venice almost five years ago and loved coming to the Punta della Dogana for a beautiful view of the city.  I was surprised by this new statue in a familiar place and also struck by how the newness and bright whiteness of the statue was in stark contrast with the city, which is known for its history, lack of change, and beautiful decay.  Unfortunately, I couldn’t find any information on the piece nearby and had to wait to get back to my hotel to Google it.  While I wasn’t particularly taken by the explanation of the work and don’t fully understand its intentions, I still find the piece to be extremely striking and perfectly located (see http://www.nytimes.com/2009/06/05/arts/design/05voge.html?_r=0 for more on the sculpture).

So how do these experiences explain my gut instinct that I probably wouldn’t engage with FLOW?  First, I can’t see myself scheduling a visit to FLOW like I had for the Minneapolis Sculpture Garden since Spoonbridge and Cherry has a special iconic gravitas, which FLOW does not.  And secondly, in my experiences with Coming Home and Boy with a Frog there was some sort of “contrast” that hooked me—in one case a contrast to my feelings and in the other the aesthetic contrast of the work and its setting.  “Contrast,” however, is not a word that I would use to describe a FLOW experience (revealing, informative, about a relevant problem, and subtly surprising are more apt words for FLOW).  Yet, two of the three experiences recounted above (2 ½ if you count my unexpected James Turrell encounter at the Minneapolis Sculpture Garden) were highly serendipitous encounters, and maybe, had I happened upon FLOW I would have had a different response.  Once I looked at FLOW through a rationalized, evaluative lens, I couldn’t turn back.  If reflecting upon these experiences has shown me anything though, it is that you never know what type of public art may catch your eye and offer a bit of unexpected meaningfulness—I suppose that’s the true value and beauty of public art.

Read Full Post »