Update: we are no longer accepting applications for this position. Thank you!

 

Position Opening: Research Assistant

 

Randi Korn & Associates, Inc., a consulting firm specializing in evaluating museum programs and exhibitions, is seeking a Research Assistant in its Alexandria, VA office.

 

Primary responsibilities:

  • Data Collection. The Research Assistant will assist with data collection.  This will likely include observations and interviews as part of formative evaluation (e.g, testing prototypes), focused observations of programs or exhibition components, and telephone interviews.  It will also include collecting interviews, surveys, timing and tracking observations—it will be important for the Research Assistant to have a deep understanding of these methods in order to train and manage data collection teams.
  • Hiring and managing data collection teams. We work with a national client base and regularly hire data collectors local to our client’s institution to conduct interviews, observations, and surveys.  Management may include onsite training of data collectors, scheduling data collectors, monitoring quotas, monitoring the quality of the data, and problem solving data collection challenges that arise.
  • Processing data. We work with a variety of data including questionnaires, timing and tracking observations, and interviews.  The Research Assistant will be the primary point person for processing the data, which may include uploading audio files to transcriptionists, preparing files for quantitative data entry (programming an online survey or creating an excel or SPSS file), entering quantitative data or contracting a data entry assistant, and organizing raw data in preparation for analysis.
  • Finalizing reports and other deliverables. RK&A produce numerous reports, proposals, and presentations each month.  The Research Assistant will help in preparing these reports for distribution by proofreading, formatting, and assisting with designing data visualizations.
  • Updating the RK&A website. The Research Assistant will help maintain the website by posting news to the site’s homepage, updating project lists and case studies, among other things.  This includes helping to update our archives.
  • Light analysis. Most analysis and reporting will be done by Associates, but the Assistant may be asked to conduct some analysis, such as coding survey responses, rubric scoring, or analyzing small samples of qualitative data.  Analysis may be something that the Assistant does more of over time given the Assistant’s capabilities and project needs.

The ideal candidate will be a recent graduate from a master’s program in social sciences, education, museum studies, or a related field. The Assistant will be working with multiple RK&A Associates so must be able to communicate effectively within the team about their workload and prioritize based on needs. The qualified candidate must be detail-oriented and able to juggle multiple projects. A passion for museums or other kinds of informal learning environments is preferred.

 

Application Instructions:

Please submit your application via email to jobs@randikorn.com. Your cover letter should be the body of the email and your PDF resume included as an attachment.

 

RK&A offers a competitive compensation and benefits package. For information on RK&A, please visit randikorn.com.

Last week the interdisciplinary journal Museum & Society (M&S) released their latest issue, entitled “Sociology and Museums,” to which I’m proud to have contributed. As a doctoral student in sociology, my research looks within the organizational field of museums – comparing art museums and botanical gardens – to explore what sociologists gain by investigating the “guts” of museum practice. This is because – in agreement with the M&S editors – I believe that through sociological research on museums we might “expect to see sociology adding something new not only to our knowledge of museums, but also more ambitiously, to our understanding of human society as a whole.”

To date, I’ve explored how museums mediate people’s sensory experience. Museums are an apposite case for exploring sensory experiences because they are organized principally around objects, and people perceive objects through their senses: our experiences of them are not reducible to text. The article I published with M&S, for example, shows if you compare art museums and botanical gardens, you can see differences in what I call “sensory conventions:” the rules that shape how we come to use our senses – and which senses we use – in particular settings. One familiar example of sensory conventions regards how we act in a coffee shop versus a library. In either place, you can work on your laptop, but you know you can’t get away with yammering on your phone in the library. The convention is to be quiet. When it comes to museums, the sensory conventions are similarly well-defined. We look, but we don’t touch.

Or at least, that’s the case with art museums. In botanical gardens, as I show, things are a little more complicated. Plants invite certain kinds of sensory interactions (they smell; they rustle in the breeze) that artworks typically don’t. Further, we value art and natural objects differently, and that impacts whether or not we are permitted to touch them. Compared to more traditional (art, history, natural history) museums, the sensory conventions in botanical gardens are not as clear. Visitor confusion persists even as garden staff try to promote a primarily “hands-off” experience, most often by distinguishing botanical garden from parks.

Plants for the Senses

My article focuses not only on how sensory conventions differ by degree (for example, we can touch more in the gardens, compared to the galleries) but also on how they differ by type. Specifically, in botanical gardens, I find people describe aesthetic experience as being organized around how “things” look: the pleasing, unmediated beauty of natural objects and environments. In art museums, in contrast, people are more likely to say aesthetic experience is about how “to” look. It’s about interpretive observation that can further a person’s appreciation or understanding of an artwork. “Aesthetics” thus means different things across museums but this is not simply because the objects are different, it is also because museum staff choose to organize and interpret objects in particular ways to structure perception. I find these differences in aesthetic understandings extend to the multi-sensory museum experiences staff innovate for visitors with disabilities. While museum staff in the gardens facilitate programs that include plants with interesting textures and pleasing scents, those in the galleries tend to emphasize the senses’ ability to further interpretation. For instance, opportunities to touch provide information on an artwork’s weight and temperature: information that is not necessarily visually discernible.

How does all of this inform our understanding of “human society as a whole”? For one, as museums innovate their practices to be more engaging and accessible to diverse audiences, studying sensory conventions can tell us something about how organizational change happens. While external conditions no doubt shape what museums do, the local meanings and material cultures of museums also influence how these institutions differently evolve the “look, don’t touch” rule into more hands-on experiences. Further, looking at the museum-going experiences of visitors with disabilities reveals the assumptions embedded in sensory conventions. Such conventions shape the kinds of perceptual experiences that are possible in a given space – including in museums – and shows how such opportunities vary across the forms of bodily difference we call disability.

C. Wright Mills famously described the sociological imagination as “the vivid awareness of the relationship between personal experience and the wider society.” Sociologists contend such awareness can promote more informed choices and deepen understanding of their effects. Accordingly, the 13 articles in M&S’s “Sociology and Museums” issue aim to foster readers’ sociological imagination of museums while also encouraging a more intentional approach to museum practice. I hope you check it out!

RKA Blog_Evaluation Design Series_LogoSampling is a very important consideration for all types of data collection.  For audience research and summative evaluations in particular, it is important that the sample from which data is collected represents the actual population.  That is, the visitors who participate in a questionnaire or interview should match the entire population of visitors.  For instance, if the population of program visitors are 75% female, the sample should include approximately the same percent of females.  When the study sample and the museum’s visiting population are the same, the sample has external validity.  And when there is external validity, we can draw conclusions from a study’s results and generalize them to the entire population.

 

There are several protocols RK&A follows to work towards external validity.  First, to select study participants, we use a random sampling method, and most often, a continuous random selection method.  To follow the method, we instruct data collectors to position themselves in a designated recruitment location (e.g., museum or exhibition exit) and ask them to visualize an imaginary line on the floor.  Once they are in place, we instruct data collectors to select the first person who crosses the line.  If two people cross the line at the same time, we ask data collectors to select the person closest to them.  After the data collector finishes surveying or interviewing the selected person, the data collector returns to their recruitment location and selects the very next person to cross the line.  It is important for data collectors to follow this protocol every time so as not to introduce bias into the sample.  For instance, data collectors should not move the imaginary line or decide to delay recruiting because the person crossing the line looks unfriendly.

 

Second, we record observable demographics (e.g., approximate age) and visit characteristics (e.g., presence of children in the group) of any visitor who is invited to participate in the study but declines.  We also record the reason these recruited visitors provide for declining (e.g., parking meter is about to run out).  These data points are important to confirm or reject the external validity of the sample because we compare demographic and visit characteristics of those who participated in the study to the demographic and visit characteristics of those who declined participation.  While the data points for comparison are limited, they are still informative.  For instance, a trend we have observed is that visitors 35 – 54 years are most likely to decline participation, so their voices are often underrepresented.  The same goes for visitors with children, which may be a subset of those in the 35 – 54 year age group; they are often underrepresented in visitor studies.  Knowing where your sample may be lacking is important context when interpreting the results.

 

For these two reasons, we aim to systematically recruit visitors for audience research and evaluation studies.  Even for studies that use standardized questionnaires, we hire data collectors who use a random selection protocol to recruit participants and track information about those who declined.  As such, we do not recommend using survey kiosks to collect data since visitors self-select to complete the survey and cannot be compared to those who decided not to complete the survey (and if you think kiosks may be preferable because you could boost the number of surveys collected, see my former post on sample sizes).  Again, there are always some exceptions to these general rules described above.  Yet, our goal is always to use protocols that promote external validity as well as document threats to it…because what you don’t know can hurt you.

 

Presentation1

RKA Blog_Evaluation Design Series_LogoSample size is a standard question we are asked, particularly for questionnaires since we will be using statistical analyses. For most audience research projects, we recommend collecting 400 questionnaires.  We are not alone in this general rule of thumb—400 is considered by some researchers (and market researchers in particular) to be the “magic number” in the world of sample sizes.  What makes 400 magical is that it is the most economical number of questionnaires to collect (from most populations) while keeping the margin of error at ± 5% (and the confidence level at 95%).  A sample size of 400 questionnaires keeps the cost of the research down while still allowing us to have high confidence in the results. 

 

To dive into this issue deeper, let’s talk about the three primary factors necessary to think about when deciding on a sample size: (1) population; (2) confidence level; and (3) margin of error.  Population is the number of people in the group from which you are sampling.  For instance, your population may be the number of annual visitors to the Museum, members, or visitors to a specific exhibition or program.  A fact that is often enlightening and counter-intuitive is that population does not have a proportional relationship to sample size.  To demonstrate this, follow my calculations by trying out one of the many sample size calculators available on the web, such as this one or this one.  Let’s start by determining a sample size for surveying the National Gallery of Art, which reported nearly 4 million visits in 2014 (3,892,459 to be exact).   Using the margin of error ± 5% and 95% confidence level, the sample size suggested is 385.  By comparison, the sample size suggested for The Phillips Collection, which welcomed 106,154 exhibition visitors in 2014, is 383.  Despite vastly different sized visiting populations, the recommended sample size for each museum differs by just two!  Again, this example demonstrates that sample size is not proportional to the population, but also, having an estimate of your population is often sufficient to determine a sample size (unless you are determining a sample size for a program with small attendance or other small populations).

 

Confidence level and margin of error (or confidence interval), as you might expect, indicate the level of confidence or how “sure” you are about the results of the questionnaires.  Here, the researcher has to make a choice about an appropriate confidence level and margin of error based on how the data will be used.  At RK&A, we generally plan for the margin of error at ± 5% and a confidence level at either 90 or 95% because it provides enough confidence in the data given how our museum clients use the data to make institutional decisions.  If we were working with a medical professional making life-or-death decisions, we would want to be more confident in the results (thus, a lower margin of error and higher confidence level).  So why not plan to be as confident in the results as possible (regardless of how they are used)?  Money.  Confidence comes at a cost because, like population and sample size, the relationship between sample size and margin of error is not proportional.  For instance, see the graph below based on the population reported above for the National Gallery of Art.  Notice that the slope of the line is steepest on the left side of the graph and more gradual on the right side.  This shows the law of diminishing returns at play.  There are great benefits when moving from a sample of 200 to 400 (margin of error diminishes by about 2 percent), but the benefits are not nearly as great when moving from a sample of 400 to 600 (the margin of error diminishes by less than 1 percent).  Thus returning to our initial point, collecting more than 400 questionnaires is rarely prudent since the cost of data collection will be going up disproportionate to the reduction of the margin of error.  For our museum clients, we do not think that increase in confidence justifies the extra costs.

blog chart

I would be remorse to end this post without a footnote.   While 400 is our rule of thumb for audience research data being collected through a standardized questionnaire, there are certainly many considerations and reasons why 400 might not be the magic number in every case.  We joke that the response to any methodological question is the often frustrating retort: “It depends.”  Sample size is no different—it depends.

RKA Blog_Evaluation Design Series_LogoThere is a reason behind every methodological decision we make as evaluators. While we give great thought to our evaluation design, our thinking is not always transparent.

 

We have decided to pull the curtain back on our thinking in a new blog series that we are calling “Evaluation Design: A Peek Behind the Scenes.” Our goals are to reflect deeply on our practice (and maybe even question the way we do things), make our evaluation decisions transparent to our museum colleagues, and challenge our evaluator colleagues to reflect on and document their own practices. Check in tomorrow for the first post in the series!

 

Also, if you have ever wondered about why evaluators do the things they do, send us your queries! Our hope is that these posts can provide relevant insight into our evaluation minds.

What’s My Job?

Ever wonder what we do at RK&A or what it’s like being an evaluator? Cathy recently tackled those questions in a column for the Tufts Museum Studies alumni newsletter called “What’s My Job?,” where Museum Studies alums shed light on what its like to work in a wide range of careers in the cultural sector. Check out her response below for a snapshot of what it’s like to work at RK&A! 

 

Sigmond_photo

Hello!  My name is Cathy Sigmond and I work as a Research Associate at Randi Korn & Associates, Inc. (RK&A). RK&A is a planning, evaluation, and research firm based in Alexandria, VA, that works to support museums and other informal-learning organizations as they pursue achieving impact.

What exactly does that mean?  Basically, it boils down to a few key things.  First, we help museum staff think strategically about and clarify what they hope to achieve (be it on the institutional level or for a specific exhibition or program).  Then, we also help them measure the extent to which they are achieving these goals (or how they might achieve them) through evaluation.  And finally, we help them make sense of the data (evaluation results) so they can move forward with their work more informed.

It means that in my job, I ask why? and what does that mean? a lot.  And as someone who loves to dig deep and find out what makes people tick, I love every second of it.

The day-to-day aspects of working as an evaluator at a small company (seven people across three cities!) aren’t always the most glamourous, but to me they are constantly exciting.  One day I might travel to a natural history museum to kick-off a front-end evaluation of a new fossil exhibition.  Another day I might spend my time managing data collectors I hired to conduct interviews at a botanic garden.  Often times I’m out in the field observing and talking with visitors myself (like when I got to observe citizen science programs in Puerto Rico!).  And some days, while I might be physically in our office in Virginia, I’m mentally holed up in the data, trying to make sense of visitors’ actions and opinions.

To be an evaluator means having a questioning stance about everything.  It requires meticulous attention to detail but also the ability to step back and understand the data holistically, in its broad context.

Perhaps most importantly, however, it requires having a passion for helping people do meaningful work.  And that’s why my favorite aspect of my job is translating the data for our museum clients (either in person or through a written report) and working with them to reflect on what evaluation results might mean for them moving forward.  I love knowing that my work helps others learn and grow into more informed and intentional museum professionals.  And I feel like I’m constantly learning and growing alongside our clients, and that’s a pretty awesome job perk.

If you’d like to learn more about all things evaluation/visitor studies/user experience (or just want to say hi!), please feel free to email Cathy at sigmond@randikorn.com or RK&A at info@randikorn.com

Amanda:

January came and went quickly.  At RK&A, we kicked off the year with a retreat.  Our main goal was to spend time together in one physical location–a rarity for our small office of seven.  We needed some bonding time… Thus, we participated in the personal response “tour.” My first experience with the “tour” was back in 2010 after reading Ray Williams’ article “Honoring the Personal Response: A Strategy for Serving the Public Hunger for Connection” in the Journal of Museum Education.  Williams describes a “tour” experience in which participants draw a question from a box populated by the “tour” guide and are “invited to look around a designated suite of galleries for about fifteen minutes to find a work that resonates with their guiding question” (page 95).  With two of my colleagues, we tried out the “tour” on a visit to the Phillips Collection.  It is one of my most memorable museum experiences.  

My prompt was to “Find a work of art that makes you feel proud,” and I really struggled with it.  Pride holds many negative connotations for me, which led me to pick a photograph of a train with a huge plume of steam rising from the engine.  I shared that pride makes me think of boastful people who are “full of hot air” or “blowing smoke.”  The work of art also made me think about how American pride in building a national railway blinded people to the atrocities incurred by laborers.  My colleague who had written the prompt said that she would never have connected that photograph to pride.  When she wrote the prompt, she was thinking about pride in her heritage and how it motivates her to be a better person.  Her reflection on my same question revealed to me how different our thinking is—but not in a bad way.  It just reminded me that we are all driven by different experiences and beliefs that shape who we are and how we act.  In that way, I think the exercise strengthened our working relationship because I could empathize with her more and understand her better as a person with unique experiences different from my own.  

I was eager to try out a personal response “tour” with my colleagues on our recent retreat.  Unfortunately, illness prevented me from sharing in the experience this time, but I was excited that they found it enriching, too.  For this “tour,” everyone responded to the same three questions or prompts, which included “Find a work of art that represents what excites you about working at RK&A.”  Get to know our staff by checking out their responses below!  

Embrace, by John D. Graham. 1887-1961.

Embrace, by John D. Graham. 1887-1961.

Cathy

“Embrace” by John D. Graham immediately stood out as the work of art that represents what excites me about working at RK&A.  It makes me think about how we all work together– embracing one another’s ideas, concerns, and supporting each other during (some very) busy times!  I love knowing that I work with such a smart, dedicated, and supportive team; one that embraces any and every opportunity to learn something new (from museum visitors, our clients, and each other).  Plus, the word “embrace” reminds me so much of another part of our work which I love–helping our clients welcome data with open arms.  When I look at this work of art, I see RK&A and our clients “embracing the messiness” and stepping forward into the unknown together, ready to absorb and learn from whatever comes our way.

 

music-room philips

The Music Room, Phillips House

Erin

We were standing in the Music Room of the Phillips House when asked to consider this question. and my answer occurred to me immediately. The house itself, an elaborate Georgian Revival home, is a work of art, and it contains the means to inspiration and learning for visitors. I think of RK&A in a similar light. RK&A is the container within which we not only support museums to achieve their desired impact with visitors, but through our work, we are also always learning and often feel inspired.

 

Dein blondes Haar, Margarethe, by Anselm Kiefer.

Dein blondes Haar, Margarethe, by Anselm Kiefer.

Katie

I selected Anslem Kiefer’s “Dein blondes Haar, Margarethe” as the work of art that makes me think about what excites me to work at RK&A.  At first glance, it may seem an odd choice; it’s a dark, brooding, abstract piece and not a piece of art I would normally stop to take a deeper look at. However, since I started at RK&A, I have found myself pushing to dig deeper and reflect on both my own practice in evaluation and how I can help our museum partners achieve meaningful impact. This means stopping and taking the time to reflect on things we might have otherwise overlooked and to tease out the details of a messy idea.  Look more closely at Kiefer’s piece and you will start to see some details missed at first glance—two eyes, a nose, a mouth, and strands of blonde hair hidden beneath the streaks of black paint. What more can we find with a pause for deeper reflection? 

 

The Little Machine Shop, by Jacques Villon. 1875-1963.

The Little Machine Shop, by Jacques Villon. 1875-1963.

Madeline

I chose “Little Machine Shop” to represent what excites me about working at RK&A. What first struck me were the colors; the blues and greens reminded me of the colors in our logo and documents. Then after reading the title and looking closer, I realized that RK&A is a little machine shop. In order for us to get out work done, we must work together and work together well. As a new team member, the idea of being a part of a small highly functioning team excites me.  And with words like “intentional” and “systematic” floating around my head from the previous day’s retreat exercises, I felt that the geometric shapes that make up the work and the idea of a machine resonated.

 

Efflorescence by Paul Klee. 1879-1940.

Efflorescence by Paul Klee. 1879-1940.

Stephanie

I chose Paul Klee’s Efflorescence as the work of art that most reminds me of my work at RK&A because of two qualities it possesses (for me): surprise and delight.  I was first surprised by the piece as I exited a gallery full of what I considered rather dull art; something compelled me to turn around and look back, and there was Efflorescence, a small work on an adjoining wall in a corridor. I was then delighted by the pulsating colors and delicate composition which seemed to vibrate with liveliness.  Even after 16 years with RK&A I am continuously surprised and delighted by my work.  Sometimes it’s the interaction with a client who has an ah-ha moment based on results of a study, and sometimes it’s my own ah-ha as something emerges from the data.  Other times it’s an encounter with a curator or scientist who shares their passions about an esoteric idea.  And oftentimes, it’s simply knowing that here I am again in another sacred, significant, or historic space that houses and cares for precious objects;  I am honored to be a part of that.

Randi

As you can see, the RK&A team is smart, thoughtful, sensitive, and courageous.  Selecting works that seem scary or prompt one to dig deeper is akin to having a curious mind and feeling secure enough within this container that we call the RK&A office to move forward methodically and systematically into our learning zones.  As evidenced from people’s descriptions, learning is at the heart of what we do.

 

Thank you to the Phillips Collection for being such an inspiring place!  All images are from the Phillips Collection