Intern Weekly Response: Evaluating Success
Every week we’re asking our summer interns to share some thoughts and responses to various experiences and readings. This week, following a professional workshop on museum evaluation, led by JMM School Program Manager Paige Woodhouse we asked them to find an additional recent article on the subject to suipplement their workshop materials and relate what they have learned to the article. To read more posts from JMM interns, past and present, click here.
~Intern Hannah Balik:
Last week, us interns had the opportunity to learn about the different types of museum evaluations from JMM School Program Coordinator Paige Woodhouse. There are four types of evaluation that a museum do, in the four different stages of putting up an exhibit or executing a program. First, there is front-end evaluation, which takes place in the beginning stages of planning, and serves to evaluate the labor and monetary investment needed to complete the project. Through the use of focus groups and street interviews, museum professionals can get an understanding on visitor interest in a certain exhibit. The second type of evaluation, formative, takes place during the design and development of the project. Often in this stage, museums will prototype information panels or interactive learning objects in existing exhibits or neutral spaces. Users can then give feedback on their experience, informing the museum on the general reaction to the exhibit. Remedial evaluations happen after exhibits have opened. They give staff an opportunity to critically look at recently opened exhibits and change any small mistakes or other things that are not working for the exhibit. This can include spelling errors on information panels and other small errors. Visitor tracking is helpful in this stage, to help staff understand what parts of the exhibit need re-vamping, and which parts are working great. The final type of evaluation is summative, which takes place at the end of an exhibit, once it closes. This is the time for museum staff to look at the project as a whole and decide what worked, what didn’t, and discuss the outcomes of the project, expected and unexpected. This type of evaluation serves as an accumulation of all the data in the previous evaluations, while looking to the future and how to improve in the next exhibit. It surprised me how many types of evaluation there are, as well as all the different ways you can record and evaluate visitor experience.
I decided to read the article “Prototyping AR in a University Museum: How User Tests Informed an Accessibility Plan Including and Beyond the Museum” by Max Evjen. This article discusses how Michigan State University Museum in 2017 decided to launch an augmented reality pilot addition to one of their exhibits on animal diversity in Michigan. Users were able to augment the existing exhibit by seeing animated animals and text appear on tablets when pointed to a mural the museum had in the exhibit. According to Evjen, who works at the Michigan State University Museum, “The Hall of Animal Diversity is a gallery with taxidermied birds, mammals, and insects arranged by theme, for instance, animals of mid-Michigan, camouflage, etc. There is a large brown bear … a couple of simple activity stations, an interactive video of different bird sounds, and the Michigan wetland mural, created by a local artist, that includes a key at the bottom for identifying animals featured in the wetland scene.” This mural was a cold spot, meaning visitors often walked right past it, or didn’t engage with it as heavily as they did with other parts of the exhibit. In order to increase interaction with this area of the museum, museum staff decided to add AR, in the form of 10 Samsung Nexus tablets. Looking back on evaluation data from other institutions, they decided to use tablets instead of an app users can download on their own devices, as they thought users would be more likely to use an already set up tablet than wait to download an app, that they know visitors will likely never use again once exiting the museum.
This endeavor shows examples of remedial and formative evaluations. Some of the museum’s galleries date back to the 1960s and have few, if any, interactive elements. Through visitor evaluations and watching what visitors interacted with most, the museum found that this mural in the Hall of Animal Diversity was rarely viewed by visitors. The use of AR through tablets would allow for an update to the exhibit without changing the exhibit itself and encouraged close looking at an often-overlooked part of their museum. Also, through the use of evaluations from other museums who created interactives that needed a downloadable app, they realized that would not be the best option for them, because most people did not download the app or ever open it again after the visit. They instead decided to use tablets ready for visitor use.
After testing the AR, the museum already has ideas for its improvement. Through the use of questionnaires and behavior sampling, museum staff were able to get a better idea of what visitors thought about the Visitors said they would like to see more audio and interactive capabilities in the software, which the MSU Museum staff have taken note of. This project shows the power of evaluation – finding the shortfalls in current exhibits, and allowing for new ones to take shape, in a controlled, visitor approved manner.
~Intern Elana Neher
As museums move into the future and integrate technologies such as virtual reality, apps, and touch screens into their exhibits, it is important to know how effective these new interactives are in engaging visitors and achieving learning goals. Recently, the Cleveland Museum of Art released the results of a two-year study into whether their ARTLENS gallery, an experiential space that allows visitors to engage with artworks in a variety of ways through technology, is actually effective or just a gimmick.
Like any other kind of surveying, prototyping, or analysis, museum evaluation has its issues. The limited amount of questions that one can ask in a survey can limit the amount of information the museum gains and can leave valuable information out. Additionally, no matter how much time or money is spent on evaluation, if the museum does not use the results of the evaluation to change, then completing evaluation is useless. These are only two of the many issues that can come up with museum evaluation. Despite these difficulties, evaluation is crucial to museums in understanding how effectively they are achieving their goals and how they can change in the future to better achieve these goals.
Additional article: Why Evaluation Doesn’t Measure Up.
~Intern Megan Orbach
Last week, along with the other interns, I attended an evaluation workshop with Paige Woodhouse. As someone who has not been previously introduced to museum evaluation methods and after reading another article about it online, I was surprised to learn that most museums employ similar guidelines for evaluating their exhibits. The commonalities between the guidelines used seem to be four main steps: front end evaluation, formative evaluation remedial evaluation and summative evaluation.
We were able to learn, in this workshop, just how important evaluations of exhibits are for future applications. Firstly, some of the statistics involved in evaluation show who the visitors are, what they thought about the exhibit, feedback, general visitor experience and more. This kind of information is useful because it can help to improve the existing exhibit, or it can give pointers for the next one. It is also helpful because it can show how much visitors learned from visiting the museum and if there are better or different ways to present information.
Some challenges I envision when it comes to museum evaluations are what methods to use in order to gain the most information on the visitor experience. For example, some questions on surveys work better than others and this is sometimes not found out until after both types are tested. Other times, I would think that feedback is not always constructive and the path to improvement is not always clear.
The article I read discussed the four different types of evaluation, just as Paige did in the workshop. It outlined to me, as mentioned above, that many different types of museums can use similar evaluating guidelines.
Overall, I was very interested in the workshop and I feel that I learned a lot about project evaluation in general. A lot of the methods used can be used outside of museums as well and as a way to analyze various types of work. It emphasizes the importance of constructive feedback which is extremely important for growth and progress. The workshop and article also taught me that it is important to consider who the intended audience is before beginning a project; the work should and can be completely different depending on who is supposed to be viewing it. Lastly, I learned that it takes a decent number of people or visitors to form a good evaluation.
~Intern Ariella Shua
Last week, we learned about different methods of museum evaluation. One of the main ways evaluation takes place is by checking to make sure that the museum actually accomplished the tasks at hand. One of these factors is keeping visitors interested and pleased with their experiences. Visitors are often quick to point out if they’re unsatisfied. If something is broken, or if they simply aren’t impressed, they’ll remember that they were unhappy.
As we learned in our workshop, museums want this kind of feedback. Without it, they wouldn’t be able to plan future exhibits. And they wouldn’t be able to fix the exhibits that are already standing. Because at the end of the day, museums want to entertain and educate their visitors. That doesn’t happen effectively if the exhibits aren’t up to par. Museums that want to impress their visitors, and draw new ones, are willing to make changes. Fortunately for locals, the Baltimore Museum of Art (BMA) is one of these museums.
It’s not often that museums decide to put food and art right near each other. I can’t remember the last time I was able to bring anything edible into a gallery. It’s basically a museum taboo. But the BMA, in an announcement on Tuesday, revealed that they’re hoping to engage new visitors by appealing to those who prioritize food. They’re opening a new exhibit space downtown, at Lexington Market. Lexington Market is the oldest American marketplace, and it’s still open. The crab cakes there are rumored to be delicious, though I can’t back that claim up myself.
But art? That’s not what most people head to Lexington Market for.
Now, the BMA hopes to change that. With the opening of BMA Lexington Market, they’re aiming to draw new audiences. The first exhibit in the new space will feature photography, perfect to casually stroll by while eating a marketplace meal. The opening is a clever one. Customers at Lexington Market don’t anticipate seeing art. But if they like the exhibit, they may be intrigued to visit the main BMA. It’s been done before- the BMA has tested five other satellite locations over the years. In this case, it’s a change, but also a reliance on what has worked before. Almost a two-pronged evaluation: taking an effective method, and repurposing it for future visitors.
As Dave Eassa, manager of community engagement at the BMA explained in an email to the Baltimore Sun, “It’s for people who have never set foot in the BMA and folks who come to the BMA all the time to meet, create, and share.” Hopefully, the prediction is right. I, at least, may soon use the pretense of viewing photography to arrange a trip to score a good meal.
~Intern Mallory Connaughton
Personally, I didn’t know much before going into the workshop. I had always connected evaluations to the end of a project. But we also discussed other types of evaluations, ones that can be done at the start of planning and during. And we discussed the various methods of conducting evaluations.
One thing that caught my attention during the workshop was when we talked about journey maps, which are maps drawn by the conductor of the evaluation. These maps catalogue a visitor’s trip through the exhibits, tracking time and attention. This is used during the exhibit and can show what points attract attention and the retention time at the various sections. It not only provides an insight into the visitor’s interests and wants from the various points of the exhibit but can also be used to evaluate the set up of the exhibit, to see if the flow of the floor is good.
Looking more into journey maps I found an article, “Visitor Journey Mapping in Museums”. This article discusses journey maps, specifically the exploration of uses and conduct in a workshop run by Allegra Burnette at the Pratt Institute School of Science. The article discusses that the journey maps can be used to look at the different personas that may be visiting the museum and see how the exhibit impacts then directly. The article explains that detailed journey maps can track not only visitor’s movements and retention times but also their emotions as they move through the exhibit, any questions they may pose, and so on.
These journey maps, both the heavily detailed and the simpler ones, can both be implemented to track retention time which can aide in exhibit set-up. This information can be used to directly change a current exhibit or can be used to positively impact the following exhibit. Before this, I hadn’t known about journey maps, and I found these to be very interesting. It is a very dynamic way to evaluate museum exhibits.
Outside of the more hands-on evaluation that journey maps are we also discussed direct communication with guests; surveys, interviews and interactive exhibits. We also briefly talked about newer technology-based methods of conducting evaluations. From a quick search there are cites which comp0ile lists of evaluation methods and research, one of these being connected to the Institute of Museum and Library Services.
These ways of interacting with customers, either active or passive, are vital in preforming museum evaluations. They can be seen in some form at all museums and are imperative to exhibit creation. I think that, especially with the modern uses of technology, there will be more evaluation methods coming out surrounding technology, and direct interactives for the guests to partake in.