The audience effect and its impact on usability testing

Have you ever tried to type when you have someone watching you, and it seems like you forgot how? It’s called the audience effect, and in a recent usability testing session, I wondered if a similar experience was occurring.  

We gave seven people limited information on the client, and asked if they would take part in a 45-minute session to help us improve a website. We asked them to perform tasks that were completely new to them, while sharing their screen, and we recorded the session for further analysis. These tests were moderated through WebEx, and only connected through audio, which avoids a certain aspect of the pressure of being watched. Still, I wondered if the participants stumbled in part due to the audience effect.

The audience effect is the tendency for people to perform differently when in the presence of others, rather than when alone. Studies on individuals’ behavior have focused on the effect by real, imagined or implied presence of others. Research shows that people perform better on simple or well-rehearsed tasks, and worse on complex or new ones when they feel they are being watched.

At the beginning of the usability testing sessions, we try and make the people feel comfortable, and usually tell them that ‘we’re not testing you, we’re testing the website. Any frustrations you have only point to the website’s usability, and it’s on us to make those improvements.’ Does this statement really wipe away any potential for the audience effect to come into play?

Jakob Nielson states that usability testing is a quality attribute that assesses how easy user interfaces are to use. If people cannot use the website, they will leave. If people become flustered because of the scenario, how representative is the testing of the actual user experience? I wonder how we can set up the usability testing to ensure a stress-free environment that ensures a true reflection of the website’s usability.

As a way to quickly test our hypothesis, we tried an approach with the group of seven participants:

  • We told some participants that we weren’t testing them, but that we were testing the effectiveness of the website
  • We didn’t tell the other participants anything about testing the effectiveness of the website

So, what did we learn? There are standard usability testing practices, however we must decide whether we adjust or fine tune these practices to account for the audience effect. Is it possible that words are not enough? Or that the audience effect isn’t a major concern in this kind of testing? We are going to continue to work with the assumption that the audience effect may have a minor impact on results, and we welcome thoughts from other testers.

Have you run into a similar scenario, or have research on the audience effect within usability testing? Something to think about the next time you misspell something in a big meeting!


Lisa is a consultant with three years of experience focused on lean process management, user experience and business strategy development. She has completed a Bachelor of Commerce degree in Marketing from the John Molson School of Business in Montreal, finishing her studies at the Copenhagen Business School in Denmark.


Comments:


Leave us a comment: * Your information is never shared