Commentary

You have to see it to believe it! Enhance market research and UX testing with remote video

Thanks to the unrelenting advance of technology we can now interact face-to-face even though we are in another place.

Video ubiquity provides great benefits to qualitative and UX research, and showing faces is just one side of it. Beyond the importance of seeing the participant’s face, researchers may be even more interested in seeing how they interact with products, services, devices, designs and content. And in many cases we can get the best of all worlds and not only see their face, but also simultaneously see their interactions, body movements, screens, cursor and finger movements, surroundings, family, friends, etc., not to mention also hear their spontaneous comments, sighs, grumblings, surprises and exclamations!

In my experience I have found remote video particularly useful for:

  • Ethnographies
  • In-depth Interviews (IDIs)
  • UX testing
  • Online/mobile communities

Ethnographies

Most researchers will agree that one the most insightful and fulfilling experiences in their professional life is the opportunity to physically visit the subject of a study in their own natural environment, breathe the air of their homes, smell the aromas of their garden, feel how the seat feels while riding in their car, or experience first-hand the commotion of their daily commute.

However, most researchers also know the pains this physical presence often entails, with long days spent racing from one end of a city to another in order to get just 2 completed in one day. And aside from lost time there are the costs, scheduling headaches, cancellations, and the always lingering questions of how much the presence of the researcher is influencing the subject’s behaviour.

EthnoVid

 

Video provides one of the least intrusive ways to get privileged looks  into participants’ homes and bedrooms

 

 

 

 

These inconveniences are becoming even more magnified as markets become more global, and budgets and product lifecycles tighten. Many times in-person ethnographies are simply not an option, and that is where remote ethnographies are an alternative that is simply faster, more economical, flexible, and less intrusive. Additionally, in many cases they can be simply more richly insightful for several reasons:

  • They allow participants to document a greater range of moments and circumstances without the imposing presence of a researcher.
  • The greater flexibility makes them ideal studies where it is vital that the key interactions be spontaneous as they facilitate in-the-moment and on-the-spot insights.
  • Also, this flexibility can be especially important in longitudinal studies where you want them to interact over time, or at various moments during the day that can vary in timing… like during breakfast or after lunch.
  • Many participants actually feel more encouraged to speak and act naturally when being observed only by a camera rather than a pair of human eyes with which they might feel impelled to make eye contact.
  • When not moderated in real-time (asynchronously), one researcher can conduct multiple ethnographies simultaneously. In one study I was able to follow over 10 young football (soccer) enthusiasts over a whole week, from mid-week practices and Playstation simulations to pre and post-game experiences, for a grand total of 10 weeks’ worth of research in just one!

In-depth Interviews (IDIs)

In contrast to ethnographies, IDIs have a long history of being performed remotely via telephone interviews. But now the richness lost by having a voice-only connection can be recovered via video, facilitating all the advantages of seeing the interviewee’s face, their surroundings, and in many cases also simultaneously their interactions.

IDI image

 

IDI’s via remote video give us insightful glimpses of the participant’s natural environment

 

 

 

All the benefits discussed under ethnographies also apply, just in a more concise way where contact is usually a one-time event, and so here it is especially important to make the best decisions regarding moderation alternatives. Real-time, or synchronous, moderating has the advantage that

  • The moderator can probe and ask for clarifications depending on the participant’s responses.
  • At least some people will respond better to speaking with a human.
  • If participants don’t understand something about the tasks or activities they can ask for clarifications.

However, automated, or asynchronous, moderating also has many advantages:

  • Time flexibility allows the interviews to happen whenever it suits the participant, so they can frequently be more relaxed or otherwise in the right mentality for the study.
  • Dozens of interviews can be conducted simultaneously in geographically disperse locations. I live in Spain but I’ve done studies in the US where I’ve gone to bed and had 20+ interviews ready for my review in the morning!
  • While some people prefer to speak with a human, others actually prefer talking out loud to the camera and feel more comfortable not being observed in real-time.
  • Many participants are more inclined to voicing criticisms and annoyance, whereas with another human present they tend to be more compliant, inclined to praise, and/or reluctant to confess that they are confused.
  • All the interviews are moderated in identical ways, avoiding any influences of different moderating styles from multiple moderators or moderators who don’t moderate exactly the same way every time.

User Experience (UX) Testing

Whereas ethnographies and IDIs are proper research methodologies, here with UX we are looking at more of an objective than a specific methodology. In fact, ethnographies and IDIs are both great methodologies with which to do UX research. However, I include it as its own entry because remote video packs a really potent punch for UX testing, mostly stemming from the relative ease of recording screens and cursor/finger movements, giving us direct visual documentation of complex user experiences.

UX pic

 

 

Many UX tools, such as that offered by YouEye, empower researchers to view and hear the participant + surroundings, track screen + cursor/finger movements, and set automated prompts

 

 

Many of the same benefits discussed above with ethnographies and IDIs also apply when using video for remote UX research:

  • Participants interact under natural conditions, and with the growing diversity of devices it has become essential that users interact via their own device, or their unfamiliarity with the device may interfere with the UX insights on the experience you’re testing.
  • And if you also record their faces and surroundings, then you enjoy getting all sorts of contextual insights into how they their natural environment influences their interactions with their device.

For automated, or asynchronous, moderation, all the advantages of time flexibility are also applicable, but even more important are:

  • Advantages relating to place and situation flexibility, which allows participants to capture behaviour in the heat of the moment at the scene of the experience. For example, a mobile app for finding deals can easily be tested at the shopping mall or on the high street. Asynchronous thereby facilitates testing in real-life situations.
  • Like with IDIs it ensures greater consistency in moderation, and also addresses the tendency of some participants to elicit help or clues from a live moderator when attempting tasks, thereby skewing success rates upward.

Online/Mobile Communities

Online/mobile communities refer more specifically to a particular kind of platform for managing projects than methodologies in themselves, and they are great tools with which to manage ethnographies, IDIs and UX testing, to name only a few. Here the real question revolves around how much you want your participants sharing and interacting amongst themselves, or exactly how and when they share and interact. These questions will decide how much you want to really foster a live community versus using a platform to capture and manage participant responses made on individual bases.

Ethosapp pic3

 

 

Apps such as the Ethosapp allow participants to rate and comment on each other’s posts, fostering competitiveness and/or collaboration.

 

 

 

And while online/mobile communities are more typically linked to asynchronous moderation, they don’t really have to be as many platforms allow real-time chat and video conversations, so live or hybrid live/asynchronous moderation is many times also an attractive option. Aside from all the advantages commented above with ethnographies, IDIs and UX, online/mobile communities with video have distinct advantages:

  • The platform allows researchers to post follow up questions and probes that then become specifically connected to the videos.
  • Activating the social aspects motivates your most social/creative/visual participants to get into competition mode, driving them to out-do the other participants. This may lead them to exhibit behaviour that goes beyond the everyday and natural, and so is not ideal for every study, but if robust and creative results are what you are looking for then it’s worth it to make the effort to make a competitive and/or collaborative community. In this sense we begin to go beyond the relatively straight-forward MR and UX techniques that are the focus of this article, and into the realm of competitive co-creation, crowdsourcing and collaborative innovation, which are exciting topics in themselves, but which will be covered at another time.

Interested in exploring ways to incorporate remote video into your Market Research and UX testing initiatives? Please feel encouraged to contact me at brian@leapvision.com to pick my brain on any challenges you’re facing.