Remote unmoderated user experience research is a valuable methodology to include in any consultant’s toolkit. The ability to conduct task-based research that collects both qualitative and quantitative data enhances return on investment (ROI) significantly. This approach can also be used to augment other research methodologies.
This methodology synthesizes clickstream data with attitudinal and task success data, all taken from the same people at the same time as they interact with a website. Data regarding user intent are correlated with information regarding how they go about achieving their goals, and why the experience does or does not work well. Not only does this methodology provide an increased ROI for the cost of conducting a study, but customer experience researchers can use it to investigate user behaviors at every stage of the website life cycle.
An Effective Research Method
Intercepting visitors to a website and inviting them to participate in a remote usability study is arguably the most effective way to understand visitors’ goals and intentions. Many companies assume that they know why people come to their site, but when they ask real visitors, the responses may come as a surprise. Other companies don’t understand why people came to their site and build their site solely from a business perspective.
A large sample size gives business stakeholders more confidence in the results of research studies focusing on the overall health and success of a website or prototype design. With increased confidence in the research data, the results can be used to inform the design process by helping website development teams prioritize changes that need to be made to the site design.
Remote unmoderated user research is an effective method providing many benefits (Table 1).
| Research Benefit | Details |
|---|---|
| Increase geographic reach |
|
| Provide a local language experience for participants during the study |
|
| Eliminate moderator bias |
|
| Provide a natural environment |
|
| Acquire research data in a timely manner |
|
| Collect large samples |
|
| Obtain comparable results to in-person studies |
|
| Obtain data from actual site visitors performing their real tasks while they perform them |
|
A Cost Efficient Research Method
Remote user research is a very efficient way to conduct research.
- Multiple sites and countries can be evaluated, simultaneously reducing cost and time commitments.
- Both attitudinal and behavioral data are captured on significantly large sample sizes as compared to more traditional research.
- Multiple research projects can be conducted simultaneously by one researcher versus needing a team of researchers.
The cost benefits for remote unmoderated user research are listed in Table 2.
| Cost Benefit | Details |
|---|---|
| Reduced costs |
|
| Fewer difficulties conducting the research |
|
| Easier (and faster) data preparation |
|
Research Results Provide Real Insights
Understanding users’ goals, attitudes, and behavior provides real insight into why they navigate in a particular way. By looking at the behavior of the users through the clickstream data, you gain insights into their attitudes. These insights lead directly to improvements in the design of the site.
For example, a drop-off analysis was conducted on a booking process funnel. While the percentage of people that dropoff at each stage in the process can also be obtained by web analytics tools, the designer needs to understand why the users are dissatisfied or having problems. Remote, task-based studies tell you why they drop off—you can see exactly which people drop off, and at the same time know why they drop off. As a result, you have greater confidence in the reliability and accuracy of the data.
Remote, unmoderated research is particularly effective for conducting “true intent” and “open” web research. Attitudinal and behavioral data are gathered from people naturally visiting a website for true intent research, just as market researchers outside a store can. This method helps us understand why visitors come to a website, whether or not they are successful and satisfied, what they actually do on the site, and what impact the site experience has on the brand perception. With large sample sizes of 400 or more, it is feasible to filter the data by visit intent.
In open web research, users must be able to freely and naturally explore the Internet. Researchers learn which sites are “top-of-mind,” which are most popular and why, how the research is approached, and how specific websites are found. Remote, unmoderated research provides the ability to trigger questions when users move from one site to another, providing deeper insights into users’ motivations and behavior.
Patterns in user behavior and sites visited emerge with large sample sizes of 400 or more in open web research. These patterns help researchers understand how users find specific sites (if at all), how the web can be used to develop awareness and preference for specific products and services, and whether or not users naturally navigate to specific sites (If so, how and why? If not, why not?)
Companies use remote unmoderated research to conduct a variety of research, such as:
- BRAND AND VALUE PROPOSITION-Are we promoting the brand and value proposition? Do visitors know what we do and who we are?
- PROTOTYPE EVALUATION -Will the next site be successful? Will users find it compelling and helpful?
- TRUE INTENT – Why do users come to the site, and how well does it deliver on those expectations?
- USABILITY – What are the strengths, weaknesses, and opportunities to improve satisfaction and conversion on my site?
- COMPETITIVE RESEARCH-How do we compare to the competition? What strategies will help us win?
- INTERNATIONAL STUDIES -How are we performing in key markets internationally?
- NAVIGATION AND ARCHITECTURE-How well does my site enable key task accomplishment?
- OPEN WEB RESEARCH-How do users find our site? How can we use the web to develop awareness and preferences?
- CARD SORTING STUDIES -What are users’ expectations of the information archi tecture? What category names should be
used for navigation? - PERSONA DEVELOPMENT -Who is visiting our site? What psychographics best match our target users?
Remote, unmoderated, task-based, qualitative, quantitative user research is an excellent addition to any consultant’s toolkit. It does not replace other methods, but with a larger sample size, geographic reach, globalization, and ability to quickly run studies it adds value on it own merits
