A World Usability Day Case Study: Conducting Multinational Expert Reviews

Posted on: 12:07 PM by UX Magazine

As products and services become increasingly global, brands must reflect cultural differences across many diverse markets. Companies need to be consistent in their Web offerings across countries to address economies of scale and reinforce brand identity. However, to be successful, they must address local needs as well. Therefore, global companies need international user research to extend past a single perspective and ensure that the design reflects the needs of the cultures where a product will be used.

There are several methods for conducting international user research:

  • Using local teams for each country
  • Identifying one team for multinational testing
  • Using remote observation
  • Performing remote testing

How do you choose between these methods? On one hand, it is important to engage local expertise in each target country because participants respond better when the interviewer is from the same culture. In addition, direct local involvement in translation and localization, session planning and administration, and outcome analysis is necessary for good results. Nevertheless, you must ensure that the testing procedures and final deliverables are consistent across locales. Otherwise, it might be difficult to compare results between countries—you will not be able to tell whether differences are due to methodology or to genuine cultural differences.

The ideal method for global user research would be to use local teams for each country, while ensuring the overall integrity of the process. One way to do this is to hire a user research team with global testing capabilities. To illustrate this approach, on World Usability Day 2006, the User Experience Alliance (UXA) and associated companies performed a series of heuristic reviews around the world. This article discusses the process and benefits of this type of review and how global usability services can both deliver real business benefits and make a significant impact for local users.

Setting up the Global Review

The UXA approached organizations with international profiles across different business sectors to help plan the event. Organizations that agreed to participate included the following:

  • Ikea A furniture retailer with a clear Swedish identity but an international presence—thirty-five sites in Europe, North America, Asia, and the Middle East.
  • Tourism BC The tourist authority of the Canadian province of British Columbia, which seeks to encourage visitors to come to BC through a range of websites targeted at specific countries.
  • A major international banking and financial services company This company has a presence in overseventy countries and territories in Europe, Asia-Pacific, the Americas, the Middle East, and Africa.
  • An international communications provider This Germany-based company was interested in receiving international user input on its mobile applications.

The UXA members and affiliates that conducted the reviews included:

  • Australia: The Hiser Group
  • Brazil: Mercedes Sanchez Usabilidade
  • China: User Experience
  • France: Axance
  • Germany: SirValUse Consulting GmbH
  • Netherlands: User Intelligence
  • Russia: UIDesign
  • Spain: Xperience Consulting
  • Sweden: Usability Partners
  • UK: Serco Usability Services
  • U.S.: User Centric, Inc.

Planning

In many ways, the international heuristic reviews were like any other evaluation; however, a lot of planning is required to coordinate a review of eleven different teams in as many time zones.

A consistent approach was essential, so weekly phone meetings involving all reviewing organizations were held well in advance of World Usability Day. As a result, teams were able to review and finalize briefing documents and common reporting templates early. The lead company (or the client sponsor for each product or application) in each locale then modified these templates to include background information on the target users and typical tasks, as well as on the service or website to be tested. These documents and other necessary items (such as mobile handsets in the case of the German telecommunication company) were then distributed to the reviewing countries.

Execution and Reporting

As World Usability Day began, UXA’s Australian partner started the review process. As the day progressed, each company completed its reviews and reporting templates and then fed the results to the lead company who worked directly with the client. Final reports were also sent to a designated “record keeper” so that all results could be reviewed together later.

Findings of the Expert Review

A website can work well internationally whether the organization is globally focused, such as Ikea, or has a clear local identity. However, any site can suffer from issues that hinder easy use. In reviewing the findings from our twenty-two completed expert reviews, certain trends emerged, some of which were not surprising to UXA members. Typically, some issues remained constant across countries, but in areas such as language, symbols, and imagery, the reviewers found unique results related to locales. Each is described below.

Translation

Following are two examples of how inaccurate translations can create issues for users.

In our reviews, we found that translations were generally professional, but one review uncovered a higher level issue that indicated that the site was not locally focused. The wrong character, a direct translation of the English word “Go,” was used as the label for the button used to execute a search. This is inappropriate for Chinese users, as “Go” is rarely used in Chinese search engines.

Text may be translated correctly but not fit well within a site structure designed for the original language if the text length increases. For example, this translation of “New!” from English into Russian is too big for the area available and, condensed, becomes difficult to read.

Cultural Issues

Context that is understood in one locale may be misunderstood in others. For example, while “BC” (for the province of British Columbia) is familiar to users in Canada, it is much less familiar in other countries.

Culture can also impact the appropriateness of writing style. While a friendly and informal tone may be appropriate for a holiday site targeted to North American users, Germans prefer a more formal style. We also noted this issue with another of the sites reviewed—the German users were addressed informally (“Du”) throughout the site. This is not typical for German websites.

Care is also required when localizing details such as dates and phone numbers, as these can become a barrier to users in other countries. For example, overseas users looking at a site aimed at North America and seeing the phone number “Call 1-800 HELLO BC” may become confused. In North America, the use of letters within telephone numbers is common practice, yet elsewhere it can be an alien concept.

Imagery

Images are important because, from them, users can quickly pick up cues as to the cultural appropriateness of a site. We frequently heard users say, “The people on the site don’t look very “.”

Note that a concentration of one type of people isn’t necessarily a bad thing—it all depends on context. For example, a travel website for Asia should have Asian people on it.

Visual style can also give users a feel for whether the site is aimed at them. For example, one home page that was tested in a range of countries was found to be appropriate in all of them except Brazil. Brazilian users perceived their version of the site as being “sad” because it was not colorful enough for them. This left them with the impression that the site was not targeted to or localized for them.

And it’s not just people. For example, a route-description icon based on local conventions at the home office can look foreign and alien in other locations.

Symbols

In addition to reinforcing that the site is not local, the wrong symbols can actively prevent users from completing tasks. For example, we noted several instances where currency symbols were not displayed properly.

Conventions as to how certain types of information such as dates, times, telephone numbers, and currency symbols should be displayed also vary from country to country. Not following these can give a clear cue that the site has not been fully localized. The example above was seen in France, where the convention is to display the currency symbol (ital)after the number, not before.

Conclusion

The trend of conducting global user research is growing rapidly. For international organizations, the need to receive feedback from users at the local level is not just important, but essential to product success. It is the responsibility of usability professionals to conduct global testing and demonstrate the impact that investing in the user experience can have. It is also our responsibility to continue to refine the process, methods, and results to produce the highest quality of results in these often complex efforts. By organizing a multinational event and sharing some of the results, we hope to have demonstrated why forging a partnership with your users worldwide can provide the best path to achieving your company’s business goals.