Skip to content Skip to footer

User Experience

The Magazine of the UXPA

User Experience

The Magazine of the UXPA

Search

Search Results for: diary studies – Page 2

Remote User Research: Opportunities for Adoption in Asia

June 5, 2022 No Comments

User researchers are an inquisitive lot, no matter in which part of the world they work. However, there is diversity in the research discipline in different parts of the world. The practice of user research in Asia has grown significantly in the past few years. Large organizations, tech SMEs and startup unicorns are aggressively recruiting UX talent, increasingly in short supply. “Why should I hire you?” today sounds like an obsolete question when hiring for UX roles.

Researchers and project teams in Asia are experiencing higher than ever stakeholder buy-in for conducting user research. The recent shift to “all things remote” behavior has led all UXers to make an involuntary shift into gathering feedback remotely. Cloud-based collaborative design and user testing platforms such as Figma, Miro, Usertesting.com, UXArmy, dscout, etc. have become the UX work essentials.

However, despite the COVID-19 work-from-home model, usage of remote research tools and platforms in Asia remains limited. This is reflected in a survey of around 100 researchers and designers in Asia conducted by UXArmy. 25% of respondents reported research experience ranging between 8 to 20 years, while over 40% had research experience ranging between 3 to 8 years. The remaining reported less than 3 years’ experience.

Sixty percent of all survey respondents agreed with the statement that “Remote UXR is not practiced in Asia as much, in comparison to the Western countries.” This article will discuss current user research practices in Asia and make the case for incorporating more remote research as standard practice.

Figure 1
Researchers in Asia self-report conducting less remote research compared to those in the West. Alt text for accessibility: Graph. 60% of survey respondents agreed with the statement “Remote UXR is not practiced in Asia as much, in comparison to the Western countries,” compared to 29% who agreed with the statement “Remote UXR is equally practiced in Asia and the Western countries.”

FIg 1. Researchers in Asia self-report conducting less remote research compared to those in the West

Current practice of user research in Asia

The majority of user research leaders in Asia come predominantly from UX Design and market research backgrounds. Few leaders have transitioned from non-UX roles like market research, analytics, product owners, project managers, etc. The responsibilities for these roles naturally required a need to make sense of user feedback and take action on it. Most people in these roles had close interaction with end user behavior in the product development process. Prior to 2010, most companies in Asia did not have a dedicated user researcher role. Instead, UX designers doubled up as researchers when required. After 2010, with the prolific adoption of digitalization in the industry, companies wanted to de-risk their investments resulting in higher demand for specific user researchers.

At junior and entry levels, a near majority of user researchers are likely to have a formal education in research-related fields or have experience doing usability testing. Newer roles at mid-senior level in the industry are now demanding user researchers to have formal education in a field related to user research (psychology, sociology, anthropology, etc.), and sound knowledge of statistics.

While budgets allocated to user research have been increasing, they remain small in comparison to design budgets. A meager 4% of those who responded to the survey said their organization’s user research budgets exceed 10% of the product development budgets. User research budgets are typically a part of Product development or Design budgets but may also come from marketing or be a separate ad hoc allocation.

Figure 2
User Research budgets are seldom dedicated; most are taken from the Product development or Design budgets. Alt text for accessibility: Graph.81% of respondents indicated research budgets were taken from product development or design budgets, while only 13.5% stated user research had its own dedicated budget in their experience.

FIg 2. User Research budgets are seldom dedicated; most are taken from the Product development or Design budgets

A good majority of the UX Research happening in Asia is design-focused, pre-launch usability testing, or competitive benchmarking. Much of evaluative user research is planned to align with development Sprints. User validation of early-stage concepts is gradually increasing in frequency. Due to less tangible outcomes and no proven method of calculating ROI, the application of longer-term generative user research to identify latent user needs, understand user behavior or ideate new product opportunities remains low, or in some organizations, even non-existent.

Figure 3

Fig 3. Most User research in Asia happens during the product design phase

Only a handful of organizations like banks, start-up “unicorns,” and global behemoths like Google or Meta (Facebook) are investing in long-term user research in Asia. The researchers’ effort-spend chart supports the observational indication that less than 5-8% of the total UX research efforts in Asia are spent on long-term generative user research. To clarify any discrepancy, the “Understanding / Ideation” measure in the graph above includes the generative research that happens as part of the Discovery phase in the product development lifecycle.

Figure 4

Fig 4. Most of the user research in Asia is still evaluative and executed in-person

Current usage of remote user research platforms in Asia

While the remote user research platform market is hot worldwide, in Asia, usage is highly fragmented. Preferences for the range of tools available vary depending on user role and desired output. While some tools have been created for and in-market, the usage of remote research platforms remains miniscule in countries like Mainland China, Japan and Korea.

Many researchers in Asia are attracted to online research platforms that produce a quick collection of usability metrics along with graphs like Heatmaps. However, the benefit of usability metrics alone seems to pale in comparison to the platforms that provide video clips of participants using the product and sharing feedback.

Researchers focused on usability testing appreciate metrics-driven platforms, while qualitative researchers tend to prefer platforms supporting screen and video recording. For most countries in Asia, especially Southeast Asia, users are not comfortable with face recording during an Unmoderated Test. For remote user interviews, turning on the Webcam is considered normal, especially when participants are offered substantial incentives.

Despite the possibilities of remote platforms, as discussed, usage is still not high across Asia. Only 30% of researchers agreed with the statement: “For organizations to improve User intimacy, Remote UX Research benefits in a big way.” Compare this to the large selection of a neutral answer which could be heavily biased due the new normal set by the coronavirus pandemic.

This suggests remote research is not a preferred mode of user research in Asia, as a majority of respondents are not sold on remote research platforms as being robust or providing enough value from an output standpoint.

Figure 5

Fig 5. Remote User research is not seen to play a significant role in improving User intimacy

A significant percentage of user researchers in Asia also report low awareness about their options for remote platforms as a reason why they are not choosing these tools as an option.

Figure 6

Fig 6. Lack of awareness and comfort level in using traditional methods are the top two reasons to avoid using of Remote UXR platforms

The following is a list of platforms reported by the participants of the survey. This list is in no specific order and no recommendation of any specific platform is being made.

UserZoom

Used by large enterprise customers in Singapore, Malaysia.

usertesting.com

Used by large enterprise customers, mainly in Singapore.

UXArmy

Used by a mix of large enterprise customers, Small and Medium Enterprises (SMEs), and startups in Singapore, Malaysia, Indonesia, Thailand and Australia.

Lookback

Used by large enterprise customers, mainly for remote user interviews as an alternative to ZOOM, Microsoft Teams, Google Meet, etc.

Maze Design

Used primarily by  designers and design researchers in SMEs and tech startups. Popular s in Malaysia and Indonesia.

Optimal workshop

Used for Information Architecture research by large enterprise customers The Card Sorting tool is hugely popular across all of Asia.

Userlytics

Reported as the most rarely used tool in Asia.

Figure 7

Fig 7. Popularity of select Remote UXR platforms tools in Asia

 

 

The case for using remote user research in Asia

To clear the air, here is a disclaimer. I do not recommend a complete switch to remote user research. Ethnographic research in real-life environments remains one of the most effective and least biased research methods. Irrespective of user interviews, diary studies, field studies or any other research methods used, the “DO“ / “DOES” part in the Empathize phase of Design thinking is critical to be able to identify and define the problem to be solved.

However, remote user research methodologies have a role to play as the demand for research grows in Asia and teams  .

The following highlights some compelling reasons for project teams, UX designers, and researchers in Asia to include remote user research in their product development lifecycle.

#1 Research Recruitment is a top challenge

Asia is the most diverse continent.  The variations in geographical landscape, weather, culture, language, and income levels present a huge business opportunity. The eleven time zones in Asia create a tremendous challenge to schedule user research interviews in multi-country projects. A popular joke in my research circle is: “To confuse a researcher, get them to schedule user interviews with participants from Bangkok, Ho Chi Minh City, Kuala Lumpur, Jakarta and Bali.” Those cities are in four different time zones and include participants speaking four to five distinct languages.

Another difficulty in finding research participants for products with a target market across Southeast Asia is the varying communication language. In Thailand, Vietnam, Indonesia, and Myanmar the majority of people have a very strong preference to write and speak in their local language. While English is popular in the Philippines and Malaysia, it is common that people speak with an abrupt mix of local language and English.

Illustration of a few of the several language scripts in Asia

Fig. 8 Illustration of a few of the several language scripts in Asia

Unlike the Euro in the Eurozone, there is no common currency in Asia. Asia has over 40 different currencies, with each country having their own. With so much diversity, research recruitment and scheduling of user interviews is best handled if automated. This makes a clear case for automated recruitment platforms which provide end-to-end recruitment services including screening, scheduling and distribution of incentives. This is a gap that remains to be addressed by technology and remote research platform creators.

#2 Growth of research teams calls for a remote research platform

Aligned with digitization and digital transformation initiatives, many businesses in Asia have been aggressively building in-house research teams. The outsourcing of research to UX consultancies has notably reduced. The UX consultancies are still engaged to augment the in-house UX research teams to secure unbiased feedback avoiding “Déformation professionnelle”. The planning of most user research studies typically starts 3 weeks ahead. However, projects with business backing can start with little notice due to the push from stakeholders for some quick decision making. On the other hand, less user-centric organizations require months for internal convincing and decision making to proceed with projects involving user research.

Team sizes in large organizations and tech startups vary vastly. Sometimes the ratio of UX personnel to Engineer ratio in tech startups can be higher than those in large organizations. Personally, I have seen and heard of team sizes in large organizations ranging between 5 to 20 user researchers. Researchers, designers, copywriters and research ops members comprising a UX team can exceed 100 members in a handful of organizations.

With UX teams getting larger and new user researchers joining the workforce, productivity has taken the spotlight. ROI of user research attracts a greater scrutiny from the top management.

Since the time and budgets are never infinite, UX teams cannot grow indefinitely to keep up with the required scale and demand coming. The management and UX leaders have recognized this limitation, and therefore the remote research platforms are being adopted more readily.

While many user researchers may be more comfortable doing in-depth interviews and online surveys, the timeline to deliver insights from usability testing during development needs to align with development Sprints. The agile model of development hardly leaves a time margin to properly plan an execution of research using the traditional user research methods. As businesses in Asia are no exception to this reality, either the research is selectively conducted for some Sprints or research teams have resorted to using a cloud-based user research platform.

 

#3 Need gaps in existing Remote UX Research platforms

Top need gaps reported by researchers who are already using remote tools include:

  • Access to participants matching screening requirements; more variation in behavioral attributes of participants than offered in the existing marketing user panels
  • Availability of platforms offering automated participant recruitment
  • Integration with existing design tools, online repositories for research, and (client-provided) user panels
  • Accommodation of technical limitations of participants. e.g. poor internet connection, lower tech savviness, fear of installing malicious software
  • Localization in existing platforms, e.g. language translations
  • Setting clear expectation on information received from remote unmoderated testing; depth of participant responses from unmoderated studies is not comparable to those from moderated interviews

Conclusion

By nature, Asia is fast moving. Being ahead of the pack to win titles and rewards is admired in Asian culture. Creating a great user experience demands a longer-term outlook as the ROI of user research is not always immediately visible. That is one of the potential reasons why the practice of user research had a delayed start in Asia. In some countries like Japan and Korea, technology teams still drive UX decision making.

Research in Asia is changing fast with rapid digitalization. More and more prominent businesses from the West are setting up offices in Asia to benefit from a “not to be missed” opportunity. Emerging tech unicorns like Grab, Carsome, Ninjavan, Carro, Traveloka, VNG, Lineman wongnai, CRED, Zerodha, are making headlines. Large organizations based out of Asia have also stepped up their design budgets. They are setting up user research teams in the quest of creating products that are highly relevant to the target audience and deliver a superior user experience.

Researchers in Asia have high expectations from the remote research platforms and are currently underserved. The variety in language, geographies, and user attributes like literacy gaps and income presents a huge challenge to the creators of remote platforms looking to market to the researcher community in Asia. With challenge comes the opportunity, and the companies building remote research platforms stand a good chance to be among the next wave of startup unicorns.

The UX of AR: Toward a Human-Centered Definition of Augmented Reality

November 26, 2017 No Comments

Augmented reality, or AR, is becoming mainstream. Sources from TechCrunch and ABI Research to Business Insider state that the industry of AR and VR (virtual reality) combined will reach more than $100 billion by 2020. Key players like Google, Microsoft, and Apple are making major investments in AR technology. Earlier this year, Apple introduced an iOS Dev toolkit called ARKit at the Apple Worldwide Developers Conference. Just one year earlier, Apple CEO Tim Cook foreshadowed the likelihood of AR’s ubiquity:

“I do think that a significant portion of the population of developed countries, and eventually all countries, will have AR experiences every day, almost like eating three meals a day. It will become that much a part of you.”

What Is AR?

But what is augmented reality? Perhaps the most common image that comes to mind is of Tom Cruise in the 2002 movie Minority Report. He’s standing in front of a large screen, looking smart, waving his gloved hands around, and manipulating hundreds of small pieces of information in front of him. This provides a helpful illustration of information layered onto reality that can be manipulated through some sort of technological element. But what does that really mean?

Merriam Webster’s Dictionary defines AR as “an enhanced version of reality created by the use of technology to overlay digital information on an image of something being viewed through a device (such as a smartphone camera).” Similarly, the Oxford English Dictionary defines AR as “a technology that superimposes a computer-generated image on a user’s view of the real world, thus providing a composite view.” These definitions both emphasize an experience with technology over an experience with the world and other people in it.

Even Wikipedia describes AR as “a live direct or indirect view of a physical, real-world environment whose elements are ‘augmented’ by computer-generated sensory input such as sound, video, graphics or GPS data.” This definition does include ideas about “a real-world environment,” but it’s still largely a tech-first definition.

The challenge with tech-oriented definitions is that they tend to keep the tech at the heart of the matter and neglect the people, or end users. As a result, applications are driven by what kind of technology is available for an AR-enhanced project, rather than being driven by the type of human experiences we want to create through augmentation. To resolve this, we need to bring user experience (UX) more prominently into the AR conversation.

Toward a Human-centered Definition of AR

First, we need to reframe the definition of AR away from its technology alone and toward a human experience. Simply put, we define augmented reality as “a human experience with the world that is enhanced by technology.”

Now, this definition could be applied to a lot of things that we’re all used to working on, not just AR. But that’s also our point. AR is a means to an end, not the end itself. AR should be less about technology doing things to people and more about people engaging with the world around them, and having that world enhanced by technology where and when appropriate.

In the sections that follow, we break this idea down and describe methods for focusing on a human experience. We then provide a set of design principles to consider when designing layers onto the real world. We conclude with a summary of technological considerations to engage with once the people and places of the experience have been sorted out.

Focusing on a human experience

The first part of our revised definition is about “a human experience” because we want the end user to be front and center. So, let’s talk about people and the kinds of research we need to do in an AR context.

We need to apply user-centered design when it comes to AR, rather than technology-driven design, so that we don’t augment reality with a bunch of stuff no one needs. To accomplish this goal we must do the legwork to uncover the latent needs of our end users to decide what augmentations to layer into their world. The first step in user-centered design is to define your users. AR makes this a little tricky. In an AR context, your users are everywhere. They aren’t just the people who are opting in to use your product; they are also the people being affected by the augmented experience other people are having. These are our intended and unintended users as shown in Figure 1.

For example, consider the now famous AR game Pokémon Go. The intended users were the players, but because the landscape for the augmentation was so extensive, many people found themselves unintentionally involved in the game. People walking down the streets were bombarded by groups of players trying to catch Pokémon or find Pokémon gyms. Players were also reported showing up in cemeteries in unprecedented numbers. Though cemeteries are public places, they are typically ones of remembrance.

The experience of these unintended users attempting to engage in acts of remembrance, while also inadvertently becoming part of the landscape for game play, is an important one to factor into our design thinking. What we’re describing are not entirely new problems. Mobile phones and other pervasive technologies have shifted our social norms around what is acceptable in public places. But the augmented reality movement will exacerbate these issues even further.

Image showing intended users with devices in foreground and unintended users in the background
Figure 1. In an AR scenario, the intended user is interacting with the app, but an unintended user is also affected by the experience.

Using methods that let us see

In order to inform and inspire design that addresses the needs of intended and unintended users, we need to use methods that allow us to see. Phone interviews and surveys aren’t going to do the job. We’ll need to dig up our applied ethnographic methods, which allow us to both observe and talk to people. Contextual inquiry is one of the most effective methods for achieving this goal.

Contextual inquiry is the practice of engaging with end users in their environment rather than in a lab-based setting. As the intended user engages in key activities, the researcher is also able to observe the unintended users in the environment who may be affected by an augmented experience.

Contextual methods can be expensive and time intensive, however, and since not all audiences will be accessible for direct observation, we need to balance them with other methods that let us see. Diary studies and photo elicitation techniques are effective, lower-cost alternatives to understanding both intended and unintended users within their contexts. For these methods, researchers give participants prompts to gather information about people’s motivations for use, social experiences, and environmental contexts. Researchers follow up with the participants to ask about their images, videos, and other artifacts to understand the personal meaning, value, and use of the objects and experiences within a given context.

Seeing the environments that our designs will live in is hugely helpful in any project, but it’s mission critical for AR. We need to know what it is we’re augmenting in order to truly enhance it. And when it comes time to test ideas and designs, do as much of the testing as you can out in the wild. Labs are fine for testing features and functions at a detailed level, but we need to really vet AR concepts out in the user’s world. It’s the only way to know whether we will positively or negatively affect our intended and unintended users.

Considering the real world

The second part of our revised definition is about the end user’s experiences with the world. Much like our intended and unintended users, AR experiences occur in expected and unexpected places. So we need to think about more than just our targeted use contexts. Some questions to consider include:

  • Where do you expect users to experience this augmentation?
  • Where else could they experience this augmentation?
  • Could this experience occur with other people?
  • Could users have to navigate in the real world while using this augmentation?

Designing for users in potentially public spaces adds many criteria to consider. As we examine the myriad places that users could inhabit while having an augmented experience, it becomes clear that ensuring user safety is important. If we’re going to enable users to have AR experiences out in the real world, then we have a responsibility to ensure that those experiences don’t place users in harm’s way. To ensure safety, we need to make sure that the real world is prioritized over AR content. A balanced AR experience shouldn’t supplant the end user’s perception of reality with AR.

Mediating Your Users’ Reality

AR is about mediating reality with technology. When we’re designing for people out in the real world, we must think about how much mediation is necessary to accomplish the goal of the augmentation. AR that focuses too heavily on augmentation will ultimately diminish the user’s experience with, and ability to focus on, their actual reality. To ensure that the experience uses the right degree of mediation, we need to keep a few guidelines in mind as we design, such as keeping your augmentation simple, designing for a narrow depth of field, and prioritizing line of sight.

Keeping your augmentation simple

Simplicity is an essential guideline for ensuring proper mediation. Referring back to our Tom Cruise example, he appears to be extremely adept at managing a plethora of digital screens, and that’s great for Tom Cruise. However, most people won’t be as effective at using intensive or distracting AR, especially if the augmentation involves navigation. With this in mind, it’s imperative that AR experiences avoid overstimulating users. The more an augmentation attempts to grab attention, the more users will focus on it, instead of reality.

In 1998, NFL broadcast networks began applying a judicious amount of AR to televised football games, which is one of the earlier mainstream implementations of augmented reality. During the broadcast, TV viewers would see a yellow “first down” line appear on the field. The yellow line spans the width of the field, 10 yards from where the team on offense has the ball, and helps to highlight the distance that the offensive team needs to reach. The yellow line is designed to appear as if it really exists on the field, as shown in Figure 2. Its application is so subtle that many NFL viewers often forget that it’s NOT real. The yellow line is a perfect example of simplicity as an effective guideline for AR, as it achieves its communication goal without users being distracted by or focusing on it.

Image of yellow line in an NFL game, which helps TV viewers better understand the team’s goal by marking it on the field.
Figure 2. The NFL yellow line is not actually on the field but an AR construct to help TV viewers understand the goal.

In recent years, NFL networks have expanded their AR repertoires by adding more elaborate visuals to accompany the yellow line to provide even more information to the viewer. In contrast to the original yellow line example, however, the newer AR elements are too visually complex. The unnatural depth shown in the Figure 3 example creates dissonance and distracts from the game. The AR takes precedence over reality, negatively affecting the viewing experience.

Image of more complex version of the yellow line in an NFL game
Figure 3. NFL networks attempt to capitalize on the success of the yellow line with more intricate (and confusing) AR visuals.

Designing for a narrow depth of field

A narrow depth of field in AR means that the augmentation will only draw focus to objects within a reasonable range. If an object in the real world is outside of the user’s natural focus, then that object likely doesn’t need to be highlighted by an augmentation. By supporting users’ natural perception of depth, the AR can more effectively integrate with reality, regardless of the location of the user. Whether the user is navigating by foot or vehicle, only displaying content within a specific range helps the user to effectively parse AR content, prioritize reality, and ensure safety.

In the example in Figure 4, an Android app displays nearby points of interest (roughly three-quarters of a mile) on top of the real world. The depth of field is very wide, which has a number of significant impacts on the experience. The absence of a visual hierarchy between the points of interest makes it difficult to quickly grasp which locations are closer to the user. If the AR narrowed the depth of field, the user could parse the information more effectively, and also retain focus on the real world.

A secondary impact of the wider depth of field is the overabundant points of interest displayed on-screen. The AR obstructs the user’s line of sight, making it difficult to see the real world behind the augmentation. This example is a tablet app, but one could imagine how problematic this would be for users navigating in a vehicle, or wearing some form of AR glasses.

App screenshot displaying too many points of interest onscreen.
Figure 4. The Android Street Lens AR App obstructs users’ view of the real world with too much AR content.

It’s easy to think about depth of field with a visual augmentation, one that presents layers of information within an acceptable range of the user. AR isn’t just about what you can see, though. AR experiences can leverage other senses to augment user experiences. A concept from Harman demonstrates how aural augmentation can enhance a user’s focus by employing an appropriate depth of field (or in this case, field of hearing). (Watch the video on YouTube: https://www.youtube.com/watch?v=r-d_DucmVzw). The AR prioritizes real world objects that are close to the user (in this case a train and a friend) to help ground the user in their real-world experience.

Prioritizing line of sight

When users are having AR experiences in a wide variety of places, ensuring adequate line of sight (or field of hearing) for your users is critical to ensuring their safety. Many popular AR concepts “pin” user interface elements onto the user’s view of reality, allowing them to see information contextual to their surroundings. As a broad concept for enriching a real-world experience, this sounds great. However, this content can also distract users or block them from seeing something important in the real world. Prioritizing the user’s line of sight leads to AR experiences that transmit only essential information to the user, and in so doing, helps ensure their safety.

Returning to Pokémon Go, the AR mobile game that launched in the summer of 2016, we see the importance of prioritizing line of sight for the sake of human safety. Pokémon Go encourages users to go explore outdoors and use their mobile devices to find and “catch” digital creatures. The game makers wanted to create something that inspired youth to go outside and be social. Unfortunately, it worked too well. Soon after launch, national news outlets reported on throngs of people wandering the streets at all hours of the night, distractedly staring at their mobile devices.

While the idea of getting your users to go outdoors is a noble aspiration, the makers of Pokémon Go didn’t consider all of the places that users might have this augmented experience. The method of catching Pokémon in the game requires users to hold a device in front of them, and use their fingers to “catch” a digital creature. This causes users to focus on their device (and the digital creature superimposed in front of reality) and not on their surroundings, causing significant line of sight issues for end users.

The notion of oblivious pedestrians wandering the streets is alarming. Even more alarming, however, is that the game makers didn’t realize that the game enabled (and to some degree rewarded) distracted driving practices. In a 10-day study conducted shortly after the game’s launch in the summer of 2016, the Journal of the American Medical Association reported that Pokémon Go was responsible for more than 110,000 incidences of dangerous driving practices. In the same period, other news outlets reported on 14 automobile accidents directly related to Pokémon Go.

The study concluded, “It is in the public interest to address augmented reality games before social norms develop that encourage unsafe practices.” And that’s really the point here. In these nascent stages of AR, we have a responsibility to consider the potential impacts of an AR experience before releasing it into the wild. When conceptualizing an AR experience, if we understand our user’s needs and we consider the expected and unexpected places that the AR will occur, we can ensure a safer AR experience.

To contrast the Pokémon Go example, the automaker Land Rover has a vehicular AR concept that prioritizes line of sight in an interesting way (Watch the video on YouTube). Instead of adding AR content to augment the user experience, Land Rover uses hidden cameras to make the front part of the vehicle transparent to drivers moving on rough terrain, so that they can safely navigate the environment. This augmentation uniquely prioritizes line of sight using AR to strip away, rather than add, obstructions. It may feel counter-intuitive to think about removing obstructions as a form of augmented reality, but when we focus on people and places, those are the kinds of innovative design solutions we can come up with to truly enhance reality.

Technology to Enhance People’s Lives

The last part of our revised definition is about how we enhance a user’s world with technology. This is the area that most of the literature focuses on because AR traditionally has had a tech-first orientation. Thomas Caudell, a researcher at Boeing, coined the term augmented reality in 1990 after creating a head-mounted device that projected unique sets of plane schematics onto electrical panels. Being able to quickly “toggle” between different sets of schematics helped Boeing electricians work more efficiently. The head-mounted device enhanced the electricians’ visual senses, so that they could see something they couldn’t readily see before.

Head-mounted displays have been considered a go-to technology implementation for augmented reality, but that form of technology hasn’t made much headway into mainstream public acceptance (beyond industrial or gaming contexts). When we start to think about the types of AR experiences that will become mainstream, think about technology that is readily accessible, easier to use, and connected to our senses.

AR is a sensory experience

Many AR examples focus on visual layers. However, AR is really about all of the senses, and augmenting what you can see, feel, smell, and hear.

  • Visual AR. Visual AR is (perhaps, obviously) about enhancing sight. If your users need to be able to see something they couldn’t see before (like a complex circuit path in an electrical panel), or they need to remove something that was obscuring their view (like our Land Rover example earlier), then a visual augmentation might be the right way to go.
  • Olfactory AR. There aren’t many examples of olfactory AR to pull from yet, but like our other senses, it’s an important one to consider. Some studies say that smell is the sense most connected to memory, so an olfactory augmentation deployed in this context could hypothetically add a lot of value.
  • Touch in AR. Touch AR can be used to enhance users’ tactile experience with objects in the real world. Your users could benefit by receiving new or enhanced information through touching everyday objects.
  • Auditory AR. This is really where we think we’ll see the most AR technology advances next. Augmenting people’s sense of hearing can be very powerful in applications ranging from navigation to meditation and more.

Enhancing Human-to-Human Experiences

One of the best examples we’ve seen that takes into account all of the key elements raised in this article is an auditory augmentation called Wayfindr. Wayfindr’s goal is to empower vision-impaired people to overcome isolation through audio-based navigation. The design team—a collaboration of the Royal Society for Blind Children and ustwo—started with people and took on a human-centered challenge rather than a technology-driven one. They thought about and designed for the needs of not just their intended users—individuals who are visually impaired—but also the unintended users, that is, everyone else trying to get to work in the morning at the same time.

The team thought carefully about the places the augmented experience would occur. They observed, co-designed, and prototyped in the wild of London’s Underground. They used just enough technology to mediate the experience to keep it simple and focused. They narrowed the depth of field so that navigation instructions were given at the right place and time. And they prioritized line of sight, or in this case field of hearing, without obscuring other important sounds necessary for navigating a subway station. Wayfindr tackles a human challenge with a human-centered augmented experience and won the Interaction 2016 conference’s “Best in Show” award.

Just like Wayfindr, as you encounter new projects and opportunities, start with people. Consider both your intended and unintended users. Think about the places they’re in and mediate their world carefully. Then select the right technologies to enhance the right senses. We are on the cusp of something becoming ubiquitous. And it’s on all of us to make sure we do it the right way, augmenting our reality to truly enhance our human-to-human experiences. As Steve Jobs famously said, “You’ve got to start with the customer experience and work backwards to the technology.”

Engaging Teams with Rich Reporting: Recipe for a Research Findings Expo

August 15, 2010 No Comments

Does this sound familiar? You plan and execute the perfect research study. You gain insights that have huge impact for the product. The research findings touch not only workflow and design but suggest major business implications addressing questions your product manager has been asking for months—and some that she hasn’t even thought to ask. You craft a detailed, voluminous report that will most likely end up in the Smithsonian as an exceptional example of 21st century user experience research. However, there’s one critical problem…nobody reads it. Or, if someone does read part of it, they forget the main findings and their application to the product as soon as another email plops into their inbox. Why does this happen? What might you have done differently?

The Study

Our team recently concluded a field study examining a new platform component for managing online display advertising. Our goals were to define the strategy around the product, to learn more about the users, and to test preliminary design concepts.

Data collection took place in four countries and involved contextual observation, interviews, and an artifact walk-through. Participants were also asked to keep an incident diary to share their experiences with the new component. The study team included two UX researchers, an interaction designer, a product manager, and the lead engineer.

Our Idea

The field study produced a large number of insightful findings that we needed to communicate to our stakeholders. However, because we had not previously conducted fieldwork for these particular stakeholders, there was uncertainty surrounding the value and substance of the research. Additionally, we were unsure how the stakeholders processed information or utilized research findings. We also wished to promote and demonstrate the value of field studies to other organizations such as support, sales, product management, and engineering.

Fearful that we might end up writing a report that would get passed over, we decided to try the new idea of holding an expo where stakeholders could “experience” the research instead of reading about it. We imagined a large meeting room with a self-guided exhibition of posters, artifacts, and videos inviting stakeholders to learn about our results. We planned to conduct this expo for a full day, during which time the study team was available to discuss research findings and recommendations with the extended team.

Preparing the Expo

During the preparation phase, we brainstormed the contents of the expo and developed a “mind map” of findings. We recruited a designer who created visual representations
of our findings and helped us design several posters for the expo. Posters included:

  • Study background
  • What is a field study?
  • Methodology
  • Participant map
  • Task workflow
  • Participant quotes
  • Themes with product implications (see Figure 1)
Image of expo poster
Figure 1. A poster with the top ten findings. Each finding has an illustration, a short title, a description, and its design implications. (Some of the text was blurred for confidentiality).

Representative artifacts gathered from participants were selected to showcase. Insightful entries from the incident diaries were also included. In addition, edited video clips from the study sessions were set up in viewing stations around the expo room.

A slide show was produced including the following topics:

  • Research questions
  • Process workflow
  • Tools and systems users use
  • Task matrix
  • Top ten issues that mess up a process
  • Players
  • Who is the product for?
  • Pictures of participants
  • Artifacts

Promoting the Expo

Marketing and promotion were crucial to gaining visibility and attendance. Several days prior to the expo, we sent email invites to the direct and indirect stakeholders. We asked key engineering and product management directors to invite their teams. We hung invitation posters in elevators, mini-kitchens, and outside the expo room (see Figure 2). The day of the event, we sent two email reminders to all of our stakeholders: one in the morning and another in the afternoon.

We used a room with a glass wall and looped the videos and presentation to attract the attention of people who casually walked by. As word of the expo spread throughout the office, more people showed up to see what was happening.

image of poster in front of expo room
Figure 2. An invitation poster in front of the expo room.

 Holding the Expo

We created a multimedia experience, and set up the room like a gallery exhibit, including video viewing stations (to watch select user clips), posters illustrating key findings and product implications, printed blog posts (incident journal entries), collected artifacts that people could pick up and discuss, and a slide show that ran in a continuous loop in the room (see Figure 3). During the expo, the researchers, product manager, and lead engineer answered questions about findings, encouraged discussions about the meaning of the findings, and shared our field study experience.

After the expo, we provided copies of the posters to the product manager, engineering director, and product management director. The following week, we gave presentations to those stakeholders who were unable to attend the expo.

photo of expo room
Figure 3. The expo room: projected presentation, posters, video stations (laptops), and artifacts (arranged on table).

 The Website

Using all the content we had prepared for the expo, we created an internal website to be launched on the morning of the expo. The site served as a repository of artifacts, diary entries, videos, and notes from the study. This interactive “report” pretty much wrote itself thanks to all the expo preparation. The website was easily discoverable through the intranet search, provided an engaging presentation format, and directly linked the report to the project site.

Outcomes

The results from the expo exceeded our expectations. Approximately fifty people attended the expo, and over 100 visited the expo website. It’s highly doubtful that this many people would have taken the time to read a standard research report. Product managers, engineers, sales representatives, support staff, and UX researchers and designers visited the room throughout the day, watching video clips, discussing the artifacts, and intensely debating the study findings and their implications (see Figure 4). The research really came to life!

The chief benefits of holding an expo included creating a high level of engagement, rendering study results more memorable, raising the profile and impact of UX research, and increasing acceptance for field studies.

Reflecting on this effort, expo attendees still utilize findings and recommendations from this study, even a year after it was conducted, and our team members ask for more studies with similar deliverables.

photo of expo
Figure 4. Expo attendees discuss incident entries.

Lessons Learned

The expo helped us to better appreciate the power of face-to-face interaction with our stakeholders. Facing so many tangible findings in an expo

setting made our stakeholders engage with the study results and recommendations.

We found that presenting findings via an expo “democratized” the experience because attendees were more willing to ask questions and engage with the material. This is less likely to happen during traditional report presentations, which are often dominated by lead product managers and one or two vocal participants. As a result, many more ideas were generated from a wider group of people.

  • Additional lessons we learned include:
  • Consider giving visitors something to take away (for example, handout of key findings).
  • Promotion and marketing are key to a good turnout.
  • Producing the materials for the expo (posters and slides) made writing the “report” (in other words, the expo site) relatively easy.
  • It pays to include stakeholders as part of the study team.
  • Having a great designer is necessary to create strong posters, presentations, and an inviting overall expo experience.
  • Including a multimedia component was very conducive to engagement.

The recipe for an expo provided in this article may, like any recipe, be tweaked and adapted by the chef to match the nature of the research being reported and the stakeholders involved. The important point is that rich reporting is an improvement beyond the traditional written report leading to a more meaningful engagement among a wider variety of stakeholders.

Expo Ingredients

  • 2 full time researchers, no interruptions!
  • 1 motivated designer
  • 1 engaged product manager
  • 1 engaged engineer
  • 5 days of preparation (including analysis)
  • 1 excellent promotion plan
  • 1 business services store to print 7 high-quality posters for $500
  • 100 yards of painters’ tape to hang posters
  • 2 extra laptops with headsets for video stations
  • 1 conference room booked for a full day
  • Wall space, glass wall
  • 1 projector
  • 1 video camera & tripod (we recorded the expo itself)
  • 1 digital camera
  • 1 sign-up sheet
  • 2 audio speakers (for music to enhance the atmosphere
    in the room)
  • 1 good music library

 What if You are on a Tight Budget?

  • Design a presentation (or two) and use a projector (or two) instead of printing posters.
  • If you use posters, print them on plain paper instead of glossy paper.
  • Create handmade collages.
  • Use stock photos instead of a designer.
  • Collect many artifacts and present them in the room.
  • Shorten the duration of the expo; three hours may be sufficient.

用户体验研究人员可能会获得中肯的见解，从而对产品产生巨大影响。研究结果不仅触及特定工作流和设计的可用性，而且还承载了主要业务和设计建议，这些建议不仅回答了您产品经理数个月一直在问的问题，甚至还回答了她没想到的问题。研究人员决定像平常一样编写报告。但问题是没有人阅读报告。或者，如果确实有人在阅读研究报告，但他们收件箱中一旦有新邮件，就忘记了这些主要成果，以及如何将这些成果应用于产品中。

这篇文章建议使用一种不同的沟通工具，以帮助将您作为研究人员所获得的知识传递给利益相关者 – 可用性成果展览。展览是一个全日活动，利益相关者在这期间会“体验”研究成果，而不仅仅是阅读。展览室中提供有自导的海报、物品和视频展示。研究小组和直接利益相关者举办展览，并可与扩展团队一起阐述和讨论研究成果与建议。

这篇文章讨论举办展览的原因，确定组织展览所需准备的材料和举办的活动，建议宣传推广展览的方法，以及对作者举办展览所获得的成果和教训进行总结。

文章全文为英文版確かにユーザエクスペリエンスの研究者は、製品に多大なインパクトを与えるための洞察を得るだろうし、調査結果は、特定のワークフローやデザインのユーザビリティに関わるだけでなく、ビジネス、製品のデザインで、プロダクトマネジャーが何か月にもわたって明らかにしたかった問いに答えるばかりか、考えもしなかった問いにも答えるほど、大きな影響を与えるだろう。研究者は、いつものように、結果レポートを作成するわけだが、問題は誰もそれを読まないということだ。または、レポートの一部を読むだけで、何か次のメールが届いたとたんに気が逸れてしまい主要な調査結果や、どのように製品にその結果を応用すればいいのかを忘れてしまう。

この記事では、研究者が得た知識を関係者に伝達するのに有益なコミュニケーションツールとして、ユーザビリティ調査結を報告するためのエキスポを提唱している。このエキスポは終日のイベントで、関係者は結果を読むのではなく、結果を「体験」するというかたちを取る。エキスポの展示ルームにはセルフガイドのポスター展示、さまざまな展示品、ビデオなどが置かれる。調査チームと直接関係者がエキスポを主催し、調査結果を発表し、そこから推奨される事柄などをさらにメンバーを加えて討議するなど、さまざまな内容を組み込むことができる。

この記事では、エキスポを開催する意義について述べ、実際に開催するためにどのような下準備が必要かを指摘し、エキスポを宣伝する方法や、著者が実際に開催したエキスポの結果とそこから得た教訓をまとめている。

原文は英語だけになります

Remote Design Teams: How to Use Cloud-Based Tools to Collaborate

January 29, 2017 No Comments

Traditionally, members of a design team work together in one location. Now thanks to the practices of working from home it’s becoming more common for design teams to be geographically distributed, recruiting designers who live in other cities, and hiring remote design agencies.

If a design team is considering adding remote team members, it must adjust its design process. If it attempts to operate like a completely in-person team even after adding remote team members, communication and creativity will suffer. It’s critical for design team managers to employ strategies and tools to ensure the effectiveness of distributed teams.

What Makes In-Person Design Teams Successful?      

Design teams that work in the same location enjoy the advantages of easy in-person communication and impromptu collaboration sessions. Such teams also benefit from the opportunity to participate in in-person user experience research activities together, such as contextual inquiry and usability testing.

The key to success for an in-person team is the ability to communicate quickly, especially when visually sharing design ideas for feedback and collaboration. Members of a productive design team are always communicating, white-boarding, and helping each other iterate. And when conducting research, it’s always best to observe users in their environment, test prototypes in the right environment, and gain contextual insights.

Although it’s not possible to completely replicate an in-person work environment with distributed teammates, it is possible to achieve effective collaboration during research, ideation, prototyping, and iteration. To create a successful distributed design team, it takes more than just online software for chatting and user testing. It requires a thorough understanding of the design process and how to adapt the practices of in-person teams to distributed teams.

Moving the In-Person Model to the Cloud

Although managing a distributed design team comes with its challenges, it also comes with unique advantages. Design teams that recruit in remote locations can attract top talent that may not be available in their area. Remote designers in different countries can offer new perspectives and cross-cultural insights that may not be possible with a domestic team. Design teams composed of designers with different backgrounds will generally produce better work since a variety of perspectives will generate more well-rounded ideas.

There are several practices that can help distributed teams work effectively. First, it’s important for team members to schedule time every day to check in about what they’re working on, if they have any blockers, and if their projects overlap. This creates visibility and the opportunity for team members to jump in with advice or feedback early on. Workplace chat programs such as Slack or Microsoft Teams offer the ability to create multiple channels, so it can be useful to create a “Standup” or “Daily Check-In” channel where designers are expected to summarize their plans for the day.

Second, if designers are in drastically different time zones, it’s vital to create charts or notes for the team about overlapping working hours, how best to reach other designers, and how to communicate if there’s an urgent need. Websites like World Time Buddy can help people schedule meetings across varying time zones without having to do mental math and potentially getting times mixed up.

Screenshot of online collaboration tool Trello
Figure 1. Trello is a card-based collaboration tool that can be used for things like Kanban boards, affinity mapping, and collecting qualitative feedback. The ability to tag, collaborate, sort, and pin cards makes Trello a powerful online tool.

Third, distributed teams must have an effective way to collaborate in real time. This kind of rapid ideation is where great design ideas come from. Design teams often use white-boarding, scenario mapping, affinity grouping, and group sketching to ideate. Luckily, many of these traditional in-person exercises can be accomplished online via cloud-based tools. Here are a few popular options for online ideation and collaboration :

  • Trello can be used for affinity mapping and organizing sticky note sessions so that designers can review and up-vote items. (See Figure 1)
  • CardBoard enables digital scenario mapping and allows distributed designers to participate in real time. (See Figure 2)
  • Red Pen allows designers to mark up design comps with their notes and quick revisions.
  • Concept Board helps designers pull together quick concepts to begin a design project.
  • Draftboard helps designers send working design files to other designers to get quick feedback on a project.

Finally, regular design reviews can keep designers on track and ensure team buy-in. Design reviews can be held via applications such as GoToMeeting or Join.Me, which allow designers to share their screens while others can watch and comment in real time.

Screenshot of online collaboration tool CardBoard
Figure 2. CardBoard allows designers to conduct scenario mapping sessions remotely in real-time with other designers.

Cloud-Based Tools for Remote User Research

Design teams benefit from having talent in remote locations. They also benefit from conducting research with users in a variety of locations by accessing diverse behaviors and perspectives without the need to travel. Fortunately, the same kinds of tools that help distributed teams work together also empower teams to conduct research with remote users.

Some user research methods are better suited for remote applications than others. Generally speaking, if observing the context in which a user is going to use a product is fundamental to a research project, then an in-person approach is best. However, if understanding other aspects of the experience is part of the goal of the research, then a remote approach can be very advantageous. Users can be anywhere, just like the members of the design team.

Some research methods that can be done remotely include:

  • Surveys: Tools such as SurveyMonkey, SurveyGizmo, and Google Forms can enable online survey data collection.
  • Diary studies: Google Forms provides a flexible platform that can be tailored to many types of projects.
  • Card sorting: Tools such as Optimal Workshop enable remote card sorting with clear instructions to participants.
  • Usability testing: Once a design is ready for testing, there is no shortage of usability testing platforms available. Two popular ones are UserTesting.com and UserZoom.
  • Interviews: Tools such as Skype or GoToMeeting enable face-to-face interviews.

Communication Is Key

If you work on a distributed design team, the key is to always be communicating with your team, always be sharing ideas and collaborating as much as possible in real time, and always be learning from users in different locations. By employing cloud-based collaboration tools that facilitate many steps of the design process, any design team can easily keep collaboration and communication working across distances, time zones, and schedules.

Positioning Field Studies for Company and Customers

September 15, 2007 No Comments

Getting Started Within Your Company

The mantra for usability practitioners is “know your users”. Similarly, when proposing field study research, it is important to understand who you are “selling” field study research to—who are the people holding the key to unlocking the resources, schedules, and budget required for the research activity to occur. Understanding this audience allows you to create persuasive arguments to gain their support and keep them interested and engaged in the discussion. It also increases the likelihood of obtaining resources and financial backing for your project.

The task of gaining support for field study research may be formidable depending on the scope of your project. Large-scale field study initiatives may require support from multiple levels in the organization (for example, directives from upper management executives, support from middle management, and cooperation from individual employees,) and from numerous divisions within the company.

When proposing field study research to executive management, first identify their priorities and discuss the benefits within the context of these priorities. For instance, if executive priorities are centered on customer satisfaction and its effect on the bottom line—sales and revenue—then position field studies as a mechanism to build and foster the customer relationship. If the priority for upper management is operational effectiveness, discuss how field study data can streamline product development by reducing ambiguity, providing clear and concise requirements early, and eliminating churn and indecision during development. Additionally, remember that upper management has limited time and a large organization to manage. Keep the conversation at a high level, but be prepared with details if asked. Try to leave the executives with the answers to what you want, the cost (in time, resources, money), and the return on investment for the company. Chances are they won’t give you an immediate answer to your request, but if you leave them with a clear sense of your proposal, they are likely to respond favorably.

When proposing field studies with product managers and marketing managers, think about their primary responsibilities and concerns; they need to specify and prioritize product requirements to meet the customer and market needs. Be sensitive rather than antagonistic about how they determine requirements. Offer field studies as a complementary method that primarily yields user requirements regarding interaction designs and usability criteria rather than business and functional requirements. For example, if the requirement is to have email functionality, the field study research would provide insight on users’ current experiences and expectations for email, which would help specify users’ functional and interaction requirements. Field study findings provide necessary information that will later facilitate discussions and mitigate conflicts during development and construction of the product. The data also helps with prioritizing requirements and specifying designs to ensure a highly usable, quality product.

Engineering teams have the burden of building the perfect product. They want to ensure the product they commit to building is truly meeting customers’ needs. Include engineering in field visits; it provides engineers with a rare opportunity to directly observe users and see how their products or similar products are being used and implemented. It offers a glimpse into whether the product is being used as intended, how it has been extended and improved, and any shortcomings. This insight is invaluable and a good reality check for engineers; the value in field studies is well-defined product requirements and use cases, which leads to less churn on product specifications and user interface designs. Engineering is always faced with schedule pressures, thus is sensitive to factors that may extend timelines and supportive of measures to minimize schedule disruptions. Field studies should be positioned as requiring time early in the schedule, but providing a depth of data that reduces time spent revising requirements and design specifications during the construct phase. Additionally, if scope changes are necessary as the project progresses, field study data helps inform decisions about the trade-offs and consequences to the user.

Getting to the Customers

Now that you have gathered support from inside your company, it’s time to identify and recruit customers to visit and observe. With B2C (business-to-consumer) products, it is a matter of determining the objectives and contacting consumers directly to arrange for visits. With B2B (business-to-business) products, identifying and recruiting customers may be more involved; you may need to convince partner consultants, third party vendors, and customers before gaining access to end users to observe.

As with all research activities, involving the appropriate participants is crucial. Participants need to address your research goals to ensure their input is valued. You do not want anyone to question the findings because the proper user group was not studied. Depending on the nature of product release, there are different user types you want to observe. For example, if the product is entering into a new industry, then the users would be non-customers. If the product is being enhanced and sales revenue is expected from current customers upgrading to the new version, then you want to target those customers. You will need to work with different people within your company to access the various user types, and you will need to develop a value proposition for each to gain access to their customers.

The sales team builds and enhances customer relationships for future sales deals, especially with high revenue generating customers, or those with potential for an upgrade or cross-sell. Field studies should be positioned to the sales teams as a tactic for fostering customer loyalty and a demonstration of individualized customer care. Provide sales with examples of positive perceptions customers will have following the field studies.

If the primary focus for the next product release is refining the existing product and addressing known issues, tap into the customers through customer service and technical support. They are well aware of the product shortcomings and have direct access to customers experiencing the most difficulties. Pitch field studies to customer service and technical support organizations as a mechanism to improve product quality and increase customer satisfaction.

Customer Value

Regardless of whom you work with to determine customers for site visits, it is important to create a value proposition for the customer. Place yourself in the position of the customer and think about the questions customers would ask: Why would the field study be important to me? How is it a valuable use of my time and my employees’ time? What are the benefits to me? Access to customers’ employees is not easy as there may be perceptions that you would be intruding on their employees’ privacy and time. Customers may fear the field study would distract employees or reduce productivity. Be sure you understand the customers’ perspectives and come up with compelling benefits before contacting the customer. For example, participating in a field study allows the customer to voice their requirements early on and is an opportunity for the customer to make their employees feel empowered because they can be involved in the definition and design of the products they use. Again, it is important to develop messages that align with the customers’ core values and alleviate their concerns. Be upfront and clear about expectations, time commitments and duration of the activities, and determining deliverables the customers may expect in return for their time and participation in the field study.

Analyzing and Communicating Findings

The ultimate goal of research is to make a difference to the product design. The best way to assure findings are applied is to give people data they’re interested in. Do not overwhelm them with the minutiae. Analyses and reports need to be tailored for the various recipients. Avoid the academic style of traditional reports, and make the report memorable and easy to comprehend. Focus on getting a few key points across to the appropriate recipients. Present the findings using formats appropriate for each group. Use marketing terms, (“Top Ten Findings”, “Myths and Realities Exposed”) to make your points memorable.

Observational data may be analyzed to provide a list of improvements they can make to the current implementation of the product. This provides some short term, easy fixes for immediate improvements. Additionally, customers may be interested in seeing a summary of how the latest version has resolved some of their issues, and which issues the company should consider when upgrading to the next release.

For the user experience team, the data are analyzed to produce the usual deliverables, such as user profiles, task analysis, wants and needs analysis, and user interface design requirements.

For the sales team, the results may lead to targeted selling. It’s possible to provide the sales team with specific selling points to encourage customers to upgrade (a summary of customer’s current issues may help them sell an improved version) or provide insight on customer needs and where an additional product may be offered to fill that need.

For the cross-functional product team, which includes—but is not limited to—product managers, marketing managers, the engineering team, quality team and documentation, field studies provide real customer usage scenarios for prioritizing requirements and determining solutions. A detailed context of how users interact with the product provides a vivid view of the users that is indispensable during product design and development. It provides data that may be referred to throughout the product development cycle to make informed decisions and to understand the consequences to the end user when making trade-offs.

Conclusion

Field studies provide benefits to many groups—users, managers, sales, marketing, and support staff. By understanding and communicating these benefits, you can maximize your ability to undertake such studies and the effectiveness of the deliverables they produce.

Measuring Emotions: Self Report as an Alternative to Biometrics

August 3, 2016 No Comments

“Yes, but I want to know if users are delighted,” our client said after reviewing the metrics we suggested for a study to evaluate new gesture-based gaming concepts. After creating a research plan with traditional usability metrics (for example, ease of use and satisfaction), we realized we weren’t hitting the mark.

We knew that there’s more to delight than ease of use. Ultimately, stakeholders want to understand the entire emotional journey of a product. Was some frustration in the middle of an experience suitable if the end result was excitement? With more and more requests from clients to understand the whole story, we started down a path to discover how to incorporate emotions into our research.

Early Explorations: Relying on Moderator Interpretations

Emotions, and the role they play in the user experience, have been in the UX conversation for some time. Biometric analysis was an obvious candidate for understanding emotions, but wasn’t feasible due to clients’ limited timelines and budgets. Price tag aside, we were concerned with the distraction that biometric-related hardware would introduce. We’ve worked hard to create labs that evoke specific settings, such as a living room, a home office, and even an operating room. The emphasis is on the user rather than the recording equipment; intrusive biometrics-related equipment would call attention to the testing mechanisms rather than the topic at hand.

We began by observing participants’ behaviors and noted moments of delight and frustration based on their facial expressions, actions, and verbal cues. We quickly realized this was far too subjective and difficult to standardize. Despite efforts to ensure that moderators would record emotional observations consistently, we encountered a greater range of facial cues, behaviors, and comments than expected. Sometimes assumptions were made by our researchers based on these verbal and non-verbal expressions and the data felt forced. Unsure of how to accurately interpret users’ cues, we began to explore the idea of relying on participants’ self report.

Later Explorations: Turning to Self-Report Emotions

Our initial approach to collecting self-report data involved asking participants the open-ended question of how they were feeling at various points in the experience. However, we observed that many participants struggled to find words to describe their emotions, and the results were difficult to quantify. We explored the idea of providing a standard set of emotions to participants that would aid in reflection and allow for researchers to easily quantify and map the data. Inspired by Robert Plutchik’s wheel of emotions, we created an emotion chart that plotted emotions along one axis and levels of intensity along the other axis (see Figure 1). Each box on the chart corresponded to a number to facilitate quantification.

Blank chart on a tablet screen showing emotions from angry to delighted and intensity from very strong to moderate.
Figure 1. Emotion chart that participants use to select emotions and intensity levels throughout an experience.

Participants described in their own words how they were feeling at distinct points throughout the session but would also make selections on the emotion chart. The goals for using the chart were to reduce moderator bias, standardize data collection, and take into account emotions along with traditional usability metrics. Our first study that incorporated self-reported emotions confirmed some of our suspicions: high usability scores were not always correlated with positive emotional ratings. Our first analysis told a richer story because we went beyond usability metrics (see Figure 2).

Graph comparing two experiences at each stage, with time on task and ease of use ratings.
Figure 2. Our first emotion chart output. Although the ease-of-use ratings were similar between the two experiences tested, the emotional journey revealed emotional differences at key points in the experiences.

Validating a Self-Report Method

While we remained confident that the self-report approach offered greater insight into user emotions, the big question remained: is this more or less effective than other emotion-capturing methods? As an agile and lean UX company, we were not interested in comparing self-reporting to methods that required major financial or time investments. We wanted to compare this method to existing products being marketed as leading emotion-capturing tools. Since time was of the essence, we were not interested in a longitudinal or quantitative study so we planned a user study with multiple rounds of testing that could be done quickly.

In our first round of validation testing, we showed emotional stimuli video to nine participants. We used the “How are you feeling?” prompt to ask them to report their emotions. We also asked them to self-report using the emotion chart. We sent session video recordings from multiple camera angles to emotion video analytics company Emotient. We also processed audio files through the Moodies Emotion Analytics service. After comparing the outputs, we found significant differences between what the analytics tools reported and what participants reported. We were surprised to find different emotions being reported between two camera angles from the same video clip. The video analysis was also inconclusive for participants who wore a hat or rested their head on their hand. Some participants had to be eliminated from the next round of testing because of these factors.

Second Round of Validation Testing

In our second round of validation testing, we asked five of the original nine participants to return one month after their initial sessions. We aggregated and randomized the outputs from the video analytics, voice analytics, self-reported verbal comments, and self-reported emotion chart data. Participants watched videos of themselves watching the emotional stimuli (from the first round of research). Videos were edited into short clips so that participants would not be overwhelmed with information and so they would be unable to hear their comments from the first round of research. We then presented participants with the randomized emotion data and asked them to indicate their level of agreement that the reported emotion represented the emotion they felt during the video clip from the first round of research (see Figure 3).

Example of Participant Data

Emotion Tracking Tool Response/Analysis Level of Agreement by Participant
(1=Low, 5=High
Facial Analysis
(Lab Camera)		 Disgust

Surprise

 1.7
Facial Analysis
(Lab Camera)		 Surprise

Neutral

Sadness

 1.7
Voice Analysis Hostile 1.7
Self Report

(Verbatim Response)

 Interested 3.3
Self Report

(Emotion Chart)

 Delighted 4.0

Once participants rated how accurately the various emotion data represented how they felt during their initial research session, we analyzed the data and averaged the ratings across all five participants. Every participant rated their self-report emotions (via verbal reports and the emotion chart) as more accurate than the emotions inferred by the analytic tools. The verbal self-reported emotions ranked slightly higher than the emotion chart (see Figure 3). Participants employed a variety of strategies for determining how they had been feeling during the first round of research, but most noted that they simply recalled how they had felt.

Average Accuracy Rating: Voice analysis 1.8, Facial analysis 1.9, Self-report verbatim 3.9, Self-report emotion chart 3.6
Figure 3. Average user ratings comparing the accuracy of Facial Analytics, Voice Analytics, Verbatim Self-Reporting, and Emotion Chart Self-Reporting.

While implicit approaches to interpreting emotions were perceived to be less accurate than self-report approaches, there are still challenges to giving participants specific descriptor words. In some cases, participants prefer descriptor words to help them describe how they feel, but in other cases, participants find words to be in the way. In some cases, participants suggested that emotions are so complex that for any given situation, they may feel multiple emotions in any given moment. In response to this, we have incorporated a sliding scale with a thumbs-down on one end of the spectrum, and a thumbs-up on the other (see Figure 4). This has allowed participants to indicate how positively or negatively they feel without having to commit to specific emotion labels.

Scale with thumbs-down on one end of the spectrum, and thumbs-up on the other.
Figure 4. Sliding scale that can be enacted without descriptor words which allows participants to respond in a more generic way.

Emotions are Complex

While the argument can be made that participants do not necessarily understand the complexities of their own emotions and cannot properly express them during a session, our research shows that users are confident when given some parameters for reporting emotion. The reality is that people make choices all the time based on how they interpret their emotions; whether their interpretations are “accurate” or not, they still profoundly affect their decisions.

There’s still a lot left for us to learn in the arena of methods for understanding emotions in UX. Though by no means perfect, the emotion chart has become the most agile, truest method we’ve found for understanding emotions. It enables research participants to quickly and accurately convey how they feel throughout a product experience, and does so in a way that is easy for researchers to quantify and communicate. The end result allows researchers and stakeholders to show a multifaceted emotional journey in a visual and meaningful way through self-reported emotional data.

And that makes us feel excited.

[bluebox]

Applications for Self-Report Emotions

  • Usability studies. Collect emotional data in order to analyze how it correlates to ease, satisfaction and likelihood of use.
  • Benchmark studies. Compare how emotions affect the experience over time.
  • Focus groups. Have participants record their responses independently, allowing the moderator to collect emotion data without group influence.
  • Out-of-box studies. Capture how emotions change during initial interactions with a product.
  • Diary and longitudinal studies. Have participants record their emotions over a longer period of time in order to paint a compelling picture of the entire emotional journey.

[/bluebox]

Scarcity: Focusing on Limitations to Explore Design Opportunities

December 7, 2015 No Comments

As UX professionals, one of the things that draws us to this work is a keen focus on people. To understand who they are and what makes them tick, we use a range of methods, including ethnography, interviews, and usability studies. But when we focus solely on people’s tasks and actions, we may be missing something. When designing for social impact spaces, whether the project is for government or for a nonprofit, the people we are designing for often face unique challenges and have lives that look remarkably different from our own. In my own work I have found considering the concept of scarcity to be helpful in order to see users’ needs in a different light. Looking at what is scarce, in terms of resources and support, can lead to new ways of thinking about users and designing for their needs.

What Is Scarcity?

The concept of scarcity comes from the field of behavioral economics and was developed by Senhdill Mullainathan and Eldar Shafir. Scarcity refers to how people behave when they feel they have less than they need. When we don’t have the money, time, or willpower we need, the state of scarcity captures our attention in a way that impacts how we act and behave.

Scarcity has several components. First, scarcity is involuntary. You have not chosen to be lacking in resources, such as money, time, or willpower. Second, scarcity consumes your attention, which diverts your focus away from other things. Third, scarcity has an upside: when resources are scarce, you work hard to maximize them and, as a result, develop unique expertise including innovative workarounds. Scarcity is a powerful concept because it cuts across many groups: busy people, lonely people, dieters, and more. For the purpose of this article, I will focus people who are facing financial scarcity—that is, the poor.

The Case of the Payday Loan

Let’s look at a common scenario facing someone who is poor and how the concept of scarcity can help us see the situation differently. Susan and her family get by, but often live paycheck to paycheck. Suddenly she has some unexpected bills. The car breaks down. One of her children needs an emergency visit to the doctor. She has to pay the bills but payday isn’t until next week. Susan takes out a short-term, or payday loan, to cover the costs. She is relieved. Later she realizes that she had forgotten to let her boss know she’d need a couple shifts off from her job at the restaurant to take care of her daughter. When she tells her boss, he sighs and says, “Why don’t you take the whole week off? I’ll put you back on the schedule next week.”

Now she’s going to be short on rent, and has to pay back the loan too. Susan doesn’t see any other options, so she takes out another loan to cover the difference. Soon the fees are piling up—even higher than the original bill—and she’s falling behind in payments and slipping deeper into debt. In this scenario, it would be easy to blame Susan, to think maybe she isn’t good at saving or planning or that she should have read the fine print about how the fees accumulate on the loan. But, considering the concept of scarcity lets us see this situation differently.

Tunneling, Bandwidth, and Slack

Susan’s experience demonstrates several aspects of scarcity. First, when resources are scarce, people tend to engage in tunneling. Similar to the concept of tunnel vision, tunneling allows us to pay close attention to the problem at hand, but it obscures our ability to notice other important things. Susan had to tunnel to figure out how to pay the emergency bills. But when she does, other details fall out of her frame: details such as how quickly the loan fees will pile up and how she will pay her other bills.

In their work, Mullainathan and Shafir conducted many research studies that show there is a decline in decision making that can be attributed to the scarcity of the resource rather than to a personal characteristic. In other words, engaging people’s minds in a scenario related to financial hardship causes them to perform poorly on intelligence or cognitive tests. They have less bandwidth for decision making. But as the authors so eloquently state, it is not that the poor have less bandwidth, it’s that all of us would have less bandwidth if we were poor.

When resources are scarce, when people don’t have a lot, they have little room for error. Susan was able to pay her bills until the unexpected happened. She and her family did not have much slack in their budget; she had no reserves to make the payment. Many people do not. There is no breathing room. When you live in a state of scarcity, small problems can escalate quickly.

In UX, there is a lot of talk about the importance of empathy in our practice. In this scenario, if we focus solely on Susan’s actions, behaviors, and tasks, we may miss the state of scarcity she is living in and how it impacts the choices she makes. We may miss the opportunity to design to acknowledge or alleviate scarcity.

Signing up for Healthcare as an Immigrant in the US

In the past year, I was part of a team conducting a study on immigrant populations signing up for health insurance through the Affordable Care Act, or Obamacare, as it is commonly called. When the Affordable Care Act first launched, signing up for healthcare was no easy task. The online process on Healthcare.gov was infamously difficult due to UX and technical problems.

Now imagine that you are an immigrant and new to the US. Perhaps the entire concept of insurance is new and you have limited English language skills. You are told that if you don’t sign up for health insurance you will be fined. These are the challenges that many people in our study were facing. Figure 1 shows a sign in a Community Health Center where Vietnamese patients go for help with insurance.

A picture of a sign in a health clinic in Vietnamese with the word “Obamacare” in English.
Figure 1. A sign posted in a Community Health Center in Vietnamese that provides assistance to patients signing up for health insurance.

Think about the complexity of insurance and terms like deductible, co-insurance, and co-pay. The concepts are difficult for many of us, but add health insurance literacy on top of cultural and linguistic literacies, and we can see how comprehension becomes even more challenging for immigrants.

In our study, one young woman revealed that she had thought the word “deductible” meant the amount you had to pay up front for insurance. Worried about her family and their health, she panicked. She went to her bank and withdrew $2,000 to pay what she though was an upfront cost. This represented all of the money she had in savings. She spoke of the stress of contemplating spending all of her savings for the next year of healthcare. She finally decided not to enroll in health insurance because of this misunderstanding.

Now think about a person supporting a family and living on limited means. The concept of scarcity helps to explain why comprehending the myriad information related to support services, such as healthcare, would be overwhelming. For people who are trying to enroll, the process itself is cognitively challenging: it’s a tax on their bandwidth, which is finite. This additional struggle has depleted their energy and ability to make other decisions, including ones that are financial, or even personal, such as an ability to be patient with their children.

Being Transit Dependent and an Upside to Scarcity

In another study, I looked at the experience of people who are transit dependent in the City of Seattle. Those that are transit dependent do not own a car and therefore rely on public transportation to get around. Not everyone who is transit dependent is poor; some rely on transit because they prefer to take the bus instead of driving in traffic. For some it is an expression of their environmental values. However, many of the participants in the study were poor. Some were homeless, some were in transitional housing, and others qualified for disability discounts due to health issues.

We saw plenty of examples of how scarcity played out for the poor in the transit dependent study and the bandwidth it required to figure out transit schedules and routes. We could see how the transit schedule dictated what jobs you could apply for and where you could live.

A screen capture of a video where a woman waits at a bus stop.
Figure 2. A screen of a participant’s video diary where she reflects on a long day of taking several buses across town to visit a particular store.

However, we also saw an upside to scarcity. The people in our study had developed deep expertise. All participants had a good understanding of how the transit system operated, but we noticed that the participants who were poor had developed a level of expertise that seemed to eclipse that of the others. They had extensive knowledge of routes, services, and vehicles. They knew which route was the best to take to get somewhere quickly, or which had a nicer view, or seemed safer at night.

The participants who were poor maximized the transit system in resourceful and unexpected ways. For example, people used their transit trips to structure their time. They would take a trip and bunch errands based on routes in order to maximize a free ride within the transfer window. Some people learned to exploit the weaknesses of the fare payment system to ride for free. We heard from homeless participants in the study on how riding the bus enabled a sense of refuge from the chaos of life on the street and provided an escape. On the bus, they could be like everyone else.

Implications for Scarcity for UX Design

Scarcity can, and does, affect us all in different ways. When designing for people living in poverty, we can use the concept of scarcity to inform research and design.

A holistic approach

As UX professionals working for a government and nonprofit, we tend to work for an organization with a specific goal or program that addresses one aspect of peoples’ lives. Although we work in silos, we know users don’t see the world in that way. We are already aware that we have to think more broadly about people and their needs that cut across silos and organizations.

When we introduce the concept of scarcity, thinking holistically becomes even more crucial. Each additional requirement or unique online system we introduce is an additional tax on bandwidth that is already stretched thin. Each new password users have to memorize, or each new form that has to be filled out in triplicate, places more of a burden on the people we are designing for. Bringing together systems and services in government is never easy, but the concept of scarcity makes the argument for doing so even more urgent.

Researching scarcity

When it comes to understanding the context of people living in scarcity, there is no substitute for deep qualitative work, such as ethnography, to understand the material conditions of people’s lives. Video diaries allow participants to tell their stories in their own words. Seeing scarcity in action helps us to appreciate how it plays out in people’s everyday lives.

When researching scarcity, consider the following research questions when looking for the ways people struggle with and overcome constraints in their lives:

  • Where do users struggling with scarcity make common mistakes?
  • How do expert users exploit their knowledge to bring about desired results or overcome a system’s limitations?
  • What strategies and tactics do users deploy when a system does not meet their needs?
  • What are the consequences when systems do not work in ways that match users’ needs?
  • How will changes to systems impact users’ abilities to achieve their goals?

In addition to qualitative research focused on understanding context, the impacts of scarcity can emerge during usability studies. When participants are dealing with scarcity, it is important to keep in mind that their performance may differ from that of others, especially when performing scenarios that are directly related to financial matters. This is important because the well-being of our participants should always be in the forefront of our minds and because scarcity impacts performance, including time on task and task completion.

Designing for scarcity

Design is an opportunity for us to alleviate scarcity. When information and systems are designed well and do not require additional bandwidth, we are reducing cognitive load. We are giving people back bandwidth they can spend on other things: perhaps more time for work, loved ones, or sleep. Or they may have less anxiety or worry.

Using plain language

The principles of plain language are especially crucial when designing for scarcity. Providing information that is written in accessible, easily understood language that removes jargon and emphasizes action is a way to help people who are overwhelmed. Content should be written in a way that makes consequences of actions very clear. It should stress the upfront and longer-term ramifications in ways that can make the decisions clearer. For example, in explaining the costs of a loan, research has found that showing a dollar amount ($400) instead of an interest rate (3%) is helpful in letting people see the real costs. For examples, see the ClearMark Awards from the Center for Plain Language.

Preventing and recovering from errors

When designing for people living with financial scarcity certain design heuristics become crucial. Error tolerance and error recovery become central because it’s easier to make mistakes or hasty decisions when you are in a state of scarcity. For example, the 1-click purchase on ecommerce web sites might be easier to use, but it also makes it easier to purchase by accident, so it may not work well for a government or nonprofit site.

Sending more reminders, charging fewer fees

Because research shows that late fees disproportionately impact the poor, instead of penalizing people for forgetting to pay their bills, consider how we can remind them in a timely way. By doing this we can design slack into systems for people who have none. For example, send drivers a text when their car’s registration will expire in two weeks and remind them how much it will cost if they pay now compared with after the deadline.

Designing for a desirable condition

When people start a new job many do not sign up for a retirement plan right away and therefore lose out on saving money and taking advantage of matching employer funds. Some organizations have started automatically enrolling employees in these plans and asking if they want to opt out. This technique could be used in designing for a desirable condition. Consider ways to provide an opt-in for the desirable or beneficial option that does not require additional action from users. Also, provide a way to opt out. This should be done carefully, but good UX research and design can help reduce the decision-making burden on users.

In addition, when people are filling out forms online, allow them to save their work and continue later rather than requiring them to finish all at one time. Sending a reminder to complete a form can be helpful as well. Ecommerce does a nice job of reminding you that you have an unpurchased item in a shopping cart. Why not do the same for a financial aid form or an application for food stamps?

Conclusion

The examples in this article have provided UX professionals with:

  • An overview of scarcity and why it matters for design
  • Stories where scarcity impacted the ways users seek information and use systems
  • Implications for designing and researching scarcity when working in social impact spaces

As we strive to design systems that help users accomplish their goals with government and nonprofit organizations, let us use the concept of scarcity to inspire us to look for ways where our designs can give back some bandwidth to users instead of requiring more of it.

[bluebox]

More reading

[/bluebox]

Finding Uses For New Technology: Moving with a Magic Thing

September 15, 2006 No Comments

Even  if  we  user  experience  professionals  would  like  to  think that  new  product  development  is  always  user-pushed,  we have  to  admit  that  technology  can  sometimes  be  the  main driver.  Some  of  the  greatest  innovations  have  been  technology-pushed.  For  example,  Gore-Tex,  the  basis  of  dozens  of  products, including fabric for outerwear and dental floss, was a technology-pushed innovation.

Another is the short text messaging system (SMS) used widely in mobile communication throughout the world. However, SMS markets might have been discovered sooner if qualitative user research using the methods such as those described in this article had been conducted.

The Method

When developing technology-pushed products, the company begins with a new proprietary technology and looks for an appropriate market in which to apply this technology. Qualitative user research can provide data that can help find these markets. We use a method called “moving with a magic thing” when our clients look for applications for new mobile technologies. The method was originally developed by Giulio Jacucci at the Helsinki University of Technology, but we have fine-tuned it and applied to fit our client projects.

“Moving with a magic thing” is a field method. Users are met in their environment and given a “black box” or a mock-up of a mobile device. They are told what functionality the device has and are asked to go about their life as they would normally do. However, they are also asked to think aloud and show what they would like the magic device to do for them as part of their daily activities. Every time the users come up with an idea to use “the magic thing,” the UX specialist takes a picture of them using the mock-up and asks a few follow-up questions about the discovered use case.

Since observing is time consuming and people can be uncomfortable when observed for a few days in a row, we can also use a “moving with a magic thing” photo diary as a method to collect more use scenarios. After the initial observation day where the users are being accompanied by a UX specialist, they seem to understand the idea behind the method quite well, based on our experience. They are given a digital camera for a week and asked to take pictures of the situations where they would use “the magic thing.” At the end of the week, users are interviewed based on the photos they have taken.

This method results in high-level “use scenarios” that do not come from brainstorming activities conducted in a meeting room but are based on user observations in real contexts. We have created a template to report the use scenarios discovered in the study (Table 1). Having a template is essential for consistency purposes, especially when conducting similar studies across countries and with several different UX teams.

table
Table 1. Template to report use scenarios resulting from the “moving with the magic thing” study.

The use scenarios are analyzed and the client gets insights into what kinds of users might need the new mobile product and for what purposes they might use the product. Also, the importance and frequency of the use scenarios can help choose appropriate markets for technology-pushed mobile products (see example in Table 2).

table
Table 2. An example of two use scenarios observed in a “moving with a magic thing” study. The mobile device in the study can be used to provide information about objects it is pointed at.

 The Theory behind the Method

The reporting template (Tables 1 and 2) that we use for documenting “moving with a magic thing” studies uses concepts from a theory explaining user experience. According to this model, user experience is defined as a motivated action in a certain context. Moreover, the user’s expectations and previous experiences influence his or her present experience.

By using the reporting template, the UX experts who conduct “moving with a magic thing” studies can report the motivation (why the user interacts with the technology and what she wants to be the result of interaction), the action (what she wants to do with the technology), and context (where, with whom, and when she is when interacting with the technology). Moreover, the experts include photos to help the reader empathize with the user and better understand the context in which the interaction occurs.

The “action” concept in the UX model can be explained on two levels: what and how. “What” is a higher level description of the user’s activity and “how” describes in detail the interaction with the user interface. With the “moving with a magic thing” method, we have not tried to study how the users would interact with the user interface. Rather, we have focused on understanding the action on a higher level. Interaction research, such as contextual inquiry, can answer the question “how” after the target group and main use scenarios have been selected—in other words, when the focus has been narrowed. Moreover, interactive prototypes can be built to test the “how” level.

However, a limitation of the “moving with a magic thing” method is that it does not reveal how the usage of the product might change with time (the time arrow in the UX model). As Jane Fulton Suri points out in her inspiring book Thoughtless Acts, people use products in unintended ways that reveal new needs. Giving high-fidelity prototypes to users to be used in their own environment for a longer time period is one method to study the time aspect and unintended uses of new products.

Another limitation of the “moving with a magic thing” method is that it does not allow user expectations to influence the user experience. If the technology under study is something very new, people might not have any expectations about it. Moreover, we do not reveal the manufacturer or the brand of the new technology, as that might affect the types of scenarios users generate. Therefore, we cannot use this method to study the expectations that knowledge of the brand might create.

chart
The time and context model of user experience.

Lessons Learned

The participant profile must be defined carefully before starting the “moving with a magic thing” study. The participants are asked to generate ideas for using new technology, so if you choose advanced users of technology who follow the latest technology trends, the scenarios they come up with may not reflect people’s “everyday” needs (at least not yet) but may be “cool.” The “cool factor” might make the product appealing to other user groups as well, even if the discovered scenarios are not high priority for them in practice. On the other hand, participants who are not early technology adopters (for example, stay-at-home moms), are more likely to produce “everyday” scenarios that would warrant frequent use of the product. Sometimes it might also be useful to study “extreme” users such as illiterate people in Africa. These users may come up with unexpected innovations that will benefit all.

After the study is conducted and documented, it is useful to work with the client to come up with the final conclusions and final presentation. A workshop where the UX experts and client representatives choose the most promising and feasible scenarios to develop further is very useful. This meeting can be followed up with storyboards based on selected scenarios. The client can use the “stylish” storyboards when selling the product to the decision makers, as the narratives and images from the reporting template are often not powerful enough to use in presentations.

Is the Method Reliable?

One might question the “moving with a magic thing” method because the users are highly involved in creating scenarios for a new technology. Can users really know what they need? However, taking the brainstorming into real contexts and involving users makes the method more reliable than if the users and/or designers were to sit around a table and generate ideas.

The user cannot always be the starting point for new products, but this method provides us with a way to involve the user in finding applications for technology-driven innovations. While other UX methods are needed during the concept development process, the “moving with a magic thing” method is certainly a good place to start.

Looking Closely at e-Learning: Vision Research Reveals Ways to Improve Children’s Experiences

March 15, 2011 No Comments

photo of children with a laptop

I’m fascinated by the educational opportunities e-learning is bringing into our homes and classrooms. I’m thrilled when I see how engaged and motivated my own five-year-old daughter is when she’s working on a computer designed for kids. But, as a parent, I’ve had my concerns, too. Will staring closely at the computer screen harm her eyes? Is it better for children to read on paper than on screen? E-learning is still relatively new; when I started asking these questions in 2009, there hadn’t yet been any direct studies on computer use and the impact on children’s vision.

Fortunately, I’m in a position to find out the answers. I specialize in the perceptual aspects of the computing experience, including hearing, touching, and seeing (not tasting or smelling—yet). As a senior human factors engineer in one of Intel’s user experience research groups, I conduct research on Intel® Learning Series (ILS) computers, which are designed especially for children. I was able to initiate a series of studies with Pacific University in Oregon to look at children’s performance and comfort when using computers. Together, we were on a mission to answer key research questions to improve the overall e-learning experience for children, starting with their visual experience.

Does Screen Viewing Impact Children’s Vision?

We began our research by identifying the visual demands placed on children while working on computers. We called on the experts and asked, “Does screen viewing impact children’s vision?” First on our list was Jeffrey Anshel, OD, president of the Ocular Nutrition Society and an authority on the topic. He answered, “The good news is that there isn’t any proof that computer use causes long-term eye health problems.”

Dr. Anshel went on to explain that the potential for visual strain from computer screen viewing is often a matter of short-term discomfort, not long-term detriment to eye health. “Of course, if a person focuses close up on anything—a computer screen, TV, even a book printed on paper—for 20-30 minutes or longer without taking a visual break, that person may experience some visual strain,” he added. “The symptoms could be eye discomfort, fatigue, blurred vision, and headaches. There’s even something called Computer Vision Syndrome (CVS), which is a compilation of symptoms that arise from extended viewing, technically when the sustained demands of the task exceed the abilities of the viewer.”

I discussed these issues with my research colleague, James Sheedy, OD, Ph.D., the director of the Vision Performance Institute at Pacific University and a leading specialist on vision and computer displays. He pointed out, “We saw a lot more visual strain symptoms with the old CRT displays. The CRTs flicker, contributing to eye strain, although now you can usually increase the refresh rate on those screens, so it’s less of an issue. The CRTs are also prone to reflections, which decrease contrast and legibility. Plus, the CRTs are bulky in size, so it’s harder to place them on a surface where they’ll allow for a safe viewing distance or provide the proper viewing angle. But the high-quality displays of the LCD screens don’t flicker at all, so they’re much easier on the eyes. They’re thin and easy to position, and they tend to have anti-glare screens, at least as an option.”

Most of the newer computer screens today, and all the screens used in my company’s classroom PCs, are LCDs. But I was determined to zero in on any possible factors contributing to children’s visual discomfort and strain, even with LCD screens.

photo of children with laptops

I turned to the research my team had previously conducted, in particular our observations that children use computers quite differently than adults. An adult will typically sit down at a computer and automatically adjust for less-than-ideal conditions (for example, raising the chair, brightening the screen a bit, or putting on reading glasses). A child, on the other hand, sits down at a computer and ignores any signs of discomfort or circumstances that could result in visual strain. As Dr. Sheedy pointed out, “Children are wonderfully adaptable. But unfortunately, they also tolerate poor viewing conditions like screen glare or inadequate lighting in the classroom. And they sure don’t care about proper posture when they’re using the computer!”

I agreed, “Not to mention they don’t think to rest their eyes like adults do. Or even blink!” “That’s right. And even if something is uncomfortable, most children will simply avoid doing it,” Dr. Sheedy said. “Besides, children aren’t aware of their own vision limitations, like an adult would be, whether it’s because of poor user conditions or their own vision impairment.”

I came away with an idea my team could pursue immediately—incorporating vision-screening software into computers. That way, teachers and parents could quickly test a child’s visual acuity and be able to determine whether the child’s environment needed to be adjusted or if the child had a vision impairment that needed correcting.

Studying Children and e-Reading

Based on these early conversations with vision experts, research questions were honed and a research plan on the effects of e-reading on children’s vision relative to the factors surrounding children’s all-around comfort when using computers—visual and otherwise—was developed. The team then embarked on a series of studies in collaboration with Dr. Sheedy’s team of vision experts at Pacific University, led by Shun-nan Yang, Ph.D. Our research looked at several aspects of children’s visual comfort and reading performance, including studies on the effects of font size and display quality, factoring in screen luminance and contrast. We also incorporated studies to determine whether frequency of breaks and reading posture interact with viewing distance and visual discomfort when e-reading on handheld devices.

Our first studies, on children’s comfort and performance while reading using LCDs compared to reading on paper, were designed specifically for second to eighth grade children. We recruited fifty children and screened our young participants for visual acuity and reading comprehension skills. We then carefully customized all study instructions and selected reading materials to be age appropriate. The computer and paper displays were set in a fixed position to standardize the visual angle of the display and to set the starting visual distance. Chair and table heights were adjustable and made ergonomically correct for each individual.

During the studies we measured reading performance (word recognition, speed, and comprehension) as well as viewing distances and visual comfort. Visual comfort was gauged using both a self-reporting rating scale and an objective electromyography (EMG), which measured the muscle “squint” activity around the eye.

We learned that the novice readers (the younger children) had consistently closer viewing distances, regardless of their visual acuity. Dr. Yang articulated our hypotheses, “These readers may be inherently poorer at visual encoding because they are just learning how to read, so they need a closer viewing distance regardless of font size or display quality. Their closer viewing distances may also be the result of the visuo-ocular response they habitually adopt; they lack the ability to strategically adjust their viewing distance in relation to font size and display condition.” We all agreed that more studies were needed to test these hypotheses.

With the experienced readers (the older children) in the group, only those with poor visual acuity adopted a shorter viewing distance. When the display quality was less than ideal, it made their viewing distances even shorter. Future studies would likely reveal additional visual factors surrounding text display that could result in a closer viewing distance. We did observe that the more experienced readers appeared to be able to adjust their viewing distances in relation to text and display changes, so they may benefit the most from display quality improvements. It became clear that correcting their vision impairments would allow them to adopt longer viewing distances, which would help alleviate the minimal, but notable, visual strain and discomfort they experienced.

Our findings highlight the interplay of individual visual capacity and display quality when determining developmental readers’ visual comfort and performance. After careful analysis, we were happy to report that children with good vision could read as well on LCD computer displays as on paper, without notable differences in viewing distances and the resultant visual discomfort.

I was relieved to learn that e-reading doesn’t appear to cause additional visual strain for children with adequate vision. Any visual discomfort that was reported was very minor. But there is always room for improvement. We took copious notes on factors to research further for the development of new screen technologies.

Exploring Other UX Factors

The outcomes of our first e-reading vision studies on children with 20/20 vision were interesting. We learned that when screen luminance and contrast are adjusted properly, children can comfortably view the screen from farther away. We had also learned about variable lighting and other conditions that make classrooms unique compared to home and office environments. This influenced the latest features being developed for our classroom PCs, like sensors that could automatically detect light levels surrounding the computer and adjust accordingly.

We continued to brainstorm new computer screen design specs and setting options based on our findings. The plan was to fast track one idea for our classmate PCs—creating software that would provide pop-up reminders every twenty minutes or so to remind kids to take visual breaks and to focus on something far away, ideally twenty feet away, for twenty seconds.

In addition, a longitudinal e-reading pilot program was in the works. Short-term usability results can be so different when children are involved. Children love the e-learning experience because of technology’s “wow” factor, so they may not report long-term discomfort or other effects. Therefore, we were preparing for a pilot study in the classroom, comparing reading on computers versus reading on paper. The goal was to follow the same students from second through sixth grade to gain a long-term perspective on the effects of e-reading on children.

photo of children with a tablet

Improving Children’s e-Learning Experiences

I’m confident that the growing—and inevitable—integration of e-learning into the classroom will be a positive, exciting development. We’re off to a good start with the vision and other research we’re conducting with our partners, but there are many more factors to be studied and improved upon. Research on the effects of children’s long-term computer use is still extremely limited; there remain key research questions that need to be answered by all of us.

Computer industry influencers and other companies leveraging UX designers and developers need to continue to set new standards for the industry. We must invest in e-reading studies, make recommendations, and initiate improvements. All of us have to be proactive about sharing our research findings. As user experience professionals, we need to step forward and do all we can to advocate for children and education.Rina Doherty 是 Intel 用户体验组的一位资深人因工程学工程师，她对儿童的视觉与计算机屏幕浏览进行开拓性的研究。作为用户体验研究人员和尽责的家长，她重视与儿童浏览计算机屏幕的时间增加相关的视觉舒适性问题，而将在线学习的好处放在次要地位。

文章全文为英文版上級人間工学エンジニアでIntelのユーザエクスペリエンスグループの一員であるリナ・ドーティ（Rina Doherty）が、子どもの視覚とコンピュータスクリーンの使用に関する先駆的調査を紹介している。 ユーザエクスペリエンスの研究者、そして問題に関心を寄せる親として、彼女は、子どものコンピュータモニターの使用増加に関連した視覚的な快適さの問題と、eラーニングのメリットとを比較検討している。

原文は英語だけになります

COMMUNICATION GAP: Designing an Interface for Chinese Migrant Workers

June 15, 2006 No Comments

While there are many problems to be solved in China, we chose to address the need for communication between migrant workers and their children left behind at home. Although these workers lack exposure to computer-based tools, they welcome changes to better their lives and thus, they make willing subjects. Their weaknesses—including a low rate of literacy—amplify their detachment from Western design issues and make them interesting subjects.

These workers’ migration has raised social concerns in China. There are 114 million Chinese migrating to urban centers to find work. Many of these migrant workers are parents who leave their children at home in the villages in the care of the children’s grandparents. From interviews, they confided that their main problem was parenting their children from afar. They believed that better means of communication with their children would help alleviate this problem.

As modernization of China proceeds, few efforts are specifically focused on the migrant worker population. The goal of this prototype is to improve the quality of communication between a migrant worker parent and his or her children while they are separated.

Background

During Phase 1 (April 2005–June 2005), we chose a scenario, interviewed stakeholders, and started to sketch a solution. During Phase 2 (June 2005–October 2005), we continued to identify specific design issues and started conducting user studies.

Previous Relevant Work

Chinese cultural traits that inform user interface design have been researched widely, and the number of studies grows every year as China gains in importance to Western technology companies. These studies, such as the ones documented in Geert Hofstede’s book Cultures and Organizations: Software of the Mind, are largely ethnographic, anthropological, or psychological. Less focus has been on user studies with interactive devices. The inaccessibility of the target users is most likely the reason for this.

As many parties are affected and interested in the phenomenon of urban migration (including local and national governments, local construction companies, and telephone companies, for example), there has been a great deal of sociological and economic research on the migrant worker population in China. These studies are largely aimed at understanding reasons for migration, consequences, economic factors, and policies that deal with this phenomenon. However, design considerations for communication between migrants and their families back home have not been studied from an interaction design perspective.

Research Findings

An initial interview was conducted with a migrant worker in her thirties who works as a domestic helper in a private home in Beijing. She has two children aged eleven and fourteen, whom she has left back in her village with her in-laws. Her husband is also a migrant worker in Beijing. He lives and works on a construction site.

Remote Parenting: Our interviewee said she spent 90 percent of her phone conversations with her children and only 10 percent with her parents. She made it clear that parenting her children was her primary concern. Therefore, in villages, the primary users of our communication system will be children and young adults.

Synchronization: To contact her children, our interviewee needs to call one day ahead of the regular market day and ask the shopkeeper in charge of the public phones to inform the children to wait near the public phones at a certain hour on the market day when she plans to call back. Unpredictable schedules often reduce communication between parents and children to one call per month. Minimal parental presence in the children’s lives as a result of such infrequent communication makes parenting difficult.

Literacy: Migrant workers have varying levels of literacy. They are not always familiar with Pinyin, the Romanization of the Chinese script, which is generally used for input on Chinese desktop computers.

Familiarity of Surroundings: Live-in housekeepers rarely explore the area surrounding their workplace. However, they do know where the local food market is.

Surveys indicate that China has 39,000 towns in total. Each town is frequented by inhabitants of a dozen surrounding villages for services such as its market, its public telephones, and such. We estimate that each town’s public telephone serves as the rendezvous point of communication for the children of about one hundred families. We estimate about 500 parents share the public phones located near local produce markets.

Challenge

  • Allows for asynchronous communication.
  • Is affordable by migrant workers given their economic conditions.
  • Is usable given migrant workers’ various levels of literacy and lack of exposure to technology.
  • Is accessible for migrant workers unfamiliar with their surroundings.
  • Scales up to the sub-population of Chinese migrant workers and to China’s vast geography.

Solution

The most helpful research conducted took advantage of our geographical location. The design process was informed by personal experiences of the group, two stakeholder interviews, and focused user studies.

As the research team is based in Beijing, shared personal experiences from all the researchers helped to increase the general collective knowledge on the migrant worker phenomenon, lifestyles of housekeepers in urban centers, Chinese telecommunication standards, rural phone use, and so on. As it is difficult to gain much of this information during the course of rapid prototyping (since villages can take days to reach), our collective experience proved valuable.

In addition to the interview with a housekeeper, a field visit was paid to an elementary school for migrant workers’ children on the outskirts of Beijing. The school’s dean was interviewed at length. After the interviews were performed, several designs for kiosks were proposed, and focused user studies were performed with female migrant workers who work as custodial staff in the research facility.

storyboard explaining how to use a telecommunications device

Design and Resulting Design Decisions

In order to make communication more accessible and affordable to migrant workers, we decided to design a system of public, shared voice mail kiosks at markets in towns and cities. These kiosks let parents leave messages for their children, and vice versa, in an asynchronous manner. We also augmented the voice mail with videos so as to enrich the communication between parents and children, thus increasing parental presence in villages where the children reside. With video, parents can also visually inspect their children and perhaps even their school work, while—we hope—feeling better connected.

By locating the kiosks in local food markets, the system can piggyback onto public newspaper reading displays, already prevalent in China. These displays are well-understood by the population as the source for localized, public information. Rural towns also use the same newspaper display system.

A parent wishing to communicate with his or her children must initiate the first contact by using our system to call home and leave a voice-mail message. To do this, the parent enters the public phone number of the home village (or of the nearest town with public phones) as well as his or her full name. This combination identifies the recipient(s) of the first message, namely the parent’s family at home, and serves as the rendezvous point of communication between the two parties for all future messages.

The combination of the home public telephone number and the parent’s full name grants access to the voice mailbox of that family. No user name or password is required. Such departure from the conventional account-access paradigm sacrifices privacy for simplicity. This design choice is justified by our observations that the Chinese are accustomed to trading privacy for other benefits, such as cost or simplicity of use.

Compared to entering the phone number, entering a Chinese name is a more difficult design challenge. Pinyin is not widely recognized by migrant workers, so Pinyin-based input methods that are otherwise used pervasively on desktop computers cannot be used in this case. We turned to hand-writing recognition as the next most promising technology. Our user studies indicated that a tablet PC interface might be suitable for Chinese name input.

Management of voice mailboxes in our system is simplified by not providing the feature for deleting messages. Instead, each message is stored for a fixed time interval. This design choice also prevents one’s messages from being deleted by someone else. This is crucial for such a system as ours in which accounts are not protected by passwords.

The most unique portions of this work are in its context: the evaluation of technology in the context of this specific user segment, the social context of the family and urban settings, and the leveraging of Chinese cultural cues. The usability studies validated some claims; our next steps included participatory design.

series of photographs doumenting stages of the test
Entrance to a typical produce market where migrants use the telephone. 2. A typical public newspaper display where various people come for their news. 3. On task A1, users hesitated little when asked to write their name and hometown with a stylus on the standard tablet PC interface. 4. On Task B1, fourteen of twenty users could not easily find the location of their hometown on a three by four foot map of China. A list was preferred.

User Studies

Three small usability tests were performed on each of the twenty participants to validate assumptions about literacy, readability, and input. All participants were female migrant workers who held housekeeping jobs. The participants were not employed in a private residence (as our intended users are), but in a place of business. One factor that may have affected the results was that all the participants

were accustomed to working among computers and other technology devices. They may not have been as intimidated by technology as a domestic worker in a residential scenario would be. None, however, had any experience using a tablet-style PC and stylus.

The first task, A1, was intended to see how a stylus and tablet could best support Chinese character input for this user segment (see illustrations). There is a common Chinese practice of writing ideographs in the air with a fingertip. This technique is used when describing characters to others or remembering the number of strokes in an ideograph. We initially explored the idea of letting users write their names with their fingertips in the air. The resulting strokes would be analyzed by a camera or transferred onto a tablet. But as there was a question about technical feasibility, user study B1 was administered with users entering characters with a stylus. For this task, a commercial tablet PC was used. Participants were asked to write their name in Chinese characters using the stylus. We predicted that participants would be able to enter Chinese characters on the electronic system easily after some initial hesitation. The study revealed that most participants had no problems using the stylus.

Another point in administering task A1 is related to the fact that all the participants were not hesitant to use the device. We acknowledge that this could be because a person who is seen as professionally superior to them was requesting that they perform this task.

We also acknowledge that, in our test, character recognition proved to be less than 50 percent accurate. This may have an effect on future designs.

The next issue was for a user to glance at the display to check the kiosk for new messages. One design displayed the names of hometowns that have recently sent messages atop the kiosk. Another design displayed a map of China with geographical indicators of the origins of new messages. Two tests were performed to verify if users could identify the town nearest to their hometown that had a public phone. In the first task, B1, participants were asked to identify the location of their village on a map of China. In the second task, B2, participants were asked to pick out their hometown on a list of towns. Observations from administering task B1 concluded that fourteen of twenty subjects did not easily find their hometown on a map of China. They relied heavily on text descriptors on the map. However, on task B2, all users were able to quickly find their hometown on a list. Therefore, our design eschews spatial knowledge of hometowns and uses a list format instead.

illustrations of these phone in context
1. A standalone version of the kiosk. The list of rural towns at night,grouped by province, lets users quickly glance at the kiosk to see whether or not they have new messages. 2. Close-up sketch of kiosk with stylus. 3. The kiosk sketched into a typical public newspaper display. Smaller versions of these displays are commonly located near markets.

Next Steps

The immediate next step for this study is to create and test a high-fidelity prototype. First, a kiosk would be installed at a market. Replies to user messages would be stock messages from the system. This would simplify implementation and focus on usability issues. Later, rural kiosks would be installed and the interface would be tested with children.

As the original aim was to uncover usability and design issues by designing for a real scenario, we present issues that will be studied in detail as we move forward with the project.

Issues for Further Research

Scale: A huge population with relatively small number of unique names;extensive settlement networks ranging from very poor to very rich; many local dialects.

Chinese character input: What is the best interface for Chinese characterinput and error correction?

Shared access: How do Chinese privacy and sharing norms apply to communal devices? We have observed that Chinese people have fewer privacy concerns than Westerners. How can this idea of privacy inform design?

Conceptual understanding of data: Can we design interfaces for users toconceptually understand virtual messages? Concepts would include performing operations (save, delete), navigating, keeping mental lists, and understanding the implications.

Designing for Chinese novices: How is designing for Chinese novices different from designing for novices in general? How can we communicate the usefulness of a product before it is actually used?

Literacy: How do designs for character-only readers differ from onesdesigned for users who know Pinyin or even for those who are illiterate? Does the process of designing icons for Chinese culture differ from designing icons for a Western audience since the Chinese lan-guage is derived from pictograms?

Socioeconomic extremes: How should designs for the same culture differ bysocioeconomic level? What are the personas for these levels?

Social hierarchy: What is the best way to design user studies and interfaces without social status affecting outcomes?

In our study, one of the two administrators was a foreigner, and both were full-time employees. The subjects seemed to cooperate without reluctance. We attribute this behavior to the social hierarchy inherent in the Chinese culture: one always submits to the command of one’s superior. One helpful measure was to hold conversation and interviews in a private setting (such as a home or empty hallway). For testing on the “high-tech” tablet PC, it helped to keep explanation of the device (and its implied monetary worth) to a minimum, instructing users to “just write on it as they would a piece of paper.” Participants were not paid or given rewards for their time, and there didn’t seem to be an expectation for such.

There are many possible avenues of exploration related to designing for emerging markets, specifically in China. This paper is part of a larger effort to gain practical and specific insight into the usability of various products in China. The design and implementation of this project has already, and will continue to, uncover specific interaction design experiments that can be carried out with Chinese participants.