Skip to content Skip to footer

User Experience

The Magazine of the UXPA

User Experience

The Magazine of the UXPA

Search

Search Results for: interaction

Human-to-Human Interaction Style in Voice User Interface Design

November 23, 2019 No Comments

Advancements in natural language processing, voice recognition technology, and speech synthesis allow voice-enabled devices to mimic human-to-human interactions fairly well. The levels of capabilities that devices and machines have to simulate human voices and generate natural(-like) language in a conversation vary across platforms, and since it is a relatively new technological innovation, users often do not have consistent expectations of their conversation with a conversational user interface (CUI). These inconsistent expectations are often exacerbated by the differences between verbal and written language when the CUI modality is voice; this is a subset of conversational UIs called voice user interfaces (VUIs; often described as “voice-enabled” when embedded into a device).

This can lead to unpredictable user behavior: when the user does not know to act, we cannot predict how the user will act. This article attempts to mitigate some of this uncertainty by outlining a few general guidelines for designers to keep in mind when working with VUIs.

Natural Language Input and Output

There are two components of any conversational user interface (CUI)—in any modality, including voice—that determine the user experience:

  • User input: How well can the CUI interpret what the user is saying?
  • CUI output: How accurately is the CUI responding to the user, and how human-like is that response?

The natural language understanding (NLU) process allows the CUI to use regular, human language as its input, rather than a predetermined list of commands or utterances with an inviolable word order. In turn, a CUI responds with natural(-like) language using the natural language generation (NLG) process, enabling back-and-forth interactions that can mimic, as much as is programmatically possible, a conversation between two humans.

When a CUI is voice-enabled—which is to say, when the CUI is a VUI—whether or not the speech synthesis engine is able to successfully mimic the appropriate pauses, intonations, and inflections of human speech is another aspect of the NLG process that layers into an appropriately human-like output. For a VUI, the generated natural language output alone is not enough to create a “human-like” experience; a human-like voice needs to accompany the output in order for a VUI to achieve a human-to-human-like feel.

Domain Boundaries

The actual domain that a CUI covers is not as important as whether or not the user understands the boundaries of those domains, and, relatedly, the functionalities a CUI can perform. The natural language understanding and generation processes of a CUI are by necessity domain-specific, but the subconscious inferences that a user makes are based on how these components behave and are, if not domain-agnostic, at least domain-neutral.

You can reasonably—if subconsciously—assume that a VUI that can tell you the weather forecast for Minneapolis can also tell you the weather forecast for Sacramento or Tokyo; however, it is much less reasonable to assume that a VUI that has just told you Minneapolis’ five-day forecast is also capable of telling you how the New York Stock Exchange is doing, even though stocks also use “forecasting.” On the other hand, if a VUI allows you to search for an item and place it in your virtual shopping cart, then it is reasonable to assume that you could also ask for the price of that item, or for information on other items in your cart, or check out and buy the item in question.

Discoverability, or the user’s potential to uncover what functionalities a CUI offers, is more difficult in a VUI than in a chatbot or other type of text-based CUI. There are contextual clues within a text-based CUI that allow a user to determine what a CUI can or cannot do; often chatbots will provide guidelines or suggestions in a greeting message in order to enhance the likelihood that the user will be able to take advantage of all of its domain knowledge. VUIs, however, cannot offer visual cues without additional hardware, and so often must rely more heavily on managing the user’s expectations within the conversation and providing contextually-appropriate responses to out-of-scope inputs.

Creating a Human-to-Human Experience

The technical limitations a designer needs to keep in mind when creating a VUI experience center around how the user is expected to interact with the VUI. Since spoken language is not the same as written language, the natural language understanding process will need to account for a wide variety of vocabulary and syntax options.

The user will expect the VUI to understand the following inputs as identical:

  • “Hey can you uh set an alarm for, an alarm for uh six six thirty tomorrow ay em?”
  • “Can you set an alarm for six thirty tomorrow morning, please?”
  • “Set alarm, six thirty tomorrow morning.”

A designer also needs to consider how the user might behave while the VUI is answering the user. Does the VUI convey enough necessary information at the beginning of the response that the user is unlikely to interrupt it? Are there technical limitations that determine whether or not the user can interrupt the VUI? Shorter VUI outputs mean that, if the VUI cannot be interrupted, there is less user frustration if the NLU misinterprets a user input and triggers a mismatching response.

Of course, if the VUI uses a particular word or phrase in an output, it must accept that as an input. The VUI should never say something it does not “understand.”

Understanding User Expectations

The human aspect of a voice-enabled smart device is a self-sustaining premise. People assume their conversational partner understands them in a particular way based on who (or what) said conversational partner is; when people treat computers like humans, the interaction design must account for that. The more human-like the device, the more the user will treat the device like a human; the less human-like its responses are, or the more robotic its synthesized voice sounds, the more the user will treat the device like a computer.

This subconscious assessment of a device’s “humanity” extends to its domain expertise and reasonable expectations of its capabilities, and there are pros and cons to creating a VUI that users subconsciously consider “more human.” If a VUI’s responses make the user feel like their interaction is ersatz, that user will begin to default toward simpler vocabulary and syntax, reverting to commands instead of questions, and limit themselves to domains in which they know the VUI is competent.

This means that users are less likely to encounter out-of-domain errors. However, if users begin to feel too comfortable while conversing with the VUI and begin to forget or at least get comfortable with the fact that they are interacting with a computer, then they may include more complex syntax, wider vocabulary choices, patterns that are found in verbal language but not written language, or content that requires extralinguistic contextual cues.

This extra trust in the VUI’s competence comes with a price. However, if the natural language understanding process cannot keep up with the user’s less formal language or if the VUI’s knowledge is too narrow to include adjacent domains that a user might reasonably expect to touch upon in a conversation, the user is essentially unable to recognize the VUI’s boundaries. When that happens, user expectations rise and the VUI is unable to respond appropriately; this is interpreted by the user as VUI incompetence rather than the fact that the user has linguistically “stepped” out of bounds.

Personification: Managing Subconscious Expectations

One way in which designers can balance user expectations with natural language generation and speech synthesis capabilities is to choose a VUI name that accurately reflects how robust the VUI’s NLU and domain expertise is. Whether the VUI has a human or human-like name or a title/descriptor in lieu of such can affect how the user interacts with the VUI: what the user says for input, what the user expects the bot to be able to do, and whether the user feels that the interaction is human-to-human.

Take for example two well-known voice-enabled smart devices with some overlapping domains: Amazon’s Echo (Alexa) and Alphabet’s Google Home lineup. Figure 1 shows an Alexa device; Figure 2 shows a Google Home device, part of the Google Connected Home roster.

Image of an Amazon Echo device.

Figure 1. Amazon’s Echo answers to “Alexa” and uses first-person pronouns. (Credit: Piyush Maru)

Image of an activated Google Home Mini device.

Figure 2. When listening to user input, the lights on the Google Home Mini light up. (Credit: Andrea Marchitelli)

As “Alexa” is a human name, using it as a wake word (the set of syllables in a name or phrase that a VUI actively “listens” for and allows it to begin an interaction) encourages the subconscious impression of a human-to-human interaction. When the VUI fails to perform a task correctly or returns an incorrect answer, the user may be evaluating the interaction at a more human-to-human level.

Google Home, on the other hand, is triggered by “Hey Google,” and there is no name assigned to the VUI persona. As the user cannot engage the VUI verbally without using that wake word, it serves as an intentional reminder that the interactions are human-to-computer, realigning user expectations.

While both Amazon Alexa and Google Home undeniably have personalities—in casual conversation, both are referred to with the pronoun “she,” and when an error occurs we use the human-like terms “she made a mistake” or “that was stupid” rather than something like “there was a backend process that triggered incorrectly”—Google Home’s lack of a single, unifying name disperses some of that personification and allows the system to settle into a human-to-computer interaction role with its user.

Evolution of Human-Like Features Within A VUI: A Case Study of Radar Pace

Three years ago, I worked on Oakley’s Radar Pace, voice-enabled smart glasses with an interactive coaching system for running and cycling (see Figure 3). The “coach” allowed users to request information verbally (user-initiated dialogue) or to receive verbal updates from the system when it volunteered time-sensitive information relevant to the user (system-initiated output). (For more information on Radar Pace, see Danielescu and Christian’s “A Bot is Not A Polyglot: Designing Personalities for Multilingual Conversational Agents.”)Image of Radar Pace sunglasses that have earbuds connected to the frames.

Figure 3. Oakley’s Radar Pace sunglasses have no visual interface, but they double as fashionable athletic eyewear. (Credit: Gwen Christian)

As an example, the user might be able to say something like, “how much farther should I run?” or “how much longer?” at any point, and the Radar Pace coach would reply with the correct response. However, the coach would also volunteer information like “you’re halfway done” or “one mile to go,” which the user might not know or be able to ask for given the activities the user was undertaking when interacting with the coach. (Being too winded to ask the coach a question was a fairly common problem among test subjects.)

The System Seemed Smart

Test feedback indicated that this was a feature that users appreciated: The system was delivering the content the users wanted before the users even knew they wanted it. This system-initiated output became a key component in making Radar Pace interactions appear human-like to the user; determining what information is relevant and then passing it along is a human-like trait.

The system-initiated output was also an opportunity to train the user in the available domains and vocabulary. The users had no understanding of the VUI domain boundaries, and the topics that Radar Pace chose to bring up in its commentary allowed the user to understand what functionalities they could access.

This did not preclude user exploration, but its guided learning provided structure for users to expand upon within that exploration, whether or not they were consciously aware of it. If Radar Pace said “your stride rate is too long,” the user knew they could say “what’s my stride rate?” or “how is my stride rate?” The more often the user interacted with Radar Pace, the more knowledge they gained about the domains and vocabulary that were available to them.

User Reactions to Human-Like Output

Users personified the Radar Pace coach and felt as though they were having a human-to-human interaction with it. Evidence for this came from user feedback to the workout compliance summary provided at the end of the coaching session when—again, a system-initiated rather than user-initiated communication—if the coach said anything about the user not completing part of a workout, the user said they felt “judged” and that the coach was “disappointed” in them. Disappointment and judgement are human traits rather than electronic ones; if you say “my sunglasses are disappointed in me,” it does not have quite the same level of melodrama.

Conclusion

There are five major takeaways for designers working with Voice User Interfaces:

  • It is not just what you say, it is how you say it. Speech synthesis can matter just as much as natural language generation.
  • Users need to understand what a voice-enabled device can and cannot do, and what it can and cannot understand, in order to use the device.
  • If you have done a good enough job with your voice user interface that users subconsciously think they are in a human-to-human interaction, they will start to act like it—so make sure you have accounted for that.
  • Voice user interface names can affect user perception.
  • Human-like interactions can have positive and negative consequences on user feedback.

Touchless Interaction: Communication with Speech and Gesture

February 22, 2014 No Comments

Humans have many ways to communicate—speech, gestures, direct touch—and as a result, communication between humans happens seamlessly…most of the time. But communication between humans and machines requires more formality. Most current technology systems rely on touch: we press buttons, type on keyboards, and swipe the screens of mobile devices. However, new systems are adding touchless control options to our vocabulary, enabling us to communicate with machines through speech and gesture.

Although gesture and sound-based input methods might seem more like a high-tech game than work, they are being taken seriously in industrial settings. These systems include touchless sensors that are useful both in simple forms, such as water faucets that turn on when we wave our hands under them, and in complex forms, such as data storage applications that use face recognition, iris detection, and voice-tracking for security and safety. An example from our daily lives is Google’s voice-based search application; if our hands are full, we can ask our phone a question and the application replies with a result. Although there are many examples of touchless controls in our daily lives, two examples of the importance of this growing market are industrial automation and healthcare.

Industrial Automation

An industrial plant can be controlled two ways. One is by HMI (Human Machine Interface) input devices that are placed near individual machines. A second is a Supervisory Control and Data Acquisition (SCADA) system in which the complete plant can be configured and controlled from a control room. Experts using these systems can give a single command to multiple devices or multiple commands to a single piece of equipment. Centralizing the control reduces the cost of production and improves unit quality and employee safety, especially when the shop floor is hazardous and time-to-market is important. The idea of the fully automated factory is not new; this concept has been around for more than thirty years. A New York Times article in 1981, for example, celebrated the “Manless Factory” as a new trend in Japan.

However, if interaction technologies are overly obtrusive or constraining, then the user’s experience with this synthetic automation world will be severely degraded. If the interaction draws attention to the technology, rather than the task, or imposes a high cognitive load on the user, it becomes a burden and obstacle to the process. Sometimes traditional mouse/keyboard-oriented graphical interfaces are not well suited to large, complex tasks. As a result, speech and gesture-based inputs are making industrial plant and process automation faster and more efficient by providing natural, efficient, powerful, and flexible interactions. Human gestures and languages are natural and flexible and may often be efficient and powerful, especially as compared with alternative interaction modes.

Robotic Systems and Simulation Environments

At a Volkswagen factory in Wolfsburg, Germany, many robotic hands, conveyor belts, and controls move simultaneously, each placing modules and completing their assigned tasks. Though the work is monotonous, automated processes do these jobs accurately and on time. Although engineers maintain machinery on the floor and manage hardware malfunctions, the close proximity required for touch-based communication is not always possible, and it can be difficult, costly, and time-consuming to rely on having an engineer go to a particular place and provide instructions. Sometimes this challenge is met by using batch processing techniques, but those solutions are specific to each application and tend to increase plant configuration costs. Imagine how much more efficient this factory could be if it was possible to manage it through touchless interactions.

One example of touchless automation can be seen at the Forth Valley Royal Hospital in Larbert, Scotland, where three robots handle a complete pharmaceutical process with touchless features. Drugs delivered to the hospital are tipped into a giant hopper and a conveyer belt moves them to a machine that reads the barcodes by using image processing techniques. The robots then stack the drugs on shelves—not in alphabetical order, but using a system that makes the best use of space—placing the most-frequently used packs for easy access.

The system continuously checks stock availability, and requests for medicines are instantly sent to robots that select and dispatch the drugs. A tablet computer has replaced the pharmacists’ prescription pads, and a color-coded screen on every ward tells medical staff exactly what stage each prescription has reached. Forth Valley Royal’s associate director of nursing, Helen Paterson, confirms that the paperless system has freed up nursing time, and hospital managers said that the £400,000 automated pharmacy has saved £700,000 from the hospital’s drug bill. The hospital is also implementing a robotic porter system where fleets of robot workers carry clinical waste, deliver food, clean the operation theatres, and dispense drugs by recognizing speech commands. Staff use handheld PDAs to call robots—which also respond to voice commands—to move meal trays, linen, or other supplies. The robot comes up in a service lift by itself, picks up the item, and returns to the lift by following a system of pre-programmed routes that use laser beams to tell the robot where it is and where it needs to go. Computers onboard the robots tell doors to open and sensors instruct the robots to stop if anything, or anyone, is in the way.

BMW is testing new robotic systems to work with human factory workers. According to a report in the MIT Technology Review, “BMW is testing even more sophisticated final assembly robots that are mobile and capable of collaborating directly with human colleagues. These robots, which should be introduced in the next few years, could conceivably hand their human colleague a wrench when he or she needs it.” This interaction could be prompted through a combination of speech recognition, image processing, and pattern recognition.

Ford Europe is also working on a fully automated plant that they call a Virtual Factory. It will be managed by a gesture-based system and augmented reality. According to Ford, “Virtual factories will enable Ford to preview and optimize the assembly of future models at any of our plants, anywhere in the world. With the advanced simulations and virtual environments we already have at our disposal, we believe this is something Ford can achieve in the very near future.”

Healthcare Automation

Healthcare offers another example of human-machine collaboration. Touch-based interaction methods are designed for people who can move physically to give input to a system, but patients may not have the mobility to interact with a touch-based system. Imagine a hospital with an automated patient-monitoring system that allows patients to communicate through speech or gesture to get the immediate attention of the nursing staff, or even robotic assistance. A CCTV camera and a wireless microphone could control these advanced automated monitoring systems. Even if patients can’t move from bed or chair, they can give instructions by voice or gesture movement to communicate their needs.

Speech and gesture can also be part of expert healthcare systems, such as diagnostic processes or medical instruments. Surgeons, for example, don’t like to touch non-sterile keyboards in the middle of surgery for sanitary and efficiency reasons. Enter the researcher willing to try something different.

Using Gestures

GestSure Technology, a Seattle-based start-up firm, uses Microsoft Kinect (most popular for its XBOX 360 gaming console) as its backend to allow surgeons to access MRI (Magnetic Resonance Imaging) and CT (Computed Tomography) scans during surgery without touching a keyboard or mouse. When activated, it follows hand movements by using three sensors to do depth analysis, and by doing so, can understand a human’s position in a room and a particular body part’s movement. As stated in an article in the Massachusetts Institute of Technology (MIT) Technical Review “Kinect hardware’s ability to perceive depth makes it easier to recognize and track objects, something very challenging using an ordinary camera.” The article claims that the gesture accuracy is better than voice recognition, which can only reach 95-98 percent accuracy (meaning it won’t work one time in fifty).

Another startup company, Jintronix, has created a Kinect-based application that guides someone recovering from a stroke through physical rehabilitation exercises. The system monitors their progress and supplies real-time guidance.

Interaction Issues

Although these systems offer a lot of promise, they can also be a challenge for people to interact with smoothly. In our work, we have observed usability problems with touchless interaction styles.

  • In the Valley Royal Hospital pharmaceutical department example, staff were concerned that the constant whirring of the robots and conveyor belts would make the pharmacy too noisy. In fact, it’s quieter than most hospital pharmacies where phones are constantly ringing. There is one problem, though: the robots can only handle small square or rectangular boxes. Still, pharmaceutical companies are already altering their packaging so that it’s suitable for a future where robotic pharmacies are the norm.
  • Touchless system users need to provide instructions in a specific pattern in order for the system to understand their commands. Most users need time-consuming training and practice to master this process.
  • The user’s commands can be ambiguous or misunderstood when multiple machines are waiting for human input. These systems need to provide a good way to indicate which particular machine is being addressed.

Building Intelligence and Standards

Gesture recognition systems depend on image processing algorithms and pattern recognition. In artificial intelligence, machines need to be programmed so that they understand a wide range of different gestures given by users. Researchers are trying to create algorithms that can learn the signal language from humans in runtime—using visual sensors or cameras, video streaming, etc.—and react accordingly, making the systems more accurate in less time. This is called “the building of intelligence.”

The main problem with speech recognition is regionalism. For example, in different countries the English language has different tones and pronunciation styles. In a normal sound recognition system, a voice or sound signal is compared with existing similar signals. Technically, this set of sound signals is known as signal datasets. To understand the different type of accents, different data sets need to be recognized by the application for it to react properly.

The scenario is different for gesture recognition, in which a common pattern is essential. Suppose a team is developing software where an application can be stopped by showing one hand with one open palm. On the other side of the world another group of developers is creating an application where two hands with open palms need to be shown for the same purpose. This is confusing, but can be solved by using a single pattern.

Future Challenges

Touchless communication is growing in maturity and spreading beyond entertainment into critical engineering fields; this trend will only continue in the future. But to be used in demanding fields such as factory automation or healthcare, human-machine communication must be more accurate and easier to use than current standards. Not only do the system themselves need to be smarter, but a common standard for gestures will be critical to the real people who must work with these systems. If this happens, it is possible to imagine a future in which touchless communication can be a widely used medium of communication between human and machine.采用非接触式交互的系统在娱乐业已司空见惯。 然而，要在工厂自动化或医疗保健等要求更严苛的领域得到应用，非接触式沟通就必须在准确度和便利性方面超越当前标准。 不仅仅是系统本身需要提高智能水平，手势也应当有统一的标准，这对于实际使用这些系统的人士而言至关重要。 只有具备上述条件，我们才有望在未来广泛采用非接触式沟通。

文章全文为英文版비접촉식 인터랙션 시스템은 엔터테인먼트에 흔합니다. 그러나 공장자동화나 의료와 같이 요구가 많은 분야에 사용되기 위해서는, 비접촉식 커뮤니케이션은 현재의 기준보다 더 정확하고 사용하기 더 쉬워야 합니다. 시스템 자체가 더 스마트해져야 할 뿐만 아니라 제스처의 공통 기준은 이 시스템으로 작업해야 하는 현실의 사람들에게 대단히 중요할 것입니다. 그 다음에야 비접촉식 커뮤니케이션이 널리 사용될 수 있는 미래를 상상할 수 있습니다.

전체 기사는 영어로만 제공됩니다.Sistemas com interação sem contato físico são comuns no mundo do entretenimento. Entretanto, para ser utilizada em campos exigentes tais como automação de fábrica ou saúde, a comunicação sem contato físico deve ser mais precisa e fácil de usar do que os atuais padrões. Não só os sistemas por si precisam ser mais inteligentes, mas será também crucial um padrão comum de gestos para as pessoas que devem trabalhar com esses sistemas. Só então será possível imaginar um futuro no qual a comunicação sem contato físico possa ser amplamente utilizada.

O artigo completo está disponível somente em inglês.タッチレス・インタラクション・システムはエンターテインメントの分野で広く使われている。しかし工場の自動化やヘルスケアといった厳しい職場環境の分野で使用されるためには、タッチレス・コミュニケーションが現行の基準よりもさらに正確で使い易くなければならない。システムそのものがより洗練される必要があるだけでなく、これらのシステムを使って実際に仕事をする人々にとって、ジェスチャーに関する共通の基準の設定が欠かせない。それが達成されて初めて、タッチレス・コミュニケーションが広く使用される未来が見えてくる。

原文は英語だけになりますLos sistemas con interacción sin contacto son comunes en el campo del entretenimiento. Sin embargo, para utilizarse en campos exigentes, como la automatización de fábricas o la atención médica, la comunicación sin contacto debe ser más precisa y fácil de usar que los estándares actuales. No solo deben ser más inteligentes los sistemas, sino que un estándar común de gestos será crucial para las personas que deberán trabajar con estos sistemas. Solo cuando se logre esto, será posible imaginar un futuro en el que se utilice ampliamente la comunicación sin contacto.

La versión completa de este artículo está sólo disponible en inglés

Evil by Design? How Interaction Design Can Lead Us into Temptation

September 7, 2013 No Comments

When you stop to think about it, the job of interaction designers is about persuading people to do something. We design juicy-looking buttons and place them “just so” on the page to entice people to click them. We remove distractions and streamline processes so that customers glide effortlessly from browsing to buying. We create whole applications aimed at getting people to check in more frequently, diet more effectively, work out more consistently, or do more, more, more of whatever it is that meets our persuasive goal.

What Is Evil Design?

Sometimes our aim is charitable: we are persuading people to do something that benefits society more than it benefits them. Sometimes our aim is motivational: we are persuading people to do something that will benefit them even if they wouldn’t choose to do it unaided. Often our aim is commercial: we are persuading people to trade something that will provide equitable benefit to our company and to them. Sometimes however, it seems aims are less virtuous, and companies seek to get customers emotionally involved in doing something inequitable—something that benefits the company more than it benefits the individual.

And that, perhaps, is the definition of evil design: to get customers emotionally involved in doing something that benefits you more than it does them. Now, your first reaction may be to deny that this would ever happen in your company, much less that you’d be complicit in it. But somewhere, someone in the interaction design profession is building these interfaces, because they aren’t accidental. They are the result of applying psychological principles or design patterns to arrive at an emotionally engaging experience.

Example Evil Design Patterns

Let Users Advertise Their Status

It’s hard to define which interfaces are truly evil because different people will derive different levels of intangible value from their interaction with a site or application. Take for example the ubiquitous email signature “Sent from my iPhone” (see Figure 1). I hope that the person who created that default signature setting got well rewarded for their work. It is the perfect embodiment of effortlessly viral aspirational content. Taken at face value, it is little more than an advertisement for a product—something that benefits the company more than the customer. Yet users seem strangely reluctant to change it. This inertia stems at least in part from what that small phrase says about them as individuals. Even more than half a decade after the device’s release, the phrase is still prevalent at the bottom of people’s emails, and it can’t be because no one knows how to change the setting.

The iPhone default signature remains partly because it’s boastful in a socially acceptable way. The designers found a way to let users advertise their status as owners of a shiny, desirable gadget. This design pattern, encouraging users to build and advertise their status within a community, lets people feel more important. It affects customers at an emotional level but serves a very financial goal for the company. Who benefits more? Financially, it’s quite clear. Emotionally, it’s hard to say.

Hands typing message on iPhone, with Sent from my iPhone on the screen
Figure 1. It can’t be that every person who uses an iPhone is incapable of changing the signature file, so why do we see message that end with “Sent from my iPhone” so often?

Pre-Pick Your Preferred Option

Psychologists have known for a long time that if they show you specific words or pictures beforehand, you’ll find it easier to recall those items, or related items, in a later test, even after you have consciously forgotten the specific words. This effect is called priming.

This is one reason why you see so much brand-related advertising. Rather than specifically telling you to buy a certain thing, the advertisers build a picture of a brand associated with a specific location (a bar), emotion (happiness), or occasion (a celebration). When you are in a bar for someone’s birthday party, all that brand priming comes straight back from your subconscious brain and hits you. You aren’t going to get a rum and cola; you’re going to get a Bacardi and Coke. You’re not going to buy a beer; you’re going to buy a Budweiser.

Couple this priming with inertia and you have the perfect conditions for encouraging people to select a given option. By making the option seem familiar beforehand, and then presenting it as the default choice, it’s easy to guide people’s behavior.

Use Negative Options

In the United States, many employers offer personal retirement savings plans, 401(k) plans, for their employees. Sign-up is typically optional and, once started, employees can choose how much to contribute. In their 2001 Quarterly Journal of Economics article “The Power of Suggestion: Inertia in 401(k) Participation and Savings Behavior,” Brigitte Madrian and Dennis Shea reported that those employers who automatically enrolled new employees saw higher rates of contribution than when employees had to make the personal effort to enroll. In both instances, it turned out that many employees couldn’t be bothered to change the default (pre-picked) state.

This inertia could have major implications for the future financial health of the country since participation in personal retirement plans reduces the burden on future government-run social services. Interestingly, many of the automatically enrolled employees kept both the default contribution level and fund allocation, which shows that it makes good sense to set the default option to the behavior that you want to encourage.

The last example almost certainly falls into the category of motivational rather than evil design. But the designer of that company’s healthcare policy leveraged the same concepts of priming and inertia that are also used to trick us into signing up for junk email. However good we get at spotting the marketing opt-in checkboxes on registration forms, we must still read them very carefully. Is the box pre-checked? Is it offering to opt us in or to opt us out? Is the second box, which talks about sharing details with third parties, also pre-checked? Is that one opt-in or opt-out? The company’s goal is to get you to start receiving its emails because you’re less likely to make the effort to unsubscribe after you’re signed up. This is the basis of the negative options design pattern: sign people up to receive something until they specifically choose to stop receiving it.

Take for example the three innocuous-looking checkboxes on the Hotel Chocolat website that can have major repercussions on the future size of your email inbox and those of the people you send gifts to (see Figure 2). Read carefully: which boxes need to be checked or unchecked to not receive emails?

Form with confusing options after logging in.
Figure 2. Your mission, should you choose to accept it: minimize the email you receive from this company by checking the correct boxes.

A study by Steven Bellman and his colleagues, described in the 2001 ACM article “On Site: To Opt In or Opt Out? It Depends on the Question,” found that simply using a negative option doubled opt-in rates. Phrasing an option as “Notify me about more surveys” with the checkbox unchecked (that is, people must take an action to get the additional surveys) led only to a 48 percent uptake. Changing the text to “Do NOT notify me about any more surveys” while still leaving the checkbox unchecked (that is, people need not take any action to get the additional surveys) produced an astounding 96 percent uptake. Of course, as Bellman notes, not all of the individuals who opted in (or rather didn’t not opt in) in the second example are actually good prospects, but the apparent consent rate is incredibly impressive. Often it’s easier just to keep doing what we’ve always done than to make a change, even if that change would save us time, money, or the hassle of a spam-filled inbox.

Make Options Hard to Find and Understand

The trouble is that persuasive design is a slippery slope. The boundaries between motivational, commercial, and evil design are wavy. They fluctuate over time. It’s typically an acceptable business practice to play up a product’s best features rather than be entirely objective. On the other hand, we also expect to be made aware of the implications of sharing data with a service, rather than having them buried in a privacy policy that nobody is likely to read. And people don’t read Terms of Use or Terms of Service documents.

It took four months and 3,000 downloads for someone to claim the reward that PC Pitstop hid in the Terms of Service document for one of their products. Only 0.06 percent of ISP Embarq’s customers read the updated Terms of Service document that allowed the company to spy on, and insert ads into, each data packet they sent through their Internet connection. Putting opt-outs in out-of-the-way places and obfuscating with hard-to-parse text seems to be a common business practice for introducing changes that are more likely to benefit the company than the customer.

Conclusion

I’m not approaching this topic from some holier-than-thou angle. I’m sure people can poke many holes in the persuasive techniques I’ve used in the products I’ve helped to design. Instead, I’m suggesting that a catalog of these techniques will help us to recognize and respond to them. Sometimes that response might be to avoid giving our business to sites that we see using the pattern. Sometimes the response might be to repurpose a pattern for good. Sometimes the response will undoubtedly be “that’s exactly what I was looking to implement in my product.”

My response has been to search out these persuasive techniques and describe them as design patterns. This article touches on four of the fifty-seven patterns I describe in detail in the book Evil by Design. As Don Norman says in the book’s foreword, “the more the tactics are understood, the more readily they can be identified and resisted, fought against, and defeated.” Please join the conversation about these techniques online at evilbydesign.info.让我们来看看四种能轻松地劝说和诱导人们从浏览过渡到购买的劝说式设计模式。

交互设计师的工作就是劝人们做某事：我们设计一些外观有趣的按钮，并“恰到好处”放在页面上以诱使人们点击，我们去除各种干扰因素，并简化流程，从而使客户轻松地从浏览过渡到购买，我们创建的整个应用旨在让人们更经常访问。作者在《蓄意而为》(Evil by Design) 一书中共介绍了 57 种设计模式，本文将讨论其中的四种劝说式设计模式。

文章全文为英文版사람들을 설득하여 탐색 수준에서 쉽게 구매 수준으로 유인하는 설득적 디자인 패턴 4개를 살펴봅니다.

인터렉션 디자이너의 업무는 사람들이 무엇인가를 행동하도록 설득하는 것이 관건입니다: 즉, 매력적으로 보이는 버튼을 디자인하고 페이지에 “보기 좋게” 배치해 사람들이 클릭하도록 유인하며, 어수선한 것들은 제거하고 일련의 과정을 능률적으로 처리함으로써 고객이 별 어려움 없이 자연스럽게 구경하는 수준에서 구매로 전환하도록 하고 고객의 더욱 빈번한 주문을 겨냥한 전반적 애플리케이션을 창출합니다. 본 기사는 저자의 책 Evil by Design에 설명된 전체 57개 중 일부인 4개의 설득적 디자인 패턴을 살펴봅니다.

전체 기사는 영어로만 제공됩니다.Um olhar sobre quatro padrões de design persuasivo que induzem e atraem as pessoas a passarem da navegação para a compra, suavemente.

O trabalho dos designers de interação é persuadir as pessoas a fazerem algo: projetamos botões atraentes e os colocamos “cuidadosamente “na página para atrair as pessoas a clicá-los, removemos distrações e simplificamos processos para que os clientes passem suavemente da navegação para a compra, e criamos aplicações completas que visam fazer com que as pessoas finalizem a compra com maior frequencia . Este artigo analisa quatro padrões de design persuasivo que fazem parte de um conjunto de 57 padrões descritos no livro do autor, Evil by Design.

O artigo completo está disponível somente em inglês閲覧している者を説得したり、気を引いたりして、楽々と購入へと進ませる4つの説得力のあるデザインパターンを概観する。

インタラクション・デザイナーの仕事は、つまるところユーザーを説得して何かをさせるということである。私たちはユーザーにボタンをクリックさせるよう、とても魅力的に見えるボタンをデザインして画面ページの「まさにそこ」と思われる場所に配置したり、気を散らせるものを取り除いてプロセスを合理化して顧客が閲覧から購入へと無理なく進むようにしたり、より頻繁に見てもらうことを目的として専用のアプリケーションを作成したりする。この記事では、筆者の書籍である『Evil by Design』 に記載されている57のパターンの中から4つの説得力のあるデザインパターンを取り上げ、レビューしている。

原文は英語だけになりますUna mirada a cuatro patrones de diseño persuasivo que atraen y convencen a las personas a pasar de navegar a comprar, sin esfuerzos.

El trabajo de los diseñadores de interacción se trata de persuadir a las personas a que hagan algo: diseñamos botones atractivos y los colocamos “casualmente” en la página para que las personas hagan clic, quitamos todas las distracciones y optimizamos el proceso para que los clientes se deslicen sin esfuerzo de navegar a comprar, y creamos aplicaciones enteras cuyo único objetivo es que las personas las visiten con mayor frecuencia. Este artículo analiza cuatro patrones de diseño persuasivo que son parte de un conjunto de 57 patrones descritos por el autor en su libro Evil by Design.

Brainsketching: Collaborative Interaction Design

March 5, 2014 No Comments

In the eyes of today’s user experience practitioner, facilitating a simple, straight brainstorming session seems archaic. The metaphor is perhaps too appropriate. Ideas strike the ground like lightning and vanish just as quick, leaving only a faint memory of what was once articulate and clear. As a discipline, we’ve evolved the ideation process quite a bit from simply shouting ideas at a facilitator who is furiously taking notes.

However, it’s becoming difficult to select from the vast number of design methods out there: affinity diagrams, design charettes, participatory design exercises, experience prototyping, bodystorming…

In my own work on design studios, I have discovered that a method called “brainsketching” might be one of the best and most versatile tools we have. It is a collaborative sketching exercise in which a group of people create interface concepts by iteratively building on each other’s work.

You may know about “brainwriting” methods as written forms of brainstorming. These approaches to idea generation avoid chaos by having a group write and share ideas on paper, with the goal of generating large numbers of ideas in a short time. A few years ago, I tried adapting brainwriting into a collaborative sketching exercise for interface concepts. I stumbled upon a great design method for lateral thinking in groups, and found a name for it in a book from the 1980s called Techniques for Structured Problem Solving by Arthur B. VanGundy.

Why Brainsketching is so Useful

In working with teams over several years, I have observed four reasons why brainsketching is so effective and versatile:

  1. Forcing Deferred Judgment. When given only a minute to silently contribute to the idea of another, each person is forced to quickly understand and build on the idea. This method forces everyone to defer judgment and be constructive.
  2. Systematic Layering of Ideas. Imagine that four people participated in a brainsketching workshop. The rapid exercise means that a sketch might have general elements drawn by one person, basic details drawn by another, and finer details drawn by still others. Each sketch layers expectations, assumptions, and current thinking about the design problem in different sequences, systematically cross-pollinating ideas. Even if people bring pre-determined ideas to the table, they may define the general layout of one sketch, but the other three sketches will demand that they think in different ways.
  3. Dissolving Personal Ownership. Having each person use the same colored markers makes it difficult to clearly see each person’s contributions to a sketch. By dissolving the ownership of design concepts, the group can have a less political discussion and critique. This is especially important when people around the table are on different levels of the organizational hierarchy.
  4. Distributing Visible Contributions. Brainsketching forces participants to think and work visually. This helps all project stakeholders (even those with business or technical roles) feel like they had a more tangible and directly visible hand in the final design. When wireframes, mock-ups, and prototypes are created, everyone can recognize design elements and feel more personally engaged in the process.

[bluebox]

A Step-By-Step Guide to Brainsketching

To start, sit four to six project stakeholders in a circle each with a black marker, a colored marker (all the same color), and a piece of sketching paper (with a pre-printed template if needed).

Present the design problem and decorate the walls with information about the objectives, constraints, and research. Tell the group what is in scope for the sketching exercise (for example, a specific platform or format).

Follow the instructions below to build sketches collaboratively. To conclude the exercise, lay the sketches out to examine, discuss, and evaluate. You can ask pre-planned questions or have an open-ended discussion. In any case, take notes in a very public way, perhaps on a white board, to ensure you’ve got consensus.

Have some participants switch seats and then repeat the exercise. Tell everyone that they can try something new or build on previous ideas.

In the end, you’ll have an abundance of design concepts and notes describing the group’s consensus about them.

Group brainsketching drawing activity

At 1 second, start a timer. As soon as it begins, tell everyone to start sketching any solutions that come to mind. Use the black marker to sketch, and the colored marker for notes about interaction.

At 60 seconds, instruct the group to immediately pass their sketches to the person on their right. If participants try to keep sketching, give them a second before being more directive.

At 61 seconds, tell everyone to start building on the sketch now in front of them. Instruct them to first take a few seconds to understand what was drawn, and then add constructively to the concept.

At 120 seconds, pass the sketches to the right again. As the sketches get crowded, instruct them to clarify things and add detail.

At 121 seconds, again instruct everyone to constructively add to the new sketch in front of them for the next minute.

At 180 seconds, the pattern should be familiar. Continue until each person has contributed to each sketch.

Illustrations by Emilie Bonnier[/bluebox]

Brainsketching in Action

While brainsketching has been instrumental for my work on typical web/mobile UX projects, I have been most surprised by its effectiveness in the face of more complex design challenges.

In one project, the Yu Centrik team worked on the UX of an online statistical database for the UNESCO Institute for Statistics (UIS). Instead of guessing the needs of the complex array of stakeholders on the project, I invited strategic decision makers, subject matter experts, and technical personnel from UIS to join our team for a brainsketching exercise. After two sessions we produced an interface that was easy-to-use, strategically appropriate, statistically relevant, and technically feasible. Further, the process unified all stakeholders in a way that clarified the common goal.

In another example, researchers at Simon Fraser University’s iSpace laboratory were working on simulating an embodied experience of flight. After an extensive literature review, the students were demotivated when realizing that similar work had already been done in Europe. I volunteered to conduct a brainsketching workshop. Not only did the workshop re-motivate the team, we ended up with a much more interesting concept for a new form of interacting in a virtual environment. Brainsketching was just as effective for designing an installation as it has been for more typical screen-based interfaces.

The Value of Design Democracy

My experiences with brainsketching are one of many reasons why I believe that democratic approaches to interaction design trump the myth of the lone genius. If you still consider design a solo activity, I urge you to give brainsketching a try and see how systematically extracting the whole team’s ideas influences your practice.头脑风暴绘图是一种协作式绘图方法，通过一组人员在彼此间的绘图作品上反复修改，从而建立起界面概念。 这种方式有助于让团队实现创意的积累，同时可将作出判断的时间延后。 这种交互式方法让整个团队都能做出贡献，并消融个人拥有的感觉，让多元化的小组更易于在内部彼此协作，减少人际争斗的讨论。 本文说明如何使用这一方法，并列举了来自两个项目的例子。

文章全文为英文版브레인 스케칭은 한 집단의 사람들이 반복적으로 서로의 작업을 기반으로 함으로써 인터페이스 개념을 만들어 가는 협력적인 스케칭 활동입니다. 팀이 판단을 나중에 하고, 일단 아이디어들을 먼저 모으는 데 도움이 됩니다. 인터랙티브 활동은 팀 전체가 기여를 하며 개인의 소유권이 없어져 다양한 집단이 정치적이지 않은 토론을 기반으로 협력하는 것을 더 쉽게 해줍니다. 이 기사는 두 가지 프로젝트의 기법과 사례들을 활용하는 방법에 대해 설명합니다.

전체 기사는 영어로만 제공됩니다.Brainsketching é uma forma eficaz de colaborar em um exercício rápido de visualização de ideias através de desenhos em que um grupo de pessoas cria conceitos de interface de forma iterativa com base no trabalho do outro. Ele ajuda que a equipe tenha ideias, postergando julgamentos. O exercício interativo permite que toda a equipe contribua e reduz a propriedade individual, facilitando a colaboração em um grupo diversificado e diminuindo a discussão política. Este artigo fornece instruções sobre como usar esta técnica e exemplos de dois projetos.

O artigo completo está disponível somente em inglês.ブレインスケッチングは、集団でインターフェースのコンセプトを作成する際に作成したものを皆で反復的にひとつにまとめ上げていく協同的なスケッチング作業である。これは決断を急ぐことなくチームがアイデアを集めるのに役立つ。インタラクティブな作業によりチーム全員が結果に貢献でき、一部の個人のみがオーナーシップを持つことがなくなり、グループ内の駆け引きを抑えながら多様なメンバー間で協同することを容易にする。この記事ではこのテクニックの使用法を説明し、具体例として2つのプロジェクトを取り上げる。

原文は英語だけになりますEl Brainsketching es un ejercicio colaborativo de bocetos en el que un grupo de personas crea conceptos de interfaces trabajando de manera iterativa a partir del trabajo de cada uno de los miembros. Ayuda a un equipo a presentar ideas a la vez que pospone las opiniones. Este ejercicio interactivo permite que el equipo completo contribuya y elimina la propiedad personal, lo que hace que sea más fácil que un grupo diverso colabore y se reduzcan las discusiones políticas. Este artículo proporciona instrucciones sobre cómo utilizar esta técnica y ejemplos de dos proyectos.

La versión completa de este artículo está sólo disponible en inglés

Multi-screen Games and Beyond: New Dimensions in User Interaction

November 15, 2011 No Comments

At the Electronic Entertainment Expo (E3) in June 2011, Nintendo introduced the next generation of game consoles, the Wii U. It’s nothing less than a radically new approach to the user experience of games. In it, two screens work together wirelessly, allowing different views for different players, and much more. Not only is this new paradigm a major shift in the player experience, it’s also a huge change in the way game designers must think.

Smartphone or Tablet as another Screen

Nintendo isn’t the only kid on the block who’s playing around with multi-screen gaming:

  • Apple is also aggressively moving into this space. They recently introduced Airplay SDK for games running on the iPad 2. The iPad 2 can now send video directly to the Apple TV 2 wirelessly, creating a similar experience to the Nintendo solution.
  • Onlive, the current leader in cloud-based games, has created an iPad app to allow use of that device in conjunction with their service. Using Onlive in conjunction with Airplay is a strong possibility, opening up some interesting game-playing scenarios.
  • Adobe has been showing plenty of game examples using Android devices as controllers.
  • Qualcom’s open source AllJoyn project also allows developers to create multi-screen experiences using Android devices.
  • Our company, Brass Monkey, created a two-screen game experience for the Star Wars Trench Run in July 2010; we enabled an iPhone version of the game to become a controller for people running the game on the LucasFilm website www.starwars.com.

Using a smartphone as one of the screens means that people have game controllers with them at all times. This allows for interesting scenarios where people can begin using their phones to interact with digital signs at locations outside of the home. Imagine, for example, multi-screen gaming with the televisions in bars and cafes. Wii sports games, trivia, Texas hold ’em poker, and a whole slew of game designs make perfect sense in these settings. Being able to engage directly with their consumers in such a compelling way is also a dream that’s possible today for advertisers.

Game Designs

Now let’s look in some detail at new game designs enabled through handheld touch devices working with a larger screen.

Multiplayer, each with a different view: Take a typical card game, for example, and imagine being able to show the players’ cards on their phones, each one being different from the next, while all the players focus on a host screen for the table and shared experiences.

Multiplayer with different roles: Nintendo demonstrated this scenario with some of the Wii U demos at E3. The player using the touch screen controller acts as a boss, looking at his personal screen for the view while other players use standard Wiimotes to control characters running around on a split-screen setup on the larger screen.

Extra screen as an auxiliary view: One of the screens can act as a separate view into important aspects of the game. Let’s say the user has an iPhone as the controller and an iPad acts as the map view. The TV screen shows play in the action game. The player is able to focus on the main screen but use the iPad as a sort of dashboard for all important info at a quick glance. This setup would be a fantastic advantage for those playing RPGs (role playing games) in the style of World of Warcraft.

Controller screen as a view into another screen: If you use the controller’s camera—supported by all smartphones and the new Wii U—to augment the view on the large screen, you will essentially be seeing through the device (an iPhone 4, for example) into the other screen. Doing so changes the view through the device’s screen.

Imagine a tank game where you control a tank within a 3D environment. The goal of this game is to shoot enemy tanks as they move about in the game, and to prevent being hit by their fire. You hold the phone in portrait mode, allowing for control of your tank’s movements by pressing a mini virtual joystick on the lower right side of the phone. You hold the phone up, looking through it as a scope. Movement of your tank’s turret (as well as view) can be controlled by simply moving the phone in any direction. The camera on the iPhone is activated to allow what is going on in the world around it to be displayed on its touch screen. Because the phone is being held the way the camera is pointed at the host screen, you can see through your phone to the action being shown on the screen in front of you. This setup allows for many possible scenarios:

  •  The device’s camera display can show content not viewable on the host screen with the unaided eye, for example, simulate X-ray vision, show invisible enemies, or simulate night vision.
  • The device’s display can show a reticle used for lining up and indicating when an enemy is locked in on for firing (as described above).
  • The display could simulate real 3D based on head-tilt position. Similar to the example at http://www.youtube.com/watch?v=Jd3-eiid-Uw.

There are literally a thousand other ways that screens can be put together to create compelling game experiences. My goal is to inspire you to think of some scenarios yourself and go out and make them a reality.

Beyond Games

Games are certainly an obvious application of multi-screen experiences, but how will this concept affect our lives beyond games? Where else can you imagine combining multiple screens to create rich user engagement? Here’s a quick list:

The classroom: Real-time collaboration applications in classrooms are a great use of the multi-screen experience. Imagine a professor giving a lecture to his students, where he periodically gives students a problem to solve. The professor has an application running on a large screen so that all the students can see it. This application shows the question being asked of the students. The students also have an app running on various devices that work with the teacher’s application. In this case, the professor’s question shows up on students’ screens prompting them for an answer. Perhaps this is a question that requires you to draw a diagram in the answer region. Along with the other students, you then draw what you think the answer should be. Some students may be using the latest iPad, where they use a finger to draw, and others may be running the application on a laptop, using its track pad to draw in the answer region.

When students are done, they click a button within the application that instantly submits their result to the professor’s program. The professor now has the results back from the students, and he can choose one student to share the answer with him by selecting the student from a list on his application, prompting the display of this student’s diagram on the large screen where all the students can view it. At this point, the professor can discuss how the student performed, perhaps making corrections to the diagram by drawing on top of it via his laptop’s mouse.

Medical settings: Hospital and patient care settings also pose interesting possibilities using portable and fixed screens. One thought is that large touch screens aren’t exactly the most sanitary devices, and people leave smudges of various germs on them. Perhaps there is another way to interact with these screens.

Imagine a doctor-patient visit. The doctor pulls up the patient’s medical records via a mobile device—let’s say an Android tablet. This application alerts the doctor that the patient’s MRI results are in, which the doctor would like to review with the patient. On the wall of the examination room is mounted a large LCD screen hooked up to a computer running a compatible medical application. When the doctor’s tablet is detected and the two endpoints are connected, the doctor can pull the patient’s MRI image onto the larger screen, so she can go over the image with her patient. The doctor can use fingers on the Android device to manipulate the image—pinch to zoom in, two fingers to rotate, and all the touch gestures we’ve become used to—on the tablet rather than on the LCD screen.

Museums, amusement parks, and other interactive experiences: Museums and other situations with interactive displays, both digital and physical, are another target for this type of technology. People love to interact with museum displays, and the more interactive the installation, the more use it usually gets. The problem is that all that use takes its toll. The display controllers often break, and maintenance of the installations can be a tough job. If we start letting people use the devices in their pockets, however, we put the maintenance responsibility back on the user.

Computers that drive the experiences of physical installations, like those in museums and theme parks, can allow for interaction with mobile screens. Imagine a museum that has an installation of prehistoric men. It includes mannequins that move and are controlled by a computer out of the visitors’ sight. Typically, museums will allow the visitor to control the experience via physical buttons on the display case. Instead, imagine that visitors can now use their mobile phones to trigger interactions.

Another installation could be a series of smaller fixed screens with which the user could interact. The possibilities for public installations are just as unlimited as the possibilities for games.

The Future

User experiences will involve us interacting with screens everywhere. Every screen, from the one you carry around in your pocket, to televisions, digital kiosks, and Jumbotrons at the ballpark, will all work together for next generation of experiences. Games will also undergo a major revolution because of all these screens being connected. It’s happening today, and it’s all very exciting.
在 2011 年夏天举办的 E3 会议上，任天堂 宣布推出 Wii U，此游戏向公众展示了相当新的多屏幕游戏用户体验。此类游戏（使用基于触摸屏的控制器）将对我们的游戏玩法产生重大影响。这种方式不仅是玩家体验的重大转变，也是游戏设计师必须深思的重大转变。这篇文章介绍一些新游戏设计，这种设计通过使用大屏幕手持触摸设备进行操作，文章中还详细讨论了支持这些新体验的新平台。

文章全文为英文版2011年の夏、E3（エレクトロニック エンターテイメント エキスポ）コンファレンスで、任天堂がWii Uを発表し、マルチモニターのゲームにおける比較的新しいユーザエクスペリエンスを一般大衆に紹介した。このタイプのゲーム（タッチパネルベースのコントローラを使用）はゲームのプレイ方法に大きなインパクトを与える。このパラダイムはプレイヤーの体験に大きな変化を起こすだけでなく、ゲームデザイナーの考え方にも大きなシフトを要求する。この記事は、タッチ操作のハンドヘルドデバイスで大型スクリーンに表示してプレイする新しいゲームデザインについて取り上げ、この新体験を可能にする各種新プラットフォームについても詳しく説明している。

原文は英語だけになります

The New Voice of YOU: Communicating Through the Digital Interface

February 9, 2015 No Comments

Digital interfaces we own, design, or develop are an extension of our face, our voice, our body, and our words. So every time someone interacts with a device or app it is also an interaction between that person and us—the creators of that interface. When we think about it that way, we can understand the growing call among the UX community for a shift in perspective from “User-Centered Design” to “People-Centered Design.”

Direct human communication is not easy. Whether with parents, children, spouses, colleagues, or even the cashier at the supermarket, opportunities for misunderstandings and miscommunication are the norm. So how do we manage even a reasonable communication by proxy? Can we design interfaces that are better at understanding and communicating with people for a more effective and satisfying interaction?

I believe we can.

There are plenty of how to guides on improving communication between people. I would like to use Dale Carnegie’s How to Win Friends and Influence People, perhaps the best known how-to guide of all, to illustrate how principles for better human communication can improve digital interactions.

A boy draws a smiley face in the fog on a window.
Figure 1. Smile!

Principle 1: Smile

Making a good first impression with users is crucial. It can tilt the scale towards downloading our app, subscribing to our blog, or purchasing on our website.

In discussing ways to make a good first impression, Carnegie demonstrates how brightening up our own demeanor radiates to those around us, making our interactions not only pleasant, but actually profitable. In his words: “..a smile says, ‘I like you. You make me happy. I’m happy to see you.’” Of course, as Carnegie emphasizes, it’s the genuine smile that does the magic. An insincere grin is immediately recognized and rejected.

But how can you make a digital interface smile and make it feel genuine?

As Carnegie shows, a smile can be more than the visible combination of eyes and teeth. It is also the tone of your voice, the shake of your hand, and the air or vibe you exude, even if your interaction is over the phone. The idea is to present a happy state of being, one that invites people to enjoy and share in this happiness.

Translating this to the design of a digital interface means a number of things:

  • Understanding what makes particular users and customers happy as people within this user group. What will put a smile on their faces?
  • Present to users that which, for their user group, is an unmistakable sign of joy or fun. It can be a witticism, a game, or visuals of sunny beaches, but it must be understood without explanation.

Here are examples of a digital smile in action:

Google doodles are fun and easy interactions, which often replace the standard Google logo and invite visitors to join the fun. (It’s clear that the people making them had fun).

Father, son and daughter riding in a home-made winged go-cart
Figure 2: Family fun in Google’s doodle for Father’s Day in Brazil, 2014

The Hipmunk logo is a sweet creature that is clearly having a lot of fun flying. The mobile application logo is one big fun smile, used both as the application icon and within the application.

A chipmunk spreading its arms in mock flight
Figure 3. The Hipmunk logo dances while is searches for flights

Of course, just as in human interactions, a smile is not always what you want to project; maybe your users expect an “I mean business” service or “This is serious stuff” atmosphere. It’s up to you to determine when to smile at your users and when it’s time for a serious talk. You should also avoid asking users to affirm this happiness; this may send a conflicting message, putting them on their guard.

Principle 2: Make the Other Person Feel Important—and Do it Sincerely

Carnegie concludes that perhaps the most important element of any interaction is making the person feel important and appreciated. Achieve this, and you have reached the person’s heart.

And yes, be sincere about it.

How do we make others feel important? What are the things that make us feel important and appreciated?

Respect who they are.

I recently flew with United Airlines. I am not an American resident and I was looking for the address of the United Airlines office in my country. The united.com website automatically identified my location and even had the flag of my country displayed to let me know they know where I am.

But while the information for the U.S. and Canada is provided within one click,  to find the local office address, I was required to browse deeper instead of it being the default information.

Respect what they want.

As the song says, “You can’t always get what you want…,” but even when a request cannot be answered—or must be outright denied—you still can get what you need: a little respect.

Learn what your customers and visitors want when they interact with you and let them know that you respect their desires, whether you can accommodate them or not.

Amazon’s “Alert me when an out-of-stock item is available” option is a good example of respecting a customer’s unfulfilled wishes. Alternately, interactive voice response (IVR) systems are a frequent example of companies’ blatant disrespect for their customers’ wishes.

Pay attention to what they are feeling.

This is perhaps the most important way to make people feel important and appreciated…and the hardest to accomplish. Especially when feelings are on the low end of the spectrum.

How can you tell what a person is feeling on the other end of an interface?

Even without artificial intelligence behind your interfaces, you can often make a good guess. For example, a person entering the “Help” section is likely to be somewhat confused, maybe even getting frustrated and angry. This is when they are the least patient and need the greatest assistance and patience from you. As in the Amazon example above, a customer whose request cannot be fulfilled is probably disappointed to some degree.

Acknowledge these emotions and, if you can, act on them (like the example of Amazon alerting users when a product is available).

Tell others.

Social sites are applying this principle so well that unless we are new or infrequent users, we don’t even notice anymore when our content is pushed to our contacts’ walls.

LinkedIn has taken this one step further by designating certain high-profile accounts as “Influencers” and promoting them to LinkedIn subscribers. The list is updated constantly with the implicit promise that you too may one day make the list. How important is that?

The basic requirement to applying this principle is to know your users. The better you know them the more opportunities you have for expressing your appreciation of them. Your users, just as your spouse, child, boss, employee, or return customer, will expect it of you.

Principle 3: Don’t Criticize, Condemn, or Complain

Criticizing others is typically human. We may criticize people for various reasons: they made an error, they didn’t understand us, or they are misunderstood by us. Even if they simply acted in a different manner from what we expected, we should not criticize them.

Carnegie warns us against criticizing people: it simply doesn’t work. Plus it has the converse effect of creating antagonism against the critic.

As if taken right out of “The UX Professional’s Manual” Carnegie says, “Instead of condemning people, let’s try to understand them. Let’s try to figure out why they do what they do.” (Did I mention this book was written in 1937?)

Alan Cooper, in his seminal book About Face, warns against negative feedback, which makes users feel stupid and angry. Error prevention and good error messages are also part of Nielsen’s 10 heuristics for usability in interface design.

The fundamental premise is, in my view, to address user errors and misunderstandings as a glitch in communication. And it is our responsibility to identify it and to fix it. Indeed, this is true in every case where the user has acted in a manner which is outside the optimal task flow.

Why is this crucial?

Well, for starters, because it’s true.

Within my interface, the interaction is controlled by the rules I set. Misunderstandings are the results of these rules. Additionally, the interaction itself was initiated by me—in creating the interface—to promote my purpose or goals. It is in my interest to push the interaction forward toward whatever goal it serves, and this means that I am responsible for anticipating that the person interacting with my interface may act differently than I had expected and provide for this.

There is an added benefit avoiding criticism. As Carnegie puts it “…it breeds sympathy, tolerance, and kindness,” and this, in turn, will make people more tolerant of my mistakes.

It may be easier than you’d think:

  • Avoiding “it’s your fault” messages is obvious. But it’s less obvious to also avoid “it’s my fault” messages. Talking about anybody’s fault can put people on guard (“Will it be my fault next?”) and is not the way to go.
  • Acknowledge that a legitimate misunderstanding or mishap has occurred. Note that you don’t have to actually “say” anything, but you do have to be prepared for this event with a reaction that can ensure that the interaction does not stop then and there.
  • If you can’t pursue the interaction without interrupting the user, provide clear suggestions on how the interaction can return to its intended track.
  • Offer explanations and clarifications as to your intentions and expectations (like using assisting text within input fields and providing help links and online chat).

A great example are the responses of Google and Bing to a search for an unclear string—displaying results for an assumed, similar search—with the option to search instead for the original string.

Message: Showing results for Abraham Lincoln. Search instead for abrm linciln
Figure 4: Google provides a suggested (corrected) string, but keeps my original request readily available.

Principle 4: Be a Good Listener

An interaction is a dialog, and as Carnegie points out, a good dialog requires that you truly listen to your partner.

Carnegie notes additional benefits of being a good listener: it pays your partner a compliment, providing them with a feeling of importance, and often generates good will.

People, says Carnegie, are much more interested in themselves than they are in you, so in order to generate a good dialog, encourage them to talk about themselves and, as he puts it, “ask questions they will enjoy answering.”

For me, the most valuable element in Carnegie’s insight is to listen “with genuine interest,” to seek to understand the wishes of the other person with the intent of making them happier and more satisfied.

In his book Improv for Storytellers Keith Johnstone adds that to listen is to “be altered by what’s said.” This means you are gathering information, not for your purposes, but rather for the other person’s benefit. And you must listen for—and sometimes ask—what this benefit is.

As UX professionals we are reminded to observe behavior and even ignore what users are saying. In many cases, this applies to digital applications as well. Since digital communication is mostly action and often involves no written or spoken words at all, we must gain insight to our users’ wishes through their actions.

One example of how this is not achieved can be seen when Google asks a direct question: do I want to allow or deny Google.com to use my device’s location? If I choose to tap the “X” rather than the “Deny” option, Google will display this question every time I search, completely ignoring the fact that I have tapped “X” on each occasion, thus indicating that “Allow” is not an option.

The message reads “Google.com wants to use your device location” with buttons to close (X), deny, or allow.
Figure 5: Google search on mobile. How many “Xs” equals “Deny”?

Principle 5: Throw Down a Challenge

Speaking about motivation and productivity, Carnegie observes that the desire to excel and to perform better is rewarded in doing the work itself as an opportunity to prove one’s own worth.

In daily human interaction, this can be compared to complementing children on their efforts not their successes and, as the research of happiness is showing, in rewarding employees with recognition, autonomy, and greater challenges rather than more money.

The main theme here is to achieve interaction and user involvement through encouragement, by talking about a person’s current position compared to where they began and how this brings them closer to their next milestone.

A bar showing progress relative to a goal with a trophy at the end.
Figure 6: With hardly a word said, NikeFuel shows that you are progressing to your next milestone and prize.

But take care to phrase your encouragements carefully. Udemy.com also encourages users to progress on their online courses, but “You completed X out of Y..” can just as easily be read as a complaint; tone of voice does not come through in the written text.

The message says “You completed 73 out of 83 lectures”.
Figure 7: Is Udemy encouraging or admonishing me?

Make the Most of the Opportunity

Human communication can be difficult when done directly, and these difficulties multiply when communication is indirect.

But when we succeed, we achieve a more satisfying and effective interaction that benefits both the people interacting with digital interfaces and those who created them. UX designers are best positioned to lead this approach by incorporating the principles of human communication in our design processes:

  • During user and business requirements research
  • Designing interactions
  • Designing the GUIs
  • Testing
  • Iterating

Digital interfaces are our extension. We are the people who create and manage them and through them we interact with other people. Let’s make the most of this opportunity.

Thank you.

😉对于“以人为本”这一设计方法的需求日益增加，因为这种方法将引导我们设计出更易于理解、更便于与他人沟通的界面。在设计界面时，我们应该考虑与人际交往领域相同的原则，例如戴尔•卡耐基在《如何赢得朋友与影响他人》(How to Win Friends and Influence People) 一书中提出的那些原则。了解我们用户的需求、启发积极思维、宽容、尊重他人甚至幽默都有助于为用户提供更加有效、更令人满意的交互效果。

文章全文为英文版

‘사람 중심 디자인’ 접근법에 대한 필요성이 증가하고 있습니다. 이 접근법은 사람들을 이해하고 상호작용하는 데 더 뛰어난 인터페이스를 디자인하도록 이끌어 줄 것입니다. 인터페이스를 디자인할 때, 우리는 사람들과 상호작용하는 것과 동일한 원칙들을 염두에 두어야 합니다. 데일 카네기가 쓴 How to Win Friends and Influence People(어떻게 친구를 얻고 사람들에게 영향을 끼치는가)에 나오는 것과 같은 원칙들 말입니다.  사용자의 요구를 이해하고, 확신을 불어넣고, 관대하고 정중한, 심지어 익살 맞은 인터페이스는 사용자들에게 더욱 효과적이고 만족스러운 상호작용 경험을 제공할 것입니다.

전체 기사는 영어로만 제공됩니다.

Existe uma necessidade crescente de uma abordagem de design centrado em pessoas que nos guie para desenvolver interfaces que sejam melhores em compreender e se comunicar com as pessoas. Ao projetar interfaces, devemos considerar os mesmos princípios que utilizamos na interação com as pessoas, como os apresentados no livro de Dale Carnegie, How to Win Friends and Influence People (Como fazer amigos e influenciar pessoas). Compreender as necessidades do usuário, inspirar positividade, ser tolerante e respeitoso, e até mesmo bem-humorado, pode ajudar a proporcionar uma interação mais eficaz e gratificante.

O artigo completo está disponível somente em inglês.

人をより良く理解し、より良く意志疎通ができるインターフェイス設計をするための人を中心としたデザイン手法へのニーズは高まっている。

インターフェイスをデザインする際、我々は、人と関わる場合と同じ原則―――たとえば、デール・カーネギーの名著、『How to Win Friends and Influence People』（人を動かす）に示された原則――を考慮する必要がある。寛大さや敬意、そしてユーモアさえをもってユーザーのニーズを理解し、利用意思を引き出すことは、より効果的で満足のいく関係性の構築に役立つ。

原文は英語だけになります

Existe la necesidad creciente de un enfoque de diseño centrado en las personas que nos guíe para diseñar interfaces que comprendan y se comuniquen mejor con las personas. Al diseñar interfaces, debemos considerar los mismos principios que tenemos en cuenta al interactuar con las personas, como los presentados por el libro How to Win Friends and Influence People (Cómo ganar amigos e influir sobre las personas) de Dale Carnegie. Comprender las necesidades de los usuarios, inspirar positivismo, ser indulgentes, respetuosos e incluso graciosos, puede ayudar a proporcionarles una interacción más eficaz y satisfactoria.

La versión completa de este artículo está sólo disponible en inglés

Future Proofing Tomorrow’s Technology: UX for an Aging Population

February 27, 2015 No Comments

The trend of an aging population is reflected across Europe. It is estimated that by 2035 nearly a quarter of the population in the United Kingdom will be over 65 years. Aging is traditionally associated with a decrease in mobility and social interaction, and this can have detrimental effects upon well being and access to good nutrition and health. Fortunately there has been much research into the effects of age upon product interaction.

We conducted a systematic review of the research literature, and based on what we learned, we created a framework to organize the issues that directly affect the use and adoption of technology by older users. It is a concise and simple tool, easy to apply across numerous products to aid design thinking and evaluation from an older person’s perspective.

A Quick History Lesson

Historically, the design community has been accused of failing to understand and engage with distinct user groups, preferring to design from their personal experience and capability: If I can do this, then so will all the users of the product I am designing. This, argued many academics (and to be fair, design professionals), risked alienating and excluding significant proportions of the population, as a lack of user understanding was transferred into products that became unsatisfactory, unappealing, and downright unusable for significant groups of users.

However, our experience in design circles today tells us that we are more aware than ever that understanding the user experience helps design more usable products. While application in commercial environments can still be a struggle, the idea that user-centered and participatory design can ensure a better end-product-fit continues to gain momentum. Industry acceptance also grows as the concept of strategically involving users within the design process equates to greater adoption and engagement out-of-the-box, and thus results in significant financial savings in development costs.

Let’s Think about the Users

In various communication-based models of design, the user’s and designer’s interpretation of products or artifacts are considered as mental models. These mental models are based in part upon user expectation and perception—expectation in terms of users’ existing knowledge and prior experience about how the product might behave, and their perception of how further interaction might occur. Both these factors can be influenced by the feedback and messages received from product features, tactile and visual cues, and the context within which interaction occurs.

Assistive technology is usually designed for those with some cognitive or physical impairment, often attributable to trauma or natural atrophy, or to lifelong conditions. However, if the user and context is not realized or understood sufficiently, the danger remains that designers may not truly appreciate the differences in personal capabilities between themselves and the users of the technology.

What’s Age Got to Do with It?

Good question! Successful interaction spans product features, both functional and aesthetic. As we age our eyesight, hearing, dexterity, and grip all begin to deteriorate. In a study conducted some years ago (see More Reading at the end of this article), 30 individuals from the ages of 16-80 were presented with a novel product. The participants were asked to discuss their preconceptions and understanding of the product before performing six tasks, and then again after performing the tasks.

Mental model development was adversely affected with age. The models that the older participants possessed contained less relevant information to drive effective interaction, and their ability to acquire and consolidate new relevant information also declined with age. Increases in age were directly related to decreases in icon and feature recognition—and understanding—both before and after product interaction.

By coding and classifying each individual’s interactional behavior, it became evident that younger participants engaged in greater amounts of skill-based interaction where behavior becomes automatic and almost unconscious, and that older individuals were noticeably slower to complete tasks and indulged in rule- and knowledge-based behavior (see Figure 1).

Chart displaying the interaction behavior findings described above. It shows that the younger participants were considerably more skill-based than the older participants who were more rules- and knowledge-based.
Figure 1: Behavior activity type classified according to age group, N=30 (Wilkinson, 2011).

Unlike younger participants, older individuals interacted at a more conscious level throughout, considering the effects of their behavior before, during, and after interaction. Accordingly, viewing these factors in combination, we realized that for intuitive interaction, product design must cater to these and other aspects for an aging population if product uptake, adoption, ease of use, and engagement are to be achieved.

So What Did We Do and What Did We Find?

We examined 50 literary articles published between 1992-2014 that referenced assistive technology and older users. Analysis revealed the themes most frequently cited as being affective and influential to the adoption, usability, and success of assistive technology. We also identified concepts that may also adversely affect the uptake and adoption of such technologies.

From the analysis of the published research, we developed the framework shown in Figure 2. The placement of the “User” at the center of the framework (process) mirrors the aim to heighten product engagement by following the principles of user-centered design.

A diagram of the framework. Surrounding the User at the center of the diagram, there are four quadrants that reflect the themes and subthemes described in great detail below.
Figure 2: A visualization of older peoples’ needs as a Framework for Design Thinking (Wilkinson & Gandhi)

Although there is no specific order in which design consideration should be applied, it is appropriate to focus on the center of the model (the User) and work toward to the circumference. As a tool, the act of physically focusing upon the user from the start helps to remind us that it is they whom we should be considering throughout.

In the introduction of the four higher level themes, moving clockwise facilitates consideration of the user initially, their experience, the supports they may require, the economic aspects that may apply, and the social implications of any existing system or design. Each of these higher-level constructs is then further segmented (and explained in greater detail below) to foster deeper consideration of the effects these aspects may have for older individuals.

1. User Experience

Interaction designers aim to create interactive technologies that are enjoyable, pleasurable, motivating, and satisfying. These goals are largely dependent upon users’ acceptance of technology, their perceptions of the technology, and their level of engagement with it.

  • User perception. User perception involves the ideas users gain about how devices work and are manipulated purely through their design. If users’ ideas of how interaction is likely to occur based on the product design do not transfer well and the design misleads them, it is likely to result in poor performance and product abandonment.
  • User engagement. User engagement can involve physical interaction, social interaction, and activities in terms of entertainment and leisure. Feedback is an important factor for successful engagement, and a lack of feedback may directly contribute to delayed or impaired rates of technology adoption and increased rates of disengagement with technology.
  • User acceptability. Acceptability is one of the fundamental requirements of technology stated by older users. The assumption that older adults would use an assistive technology purely because they need it is misguided. However, an interdisciplinary approach toward needs-assessment and design, understanding the user physically and psychologically, and designing accordingly, decreases the risk of subsequent equipment abandonment.

2. Physical and Psychological Supports

Elderly and older adults often require guidance, physical support, health monitoring, and the scheduling of their medicines in their daily life. Therefore, assistive technologies need to provide support and address older user’s individual physical, physiological, and psychological requirements.

  • Physical support. Problems with health can be critical for older people as it restricts them from moving freely in the environment. Older adults possessing limited physical capabilities often experience difficulties in conducting daily activities, particularly regarding self-care and mobility. Effective assistive technology has the potential to enhance their ability to perform daily tasks and transform their mobility.
  • Emotional support. Some older adults feel using assistive technology is seen as a symbol of dependence and frailness, and felt stigmatized by association. They expressed that a solution that merely addressed a clinical need was not enough; any feature or functionality also had to “look good.” However, well-designed mobility aids can bolster feelings of safety and self-worth by consequently increasing personal independence. This, in turn, can help older adults maintain access to networks of emotional support and facilitate their independence.
  • Cognitive support. Assistive technology can support memory loss and dementia by providing support for decision making and activity reminders, and support tasks requiring higher cognitive function. Older adults with cognitive impairments also perform memory tasks more effectively by using external memory aids, so the use of appropriately developed assistive technology in such situations should be encouraged.

3. Economic Aspects

Less financially affluent individuals often avoid using assistive technologies for activities of daily living due to the perceived costs of intervention. Innovation is not the only motivation for design: economic and social issues should also provide an impetus for the design and development of assistive technology.

  • Intrinsic cost. The direct financial cost to the user of assistive technology can be high, and this adversely affects rates of user adoption. Designing assistive technology for a more widespread market may assist in reducing the economic barriers and stigma associated with assistive technology use.
  • Extrinsic cost. Long term healthcare for the elderly is resource intensive and expenditure in this area is high. However, appropriately designed assistive technologies can reduce healthcare costs, including costs related to institutional care and in-home nursing. They can reduce healthcare practitioner visits as well as slow the rate of decline in patient capability.

4. Social Aspects

An older adult’s quality of life is particularly dependent on a network of social relationships. Failure to consider social aspects can result in the failure of innovative technologies. Well-designed technologies can enhance older adults’ feelings of social connection and reinforce their sense of self-identity.

  • Social connection and interaction. It has been argued that technological, economic, and social changes have increased social isolation, and that assistive technologies for the elderly have overlooked this issue. Assistive technologies can be designed to broaden social connectivity functions. Helping users move more easily outside the home, and building connectivity into the design, allows users to maintain access to social networks and encourages both physical and virtual interaction.
  • Personal identity. One aspect of personal identity is derived from an individual understanding their precise location within a group or network of relationships. Assistive devices have been viewed by users as a threat to this identity. However, technology designed with enhanced understanding of users’ needs and requirements may help users to reestablish a sense of normality, personal comfort, and individual identity.

Applying the Framework

The framework presented can be used as a benchmarking or scoping tool in thinking about the design of assistive technologies that particularly consider older people’s needs. It can easily be applied across numerous products to aid design thinking and evaluation from an older person’s perspective.

It also reminds us all to think outside the box and outside of ourselves by placing the user at the center of our consideration. The hope is that this will enthuse both designers and users of assistive technology and encourage increasing dialogue between these groups. In doing so, we hope to foster and facilitate an environment where older peoples’ needs are understood and more readily met by the designers of tomorrow’s technology.

 

[bluebox]

Further reading by Chris Wilkinson available on the studies referenced in the article.

[/bluebox]

基于当前关于老年人需求研究文献所建立的一个框架遵循了以用户为中心设计原则。框架中的主题包括用户体验、生理或心理支持以及经济和社会因素。在引入辅助技术和其获得成功的过程中，它们常常被认为是有效和发挥作用的要素。

文章全文为英文版

노인들의 니즈에 대한 수많은 문헌 분석에 기반한 프레임워크는 사용자중심 디자인 원칙을 따른다. 이 프레임워크는 사용자경험, 신체적-심리학적 지원, 경제적-사회적인 관점 등 다양한 주제를 포함한다. 이러한 주제는 보조기술의 성공과 확산에 아주 중요하고 효과적인 요소들이다.

전체 기사는 영어로만 제공됩니다.

O framework, criado com base na análise da literatura atual sobre as necessidades dos idosos, segue os princípios do design centrado no usuário . Os temas do framework incluem experiência do usuário, apoio físico e psicológico , aspectos econômicos e sociais. Estes são os elementos mais citados como sendo eficazes e influentes na adoção e no sucesso da tecnologia assistiva .

O artigo completo está disponível somente em inglês.

年配者のニーズに関する最新の論文の分析に基づく評価の枠組みは、ユーザー中心設計の原則に従ったものである。この枠組みに含まれるテーマには、ユーザー体験、身体的・心理的なサポート、経済状況、社会的状況が含まれる。
支援技術をうまく取り入れてもらうために、これらはもっとも効果的で影響力が強い要素であると言われる要素である。

原文は英語だけになります

Un marco de referencia basado en el análisis de la literatura actual sobre las necesidades de adultos mayores siguiendo los principios de diseño centrado en el usuario. Los temas en el análisis incluyeron experiencia de usuario, apoyo físico y psicológico, aspectos económicos y sociales. Estos son los elementos más frecuentemente citados como efectivos e influyentes en la adopción y el éxito de la tecnología asistida.

La versión completa de este artículo está sólo disponible en inglés

What Shall We Watch Tonight? Using Mobile Devices to Plan TV Viewing

September 15, 2007 No Comments

Interactive Television (iTV) offers an exciting future full of dynamic viewer-channel interactions. However, while increasing numbers of consumers purchase iTV capable devices ,and channel providers experiment with services, a range of usability challenges remain.

One problem that warrants investigation is that of planning what to watch on television. With an increasing number of available channels, each offering many different programs, it is clear that many people may develop a major headache navigating through the maze of possibilities to find a program they want to watch.

Electronic program guides (EPGs) developed to help television viewers deal with this increasing number of channels are now regularly used by many people. Program guides display the television schedules on the television screen and allow viewers to choose what to watch directly from the on-screen list.

Although they are certainly popular, program guides on their own are not necessarily best suited to the task of television planning. Unlike printed television guides, they can only be used while in front of the television set—the user cannot use them while on the way to work or waiting for a train to arrive. Furthermore, the viewer must be focused on using the system and be actively engaged in setting viewing preferences and browsing the on-screen listings. This does not come naturally to most television viewers because viewers like to “lean back” and passively watch television, rather than “sit forward” to interact with their television set.

Our research efforts have been focused on how handheld technology—mobile phones, Personal Digital Assistants (PDAs), and the like—present an opportunity to augment traditional program guides and to better support television planning. Because it empowers viewers to plan their television viewing anywhere and at any time, it means that their natural style of interaction is better supported.

a list of tv shows and what channels they play on
Figure 1. An Electronic Program Guide (EPG)

Electronic Program Guides

A wide range of research has been conducted to look at how best to design program guides. These include studies evaluating the usability of program guides and studies looking at how peoples’ television viewing and planning habits might influence and inform the design of future program guides. Such studies often result in design guidelines and recommendations. However, existing guidelines are primarily directed toward on-screen guides, accessed directly via the television set. No such guidelines exist for a program guide running on a handheld device, or for a system integrating an on-screen program guide with a handheld device.

Some program guides have made the transition to handheld devices, but all are very much technology driven. The push appears to have come from the technology behind the personalization and artificial intelligence features, and not necessarily from any identifiable user needs for such a system. Furthermore, existing handheld systems seem lacking in any real integration with their homebound TV counterparts. They provide recommendations but do not allow users to directly act upon them, for example, by requesting recommendations to be recorded. The only notable exception to this is PTVPlus-GuideRemote. This system not only enables a user to directly control the television, but also provides personalized television listing information and program recommendations. However, being based around a universal remote control and not a PDA or mobile phone, the system clearly has limited scope for use away from the television set.

Multiple Device User Interfaces

Multiple device user interface systems for digital television tend to be oriented toward either delivering what has been termed “extended television”—that is, giving people further opportunities to view or participate in a program away from their TV set—or toward directly controlling the television channels and functions using a handheld device. However, not much work has been directed toward television planning.

Extended television can be seen as an attempt to extend digital television beyond the television set itself, allowing the extension of television brands to a range of different electronic devices. So, viewers can feel connected to a television program through a mobile phone or the Web even when it is not on air and there is no television within reach. Such services can involve synchronous interaction, where the interaction occurs while the television is turned on, or asynchronous interaction, where the interaction occurs either before or after the program has been broadcast.

Paces of Interaction

As users’ context changes, so might the desired level of interaction with the technological resources available to them. We might then differentiate between the “lean-back” interaction seen in conventional TV viewing, and the “sit-forward,” highly engaged form seen with computer use.

Studies all agree that television traditionally supports a very passive style of interaction, but that the level of interaction increases with the introduction of increasingly interactive services, such as program guides. There is a clear conflict between the traditional style of television interaction and the more interactive television services. It is therefore not surprising that studies have shown that not all viewers are eager to switch to a more involved level of interaction, preferring the old and the reliable to the new and suspect. Researchers have sought to reduce this level of interaction by pushing more personalized content through program guides. However, to date, little work has investigated the possibility of moving it away from the television and onto a different platform, such as a PDA or mobile phone.

Television Planning Diary Study

We asked eight people in the United Kingdom, between ages 18-35, to keep a week-long diary of their TV planning activities. We asked them to note the resources they used, when they were used, and for what purposes. We then used these entries as a basis for a mid-study phone interview and an end-of-study face-to-face discussion.

chart of tv planning and scheduling
Figure 2. Example of a diary entry.

 Planning Can be a Social Activity

All diarists indicated that they often watch television with other people. This seems to have a real impact on how they plan what to watch because consideration must be given as to what others want to watch.

Although most diary entries were of individuals using television planning aids, a number referred to collectively planning when seeking an agreement about what to watch together. Changing channels or on-screen television planning aids are used to enable everyone to see what programs are available. Perhaps the ingrained social nature of watching television makes on-screen television planning aids more suitable for collective planning. Watching television is frequently a social activity, so it seems natural that using a television to plan television viewing also would be a social activity.

Television can also provide a social commonality. For example, diarists spoke of chatting to friends about programs that they watched. There was also evidence of diarists recommending programs and movies to friends. One diarist spoke of a friend with whom he regularly sent and received text messages with television program recommendations. Another saw a program that she felt a friend might be interested in and phoned to let them know that it would be on. There appears to be evidence of people not just looking for themselves, but also looking for television programs that might interest others.

Viewers Look for More of What they Like

The type of program and whether the diarist had seen it before and liked it were the most important factors when deciding what to watch. The channel broadcasting the program was seen as being important because it is recognized that some channels—such as MTV—only show programs of a certain type (music in this case). Furthermore, there is a clear issue of trust. It is recognized that some channels, primarily the terrestrial ones in the UK, can be relied upon to show programs worth watching. That is, the quality of the content is seen as being higher than what some of the other television channels offer.

Often diarists would talk about watching a program because there was a lot of discussion about it both in the media and in their communities. Furthermore, they watched programs because they were recommended by others. Such recommendations might come from friends, family, or from editorials and television critics. Recommendations are especially influential for films and one-off television programs, where there is little prior knowledge of the program.

Planning is Often Short Term

Predominately, people used the guides to map out the coming evening’s viewing or while sitting in front of the TV to consider what was available. Longer-term planning was focused on programs users particularly wanted to watch or record (such as a film or documentary).

The Design Concept

We developed a low-fidelity mobile prototype to support viewing planning in three ways:

  • By providing a recommendation facility where one user can send suggestions to another’s mobile, and where “recommender services” (such as broadcasters) can also send alerts. Suggestions are sent in a text message mode that alerts the recipient as soon as one is sent.
  • By focusing the mobile guide on personal highlights for the user by default listing only the upcoming day’s highlights. Users can create lists of their favorite programs and program types.
  • By reminding them of a program they indicated they wanted to watch or record.
PDA with a tv schedule on it
Figure 3. The paper-and-PDA mock-up used in evaluation.

 Evaluation

We developed and facilitated a proof-of-concept study with twelve users (five female, seven male, aged 18-35). Each participant worked with the prototype, which consisted of mock-up dialogue screens attached to a hand-held computer manipulated by the investigator in a naïve Wizard-of-Oz form. That is, the prototype had no built-in interactivity, but was animated by the investigator choosing what to display next. We asked users to work through a series of tasks that demonstrated the extended program guide facilities. At the end of their session, we asked them to rate the features and provide additional feedback during an interview.

We found that the participants saw the personalized TV highlights as the most useful feature. The reminder function was second in popularity. In terms of the recommendation features, users expressed a strong preference for family/friend recommendations over third-party ones. Even so, many participants reported that they would prefer to phone or text someone, rather than explicitly recommend a television program. There were also concerns about being alerted each time a new recommendation arrives. Participants saw a less intrusive system, allowing recommendations to be dealt with when it is convenient to do so, not necessarily when they arrive, as preferable. That is, a system much more like email than texting. Television programs that viewers are currently watching provide the only instance where recommendation alerts are desirable. However, most participants still reported that they would rather call because there is no guarantee that their friend or family member is watching television at that moment.

photo of a woman holding a PDA
Figure 4. A participant using the prototype during an evaluation.

Conclusions

We have presented some initial arguments and evidence that suggest conventional electronic program guides need augmenting. Program guides do not support planning away from the television set, and do not support the engaged, social style of interaction required in some TV planning scenarios. Migrating some of these activities to a handheld device appears to be a promising approach that merits further investigation.

While the social element of the system—the personal recommendations—was viewed favorably, our study suggests that there is much further work needed to understand how to integrate existing collaborative viewing practices (such as word-of-mouth promotion) within emerging iTV technologies.

Touch Me! Reinventing a Business Application for the Touchscreen

August 15, 2010 No Comments

How do you take a physical keyboard/mouse-based business application and design it specifically for touch interaction? Your starting point is a set of physical keyboard/mouse-based widgets like list boxes, dropdown lists, checkboxes, and radio buttons. Your challenge is to somehow find a way to map their functionality to a touch interaction paradigm. For applications with data grid displays, row selection must also be mapped to touch. Designing touch-friendly navigation in an online help system is also required, since online help is vital for mission-critical business applications.

As I worked through these design challenges for our retail point-of-sale (POS) application, several user interface (UI) patterns emerged that do the job efficiently and effectively for our users. I’ll describe these patterns and explain why they work well for a full-touch business application. However, to understand the scope of the effort, it’s important to lay out the two major challenges associated with designing interaction for business touchscreen software; first, the greater functionality associated with the business application, and second, the environment in which the application must function.

The Challenges

One difference is that business applications have greater functionality than other applications, so much so that considerable user training is typically required to master a business application. This increase in functionality creates a special UI design problem for business touchscreen software. As Jakob Nielsen mentioned in the Forbes.com interview, “Touchscreens work best for applications where you have very few options, such as small, portable devices with little room for larger numbers of buttons.”
In the retail POS domain, increased functionality requires designing for a large number of user task flows, each of which has multiple steps and travels down a broader set of branching pathways than a self-serve kiosk, mobile device, or even a restaurant ordering system. The resulting complexity increases the effort of designing consistent navigation for saving, deleting, backing up, modifying, and canceling completely at any point along a path.

The variety of POS user tasks also increases the types of interactions that need to be designed. Our users may need to calculate amounts, select individual list entries and apply various actions to them, control printing and other options, search for items, combine and separate groups of items, enter customer information, sell items that cannot be scanned, handle interruptions in individual sales, balance registers, and administer user and system privileges (see Figure 1). Enabling touch for these interactions presents a formidable design problem.

Point of sale system screen images
FIgure 1. The original point of sale system screen.

In addition to the greater functionality, the business user’s environment gives rise to a second design challenge: most of the time, self-service kiosks and mobile phone applications are operated for leisure and entertainment purposes. In cases where these devices are used for business purposes, there is typically a back-up process for accomplishing the task. For example, a customer service desk or a full-sized computer email system typically backs up mobile or kiosk ordering devices.

In the business environment, on the other hand, a dedicated, domain-specific business application is used to accomplish mission critical, workaday tasks. There is no back-up process for selling items at a store register. Here the business user’s ability to operate workplace-specific applications is intimately tied to success or failure at work. Critical dependency on software makes business users unwilling to sacrifice time and resources on unproven applications. In the retail world, that means a full-touch UI cannot compromise speed or accuracy in any way.

Designers clearly have a big hill to climb when they try to take a business application into the touch paradigm. Not only must touch-screen business applications support rich functionality and multiple user interactions, but to be competitive, the user experience must be better than the physical keyboard/mouse implementation that is being replaced.

Touchscreen Design:
Reinvention vs. Redesign

I use the word “reinvention” rather than
“redesign” to describe the touchscreen design experience. “Reinvention” signifies the novelty of thinking required for this kind of UI design, and I would like to explain how I came to this important design realization.

The journey started when I spent time evaluating the few POS applications running on a touchscreen. They used larger versions of physical keyboard/mouse widgets, which were obviously decreasing the user’s speed. Imagine, for example, using your fingertip to scroll line-by-line, up or down, in a dropdown list box of fifty text options.

Simply redesigning widgets to be bigger clearly wasn’t working for touch, and there were no practical touchscreen design guidelines to be found. Facing project deadlines, I decided to go by what I had observed: the only objects that users could touch quickly and accurately were large, finger-sized buttons. With this in mind, I reinvented widgets that used button interactions.

Option Selection Widget

In the physical keyboard/mouse world, users select options from simple text lists, dropdown lists, checkboxes, and radio buttons. Depending on the widget, one or multiple selections are allowed. I reinvented these in a single widget that uses a button array UI pattern, which is shown in the top left of the screen in Figure 2.

Screen image
Figure 2. Touch widget added to the screen.

Touch friendly buttons are sized to accommodate the typical adult fingertip. The option label is put on the button. The array can be programmed to allow one or multiple buttons to be selected, depending on requirements. When an unselected button is touched, the visual presentation of the button changes to indicate that the button is selected. When an already-selected button is touched, it reverts back to the unselected visual presentation mode.

Users don’t have to learn a variety of different widgets for option selection. They are always presented with touch friendly buttons that visually turn on or off. If a large number of options must be presented (for example, selecting one U.S. state out of fifty), a separate screen can be used.

Grid Row Selection Widget

When presented with a grid of information in rows and columns, users sometimes need to select a specific row in the grid to view additional details about that row. In the physical keyboard/mouse widget, either the entire row is a selectable object, or a particular cell contains a selectable text link.

I reinvented row selection in a widget that uses a row button UI pattern, in this case a blue round button with the label “i” (see Figure 3). Although it is small relative to our other buttons, the row button is a touch-friendly target, in contrast to a text link or row of text. The row button must be repeated in each row, which may require enlarging the rows to accommodate it. This is not an issue for most business applications, as the rows typically have to be enlarged anyway to accommodate wrapped text. The challenge is to design a button that is both obvious and touch friendly without overwhelming the visual layout of the grid.

Screen image
Figure 3. List screen

Online Help Window Management Widgets

Online help for physical keyboard/mouse applications is typically provided in a separate screen, which can be manipulated independently from the application screen that it references. Links to help topics are provided in a selection list. Applications may support the automatic display of context-sensitive help topics through a special interaction mechanism, such as the F1 key. Otherwise, the user must navigate to the relevant topic from the primary help screen, typically with the assistance of a search function.
Since manipulating independent windows is difficult on a single-touch UI, I reinvented the help display in a widget that presents context-sensitive help inside the application screen (see Figure 4). After the user touches the Help button at the top right of the application screen, several changes occur. First, the application screen resizes proportionally in both dimensions, while remaining active and touch friendly. Second, the context sensitive help topic is automatically displayed in the yellow column on the right. The user does not have to navigate to the correct topic. Paging buttons are provided at the bottom of the context sensitive help for navigating within the topic, and a zoom button enhances readability. Finally, as the application screen changes, the help content changes to match. If the user leaves the help open all the time, it becomes a self-training tool.

Screen image
Figure 4. Context-sensitive help

These UI patterns are currently limited to single-touch interaction, but we are moving to multi-touch as well as to wide screen touch monitors. With more interaction options and additional screen real estate, we’ll have a powerful design environment.

From Practice to Theory

When in need of ideas and inspiration, designers often turn to best practices. These are tried and true guidelines that are backed up by a theoretical foundation. Not only do the guidelines work in practice, the theory tells us why they work. This combination increases our confidence level in using them. But when we are innovating, the best practices that work within an older paradigm may not apply, as I discovered. In this case, we have to start with what we think will work based on observation, and later investigate whether there is a theoretical foundation for it.

After seeing that the reinvented widgets worked well for our users, I wanted to know why. Within human factors and cognitive theory, I found two principles that were directly related to touch-screen interaction:

  1. Human eye-hand coordination is highly refined and accurate, particularly in the act of pointing (wonderfully expressed by neurophysiologist William Calvin in How Brains Think).
  2. Fitts’s law: The time it takes to acquire a target is a function of the distance to the target and the size of the target. We see a speed-accuracy tradeoff associated with pointing, whereby targets that are smaller and/or further away require more time to acquire (described in various sources such as wikipedia.org).

Reflecting on how these principles work together in the touchscreen domain, I found a strong explanation for why touchscreen interaction is more intuitive and direct than physical keyboard/mouse interaction:

  • By combining the presentation surface and the interactive surface, touchscreen interaction enables finger-pointing for target acquisition.
  • Finger-pointing brings highly-evolved human eye-hand coordination into play for target selection and is, by nature, an intuitive movement. Speed and accuracy are increased because users don’t have to look down at a keyboard surface and orient their fingers, or locate, grasp, and orient a mouse on a work surface, and then look back up at a screen to acquire the target. Even proficient keyboard/mouse users are slower because attention must still be divided between two surfaces.
  • Touchscreen interaction enhances accuracy by increasing the proximity of the interactive surface to the visual feedback.
  • The user is closer to the target (the touchscreen) when pointing than when using a physical keyboard/mouse. This proximity makes the human finger even more accurate at hitting a target directly in view on the screen than a human hand would be in guiding a remote pointing device (mouse) or than a finger would be in touching a key on the physical keyboard.

This theory explains why the reinvented widgets worked. A useful guideline for touchscreen design follows from it: a successful touch application maximizes the ability of the human finger to point to the on-screen target. For short, I call this “maximizing touchability.”

It is possible to port a traditional business application to a touchscreen environment, provided you are willing to go beyond simply re-rendering the existing interface on the new device. Maximizing touchability is the key to success.您如何将基于物理键盘/鼠标的商业应用改而设计成基于触摸交互功能？移动设备、自助服务设备和餐馆应用都是触摸屏领域的明星。但对于重负荷商业应用，例如客户支持、零售销售点 (POS) 和办公管理等应用，情况如何呢？在这些领域，触摸屏是否也可以取得成功？2009 年，Jakob Nielsen 在接受 Forbes.com 采访时似乎并不这么认为：

“[触摸屏]之所以具有吸引力，其实原因很简单：与鼠标有些不太直接的风格相比，触摸屏更直观、更直接。对于选项较少的应用（例如空间很小，因而无法放大量按钮的小型便携式设备），触摸屏就非常适合。”

Nielsen 的观点似乎不容置疑，因为当今在使用的全触摸式商业应用是如此之少。作为面向零售应用开发商的用户界面设计者，我知道，客户很希望在其店铺运营中使用触摸屏获得其所公认的可用性好处，这一点确凿无疑。我们已经在全触摸 POS 应用上取得了成功。

转向触摸应用并不会自动保证获得其可用性好处。用户界面必须进行专门设计才能利用触摸交互所带来的好处。仅仅将基于键盘的应用转向触摸屏，并增大用户界面对象的尺寸是起不到什么作用的。那么专门针对触摸交互进行商业应用设计需要考虑哪些因素呢？这就是这篇文章所要解决的问题。

文章全文为英文版キーボードやマウス操作を使った業務用アプリケーションはどのようにしてタッチ操作用にデザインし直せばよいのだろうか。携帯機器、セルフサービスタイプのキオスク、レストランで使用されるアプリケーションなどは、タッチパネル利用分野におけるスターだ。しかしカスタマーサポート、小売店のレジやオフィス管理などに使用される重要な業務用アプリケーションではどうだろうか。タッチパネルで同じように成功を収めることができるだろうか。ヤコブ・ニールセン（Jakob Nielsen）は2009年の時点でそれには賛同しなかったようで、Forbes.comにおいて次のように述べている：

「（タッチスクリーンの）魅力は実にシンプルです。何か直観的で直接的なものがあり、マウスはそれに比べて少しだけ間接的なスタイルなわけです。タッチスクリーンはオプションがほとんどない、つまり小型のポータブル機器でたくさんのボタンを配置するスペースがないような用途に最適です。」

今日業務用アプリケーションには全てタッチ操作で使用できるものはほとんどないため、ニールセンの見解は正しいように見える。ユーザインタフェース（UI）のデザイナーとして小売り用のアプリケーションのベンダーに勤める私は、クライアントが店舗業務のためにユーザビリティの面で評価の高いタッチパネルの利点に関心を持っていることは知っている。我々は全てをタッチ操作で行うPOSアプリケーションですでに成功を収めている。

しかし、単にタッチ操作に切り替えることで自動的にユーザビリティの利点が得られるわけではない。UIは、タッチ操作のインタラクションを上手く活用するようにして特別にデザインされる必要がある。ただ単にキーボード操作ベースのアプリケーションをタッチパネルにしてUIのオブジェクトを拡大しただけでは上手く機能しない。では業務用アプリケーション、特にタッチ操作でインタラクションを行うタイプのデザインには何が必要とされるのだろうか。その質問に取り組んだのがこの記事である。

原文は英語だけになります

Playful Interfaces: Designing Interactive Experiences for Children

May 25, 2018 No Comments

Practicing designers can tell you that designing mobile touchscreen apps for children is different than for adult users. Still, what does science tell us about what interface differences are critical to remember? We are engaged in the science of child-computer interaction. Our empirical research has focused on capturing how the cognitive and physical traits of young children under the age of 10 affect the success of their interactions with touchscreen interfaces. We, and others, have produced research-driven design recommendations to consider. Here, we share our top seven guidelines for designing for children under the age of 10 and the evidence that led to them.

Designing Interactive Experiences for Children

Children can be a delight to design for when creating interactive touchscreen apps. Their excitement and interest in apps with bright colors, fun characters, and their natural propensity for wanting to play can be a rewarding space for designers to explore. However, the standard methods and techniques of designers that are used for adults may not always apply, or may have to be adapted to work with children. In particular, children’s rapid development, both physically and cognitively, can make design work a constantly moving target. What works well for a 4-year-old, can be drastically different from what engages an 8-year-old. A minor usability problem for an adult can make an app completely unusable for a child. What is more, the varying literacy abilities of children means you may need multiple methods of engaging children, or to stretch your thinking beyond your typical toolbox of design patterns, menus, and icons when generating touchscreen interface designs.

Children’s ability to participate in standard user-centered design methods, like interviews, surveys, and usability studies, is also quite different, and more care needs to be taken to ensure that designs are appropriate for children. The good news is that by learning to design for children, you will likely make applications easier and more delightful for all, so it is well worth the effort. Allison Druin, a founding figure in the science of designing for child-computer interaction, outlines four ways that children can be engaged in the technology design process, each with varying degrees of involvement:
1. Children as users
2. Children as testers
3. Children as informants
4. Children as design partners or co-designers

While “children as design partners” is the most involved method for engaging children, it is also the one with the biggest payoff. Design studios that focus on children often form co-design teams with children, who help adult designers come up with new ideas, sketch, and mockup prototypes on an ongoing basis. If you can’t create your own co-design team, you can still work with children as part of this process, either by studying them as users or engaging them as informants or usability testers. While we hope that the guidelines below can provide a shortcut to designing great experiences for younger children, nothing will replace engaging them as part of your design process.

Designing for Children’s Physical Capabilities

Anyone who has watched a small child try to use a fork or draw with a pencil can understand that children’s motor skills are not always up to the task. Gross and fine motor coordination develops in phases over the course of a child’s first 10 years of life. Gross coordination, such as reaching one’s arm out and grasping a cup with two hands, is fairly well-established behavior by age 4 to 5 months. Meanwhile, the fine motor dexterity required to hold a pencil, assemble and disassemble small LEGO blocks, or braid someone’s hair, takes several more years to develop.

Using touchscreen devices like smartphones and tablets calls on more fine motor control than gross, especially given that on-screen widgets are often quite small. You might assume that little fingers would have an easy time touching little targets (for instance, children don’t have the “fat finger” problem so often cited in mobile app design for adults, in which one’s finger obscures a significant portion of the surrounding content on screen). However, the dominant factor in hitting these targets is the fine motor skills required to zero-in on the target and touch it successfully. Our work has shown this effect is true regardless of target size or screen real estate. In human-computer interaction, Fitts’ Law dictates that the time it takes to successfully hit an on-screen target is a function of the distance to the target and the size of the target. As the size increases, the time decreases. Children’s developing motor skills make smaller targets too challenging, leading to our first design recommendation:

Guideline 1: Use larger on-screen widgets for children; the younger the child, the larger the target

(left) Thinkrolls application screenshot showing six round targets with different “monsters” inside them. (right) Thinkrolls application screenshot showing a monster in a maze with simple graphics and large target areas.
Figure 1: Thinkrolls, a logic puzzle game designed for preschoolers, uses large targets with generous “hot spots” that allow children with limited fine motor skills to move the characters and pieces around.

A child’s motor skills development also affects their ability to perform expected interactions in touchscreen apps, like drag and drop, or other gestures. Our work and that of others has shown that children under the age of 5 cannot use drag-and-drop interactions effectively: They often accidentally stop making contact with the device during the drag, losing their progress. A more successful approach with this age group is “tap-and-tap,” that is, allowing them to tap the item to be moved first, then tapping where they want to move it. Thus, our second design recommendation based on children’s physical capabilities is:

Guideline 2: Don’t use drag and drop for children under the age of 5; provide alternatives like “tap-and-tap”

Research on child-computer interaction has also explored children’s use of gestures such as tap, swipe, zoom, and rotate. While children as young as 4 can easily perform simple gestures like tap and swipe, more complex gestures like zoom in, zoom out, and rotate are more difficult for children to use even up to the age of 8. In our work, we have also examined shape-based gestures such as squares, circles, arrows, and letters or numbers. Children might be asked to use these gestures in apps to select items or to practice their writing skills. Our work has shown that gesture interaction like this often requires too-fine motor control for children under the age of 6 or 7.

Decades of child development science have resulted in the creation of neuropsychological tests of visual-motor integration skills. Here, children are only expected to begin to be able to spontaneously draw certain shapes as they pass through developmental milestones. For example, at ages 3 to 4, children may only be scribbling; by ages 5 to 7 they can create shapes like circles and squares; by about 7 to 10, children are able to produce well-formed geometric shapes. Providing tracing guidelines and being tolerant of children losing contact with the device during their gesture can help children make some of these gestures more successfully. This leads to our third design recommendation:

Guideline 3: Avoid complex shape-based gestures for children; either provide tracing guides or use simpler gestures without tight corners and long paths.

Finally, on the motor skills side, our work has shown that children’s slower reaction times affect their interactions throughout the interface in various ways. For example, across our studies, we have consistently found that children tend to be slow to notice when an on-screen widget has been activated, leading to held-over touches, double-taps, and unintentional interactions. We have also seen that these behaviors are amplified as more and more visual information (for example, animations and graphics) is available onscreen. Figure 1 shows Thinkrolls, an app that has higher tolerances. Combined with young children’s higher level of inaccuracy in tapping onscreen targets (for instance, tapping just outside the bounds of the target), this leads to our fourth design recommendation:

Guideline 4: Increase tolerance in time and space for children’s slower reaction times and inaccuracies in their interactions.

Designing experiences that are physically appropriate for young children is not enough. While preschoolers may have no physical barriers preventing them from watching Game of Thrones, its content was not designed to be cognitively or emotionally accessible to this audience. Through our reviews of content for children in app marketplaces, we have found that designers targeting young children routinely draw on design paradigms developed for adult users, not all of which are a good match for children.

As we described earlier, designing interactions driven by gestures that children can perform is an important component of creating apps they will be able to use. Nevertheless, it is equally important to help them understand what gesture to perform and when. In one study we looked at the age when children begin to understand common types of prompts and instructions embedded in the apps they use (for example, common design patterns such as audio instructions, blinking items on screen, or a cartoon hand demonstrating an action). Children under 3 were only able to interpret instructions when they came from an adult model. However, children made rapid gains between the ages of 3 and 3-and-a-half, at which point they were able to follow certain types of in-app prompts users are likely to encounter.

In this work, we did not uncover systematic techniques that enabled our youngest participants to understand app directions unless they were supported by an adult. For 2- and 2-year-olds, designing experiences that encourage adults to participate and help children figure out what to do is especially useful. Apps that support exploration and do not require children to figure out the one “right” action are also a good match. These findings lead to our next design recommendation:

Guideline 5: Encourage adult modeling of interactions for children under 3. Later, audio or onscreen demonstrations can be used. Avoid visual effects for teaching gestures and interactions.

Four panels showing different types of interaction prompts: one with a dog and a bowl, next to an animation of a bone sliding up a container to prompt children to do that motion; one with a dog tilted onscreen to prompt children to shake the screen; one with a shadowed dog pointing toward a solid dog to prompt children to swipe across the screen; and the last, with a dog with a glowing outline to prompt children to tap the dog.
Figure 2: We experimented with different ways to teach children novel gestures, including voice prompts, symbolic demonstrations, and visual cues.

Still, children do more than just manipulate onscreen objects; they also need to interpret the content on-screen and make sense of what it means. Decades ago, psychologists realized that young children gradually develop the ability to link symbols to the objects they represent. For example, the ability to see one object (a globe) and know that it is two things at once (a cardboard sphere and a representation of Earth) is a skill that takes children several years to master.

Why does this matter for app designers? In a review of popular apps for children under 5, we saw that about half used symbols as an important part of the interface, in the form of progress bars, badges, maps, and other representations. For example, Figure 3 shows two versions of a mini-game in which the player fills a cup with water from a faucet. In both cases, each time the child taps the faucet, a small amount of water pours in. In the version shown on the left, the water can be tracked through direct observation, while the version shown on the right requires the player to interpret a symbolic progress bar. We saw that even children as young as 2 years of age could easily detect when the cup was full if they could also track the water directly. Still, children were nearly 4 years old before they robustly understood that a full progress bar meant the water had reached the top. This leads to our next design recommendation:

Guideline 6: Avoid symbolic representation in apps for children under 4. For older preschoolers, make symbols as literal as possible (for example, represent the cars they have collected with tiny car icons, not stars).

(left) A transparent water cup with different levels marked on it, so that as the child taps the onscreen lever to turn on the faucet, the water level rises visibly in increments. (right) A water cup that is opaque so that as the child taps the on-screen lever to turn on the faucet, the water level does not rise visibly and is tracked by an onscreen progress bar next to the cup instead.
Figure 3: Symbolic representations are more difficult for children to understand than direct representations, as shown in this example of water level in transparent versus opaque cups.

Finally, when families choose content for children, they consider more than just usability. For many parents, the way in which an app, a game, or a video fits into daily life is at least as important as how it’s played. In interviews, parents have consistently told us that one of the main reasons their young child used apps and games was because it freed parents to take care of other essential chores, work, and self-care. They also said that although they thought this was a “win-win” situation that benefited the whole family, it can sometimes be a struggle to set boundaries and they wished it were easier for children to set down apps, games, and videos. In exploring this further, we have found that families find it easiest to align their boundaries with natural stopping points within a game or app. Like chapters in a book, natural stopping points make it easy for families to take stock of how the app fits their current situation and decide whether it makes sense to continue engaging or set the experience aside, leading to our final design recommendation:

Guideline 7: Design for natural stopping points within the game to encourage children’s self-regulation and ability to transition off screen-based media. Avoid including features that auto-play or advance new content that children select without taking breaks.

Conclusions

The evidence is clear: Children under the age of 10 need different interaction support than other age groups. Referring to these seven guidelines will help you design children’s touchscreen apps that are more successful for this age group by supporting their natural development and growth. Moreover, including children as part of the design process—whether they as testers, informants, or co-designers—will ensure a better experience for all. By considering these tips, we hope you will be able to focus on the fun factor of designing for kids!

If you’re interested in learning more, such as the best font choices and colors for children’s apps, we highly recommend Sesame Workshop’s Best Practices: Designing Touch Tablet Experiences for Preschoolers.