Sometimes people make up their minds about certain aspects of technology when they hear about or see it, before they physically interact with it. How likely are users to change their minds after they interact with the technology, particularly when they find their original opinion was incorrect? Are certain aspects of technology more or less influenced by subsequent physical interaction? If so, are there thematic differences between such aspects? This article explores these questions and discusses implications to user testing where, even in the face of serious usability failings, users do not change their original opinion.
As a participant in a large research study, Betty was interviewed about her impending purchase of an MP3 player. The interviews were timed to be pre-purchase, immediately post-purchase, and a few weeks later. Betty was a university undergraduate and a drummer, and felt that music was very much a part of her life, saying, “Music is what I’m about.” When asked which MP3 player she was going to buy, she said, “A lot of people that I know, their iPods have broken. So that’s why I decided to go for a different brand.”
During the post-purchase interview, however, she revealed that she actually ended up getting an iPod, explaining, “It’s so easy to use, has good software, and is cool…it’s quite stylish as well.” During our third interview a few weeks later, we again asked her about her iPod. She reported, “It broke.” Worse still, she had a horrible time being bounced between the manufacturer and the supplier, and finally returned it for a refund. Betty concluded that she had an overall negative experience. Then we asked if she would recommend it, and Betty answered, “Yes, I would recommend it. As I said, it’s cool, easy to use, it’s a good MP3 player. It is good, it is good.” Her opinion clearly conflicted with her actual experience.
In contrast to Betty’s story, participants in another study were asked to view a full-size color photo of an MP3 player, rating it on aspects such as ease-of-use and understandability. Then participants interacted with the actual device and rated it on the same aspects. One participant, Kate, initially rated the MP3 player in the photo as very easy to use. When Kate was given the actual device, her three-minute experience unfolded as follows: “It’s quite chunky, but looks quite interesting…You can fit it on a key ring…How do you switch it on?…This is actually quite hard to figure out how to use…I don’t like it, you can’t really navigate through it…[shaking head] I don’t understand it…I give up!” Kate had never heard of the brand (iRiver) and summarized her experience with this device as “strange,” and used words like “confused” and “complicated” during the post-interaction interview.
In the same study, participants reviewed a device by a well-known manufacturer (Sony) and found signs of low usability and construction quality issues. One participant, Steve, said, “The hold function is quite cool, although I imagine it would break quite easily… seems liable to come off when you don’t want it to.” After interacting with this device, participants didn’t want to give it low ratings, suggesting they overlooked their difficulties. They concluded by saying it was a “good” MP3 player.
What’s Going On?
One way to understand Betty’s behavior is to attribute it to her motivation to reduce cognitive dissonance in which a person holds conflicting views. This analysis, however, would be less than helpful to a researcher running a usability study, and it could mask valuable data.
In this example, additional questioning around other aspects of Betty’s life revealed pertinent information. It turned out that she enjoyed sharing music with a close friend who received pre-release music. Because her friend set up his gadgets in a way she thought made it easy to share music via an iPod, she was highly motivated to get an iPod. For her, having early access to pre-release music correlated with her strong sense of personal identity with music—“Music is what I am about.”
The contrast between the Sony and iRiver devices gives us interesting clues. Participants seemed to be less tolerant of the iRiver device than of the Sony device. This is possibly because most of the participants had not heard of iRiver, while all had heard of Sony and considered it a “good and reliable” brand.
We believe it may be easier for users to continue with the same opinion rather than change it, even when faced with contradictory evidence. People have an intricate web of implications built into their opinions, and changing one could mean revisiting and possibly modifying subsequent ones. Based on Kelly’s Personal Construct Psychology, researcher Dennis Hinkle suggested that the more meaningful aspects of people’s experience are those that have the most implications. Anticipations with deeper ranging implications may be more likely to influence user preferences and ratings. In the above examples, Betty is fighting a conclusion not to buy an iPod because the implications of not getting one jeopardize an activity that is important to her. On the other hand, users who have not yet formed a consequential opinion about an iRiver device might easily change their opinion.
Paradoxes, such as Betty and Steve maintaining opinions that ignore experiential evidence, should always warrant further research. Ultimately, we found that these paradoxes and resistance to change were related to a network of implications. Episodes like Betty’s story provide a researcher with important clues towards discovering how a user is experiencing not just their technological world, but also their position within this world. This directly influences how the user will experience and judge the artifact during the usability test.
When Does Interaction Really Occur?
In these cases, interaction took place long before the research study: seeds of expectation were sown during the early experiences with different brands—both well-known and not—beginning with indirect experiences that may include watching other people use the items, or viewing television advertisements. If a user’s expectations and anticipations are not uncovered in the recruiting screeners, or if usability studies do not explore the implications behind opinions given during a testing session, unexplained contradictions may occur in usability findings.
Uncovering the experiences and expectations people have before they actually use the technology in the usability test is just as important as exploring the physical interactions that occur during a test session. We used semi-structured interviews to explore participants’ relevant experiences, and we paid attention to both direct and indirect experiences. I queried Betty further, saying, “This is really interesting, and seems like a contradiction. What do you think might be going on?” Suddenly Betty and I were looking for answers together, rather than her feeling that she was being interrogated. By positioning it this way, we invited her to be a co-explorer, uncovering the deeper motivations behind her opinions. We recommend that questions about a user’s pre-interaction experience become part of the experimental protocol.
Many factors influence user experience. One important factor is that what is usable is also a matter of implications. Users seem to forgive bad design if the implications of calling it “bad design” have multiple repercussions. The above stories show how users can ultimately forgive design trespasses if changing their perception about the brand has deeper implications for their self-perception. Further studies are needed to determine the threshold beyond which consumers will finally revolt against bad design.
Our research suggests that we should always explore why users seem to forgive bad design. Our findings become richer when we examine seeming contradictions. By maintaining a holistic view of user experience, we gain a fuller understanding of how people experience technology, and enrich our model by highlighting implications, self-identity, and convenience as core themes in user experience.为什么向朋友推荐不完美的产品可能是合理的？移动媒体播放器的用户研究揭示的一些事例会让我们看到一系列蕴含关系，表明此类行为是完全理性的。同时我们发现，如果只参考可用性研究中的定性或交互数据，而不去研究参与者的历史，可能造成有误导性的狭隘观点，并丢失丰富的研究结果。
文章全文为英文版결함이 있는 제품을 친구에게 추천하는 것이 왜 이치에 맞는 것일까요? 휴대용 미디어 플레이어에 대한 사용자 조사 세션에서의 몇몇 예를 보면 이러한 행동이 완전히 이성적이라는 것으로 보이게 하는 일련의 연관된 관계성을 볼 수 있습니다. 즉 참가자의 이력을 살펴보지 않고 사용성 연구의 정성적 또는 상호작용 데이터에만 격리된 주목을 하게 되면 오해를 할 수 있는 좁은 관점을 가지게 되어 풍부한 연구 결과를 놓칠 수도 있다는 사실을 우리는 알게 되었습니다.
전체 기사는 영어로만 제공됩니다.Ao olharmos alguns exemplos de sessões de pesquisa com usuários explorando tocadores de mídia portáteis, é possível destacar uma série de implicações que mostram esse comportamento como sendo completamente racional. Descobrimos que analisar isoladamente os dados qualitativos ou de interação em um estudo de usabilidade, sem explorar o histórico de um participante, fornece uma visão limitada que pode ser enganosa, além da possibilidade de perder importantes descobertas.
O artigo completo está disponível somente em inglês.欠陥商品を友人たちに薦めることがなぜ、理にかなっている可能性があるのか。携帯メディアプレーヤーに関するユーザ調査のいくつかの事例から、そのような行動が実は全く合理的なものであると示すことができる。調査対象者の履歴を考慮せずに、ユーザビリティに関する定性的データやインタラクションデータを分析することは、視野を狭めて誤解を招いたり、貴重な発見の可能性を見逃してしまったりすることにつながると分かった。
原文は英語だけになりますAl revisar algunos ejemplos de sesiones de análisis de usuarios navegando en reproductores multimedia portátiles, es posible resaltar una red de implicancias que identifican dicha conducta como algo totalmente racional. Hemos hallado que el hecho de tener una visión aislada de los datos cualitativos o de la interacción en un estudio de usabilidad sin explorar el historial de un participante, arroja una mirada acotada que puede ser errónea y que puede llevar a dejar de lado conclusiones significativas.
La versión completa de este artículo está sólo disponible en inglés.