In traditional usability testing, the participant is trying to accomplish two things at the same time: learning to use an unfamiliar product and thinking aloud. While this method might work well for adults, it doesn’t fully represent children’s natural behavior of learning through play. It also doesn’t take into account the fact that multi-tasking skills do not develop until later years, and a concurrent verbal protocol can seem very artificial to a child. Not to mention that a child might feel uncomfortable talking to a stranger.
So how can we better engage and encourage children in a usability test situation?
We found Peer Tutoring to be well suited for usability testing with children. The method originated from educational sciences and is commonly used in schools. In Peer Tutoring, a student helps a peer learn something new, and at the same time, that student learns as well (see Figure 1). An important benefit of the method is that the children establish a rapport because the tutor naturally gives advice to the peer using his or her own language. Also, there is equality in authority and knowledge between the peers, which can make their communication easier and livelier.
Over the past year and a half, we’ve tested a number of digital LEGO products using Peer Tutoring. We’ve found that the biggest difference between this method and the traditional usability test is in session moderation: an adult moderator interacts with two children instead of one, gently facilitating the children in an unobtrusive way. This leaves more space for the children’s own spontaneous dialog and discovery. One of the highlights we experienced with Peer Tutoring occurred fifty minutes into a session, when an eight-year old girl (the tutor) asked the moderator when the actual interview was going to start, because her mom told her she would have to answer some questions.
Planning a Peer Tutoring Usability Study
When setting the objectives for a Peer Tutoring study, we typically have the same goals as we would for a traditional usability study. We still want to know how useful the product is to the target group, which features are “cooler” than others, how fast the children are able to learn to interact with the product, and how satisfied or happy they seem after using the product.
The unique aspect of study planning is in our participant screening criteria. When recruiting dyads for each session, we look for children who are the same age and who already know each other. To prevent too much similarity in responses, however, we try to make sure that the children are not best friends.
Moderating Test Sessions
After the two friends and their parents have arrived at our office, we explain what will happen during the session, and ask the parents to provide written permission for the children to participate in the study. After assigning roles to the two children, we tell the tutee, “First, your friend will try out something new on the LEGO website, and then we’ll come out and get you so that he can show you what he did on the computer.” The tutor and the moderator then go into the test room.
The tutor can try everything out and provide any comments he wants to, but he doesn’t have to think aloud. The moderator tries to guide the child as little as possible to prevent bias from occurring, but if the child is not spontaneously using the features we want to test, the moderator encourages him to use them. At some point the moderator asks the tutor, “Do you think that you can explain this tool to your friend now? Or do you have any questions?” As soon as the child thinks he is ready, we invite the second child into the test room.
Now the tutor starts teaching his friend by showing what he just did on the computer. In a study of a Comic Builder tool that allows users to build their own comics (see Figure 2), a conversation between the two participants might sound like this:
TUTOR: That’s the LEGO website. And there [pointing on the screen] you can make your own comic. I just did one.
TUTEE: Where do I start?
TUTOR: First, pick one of these [pointing to the library with the background images]. See, you can zoom in and out if you want.
TUTEE: Where? [trying to zoom by double-clicking on the image] This doesn’t even work!
TUTOR: It does. It’s there, this one with the brick. But I didn’t know how to do that too at the beginning.
The moderator observes the children and acts as a helper in case they get stuck. If the tutor forgets his role, the test moderator prompts him to help his friend, “Can you tell Sarah something about the blue ribbon there? Remember? You used it before.”
After the session, the moderator asks the children a few general questions related to satisfaction and enjoyment. Also, the participants can ask questions about the product we’ve just tested or about any other LEGO products.
Analyzing the Results
All sessions are recorded for internal use. We transcribe the videos afterwards to accurately track how many participants were able to use a particular feature. The videos are also used to trace children’s behavioral workflow when interacting with our product, and where the product did not meet their expectations (as in the comment above, “This doesn’t even work!”).
Based on the study results, actionable design recommendations are made to the internal client commissioning the study, usually a producer or project manager. Since the producer or project manager is involved in planning the study and usually also watches most of the sessions, the results and recommendations aren’t a surprise.
Want to Use Peer Tutoring with Your Product?
- Peer Tutoring works best when testing software applications or games that have a rather linear workflow with a clear beginning and end. The example used in the article was a LEGO Comic Builder tool. At the beginning of the session, the children are presented with an empty sheet on the screen. By the end of the test, they have built their own comic. This means the children had to follow certain steps to get to the predetermined goal.
- If the tested product is a website, the moderator has to make the pre-defined task scenarios or the features she wants to test obvious to the tutor, if the tutor is not using them spontaneously.
- When selecting tutor and tutee, choose the more talkative child as the tutor. We usually put this in our screener questionnaire we send to the recruiting agency. However, we might also determine the tutor shortly before the test session, when we’re talking to the parents and children in person.
- Make sure that the children remain focused on your product instead of playing around together. The moderator might have to gently redirect, “It looks like you’re done. What do you think about this game (or website)?”
- Try to make the test session interesting to the children. Peer Tutoring might not be ideal for simplistic task scenarios or products, as the tutor might not want to teach a friend about something too obvious.
- Avoid using abstract wireframes or sketches. The higher the fidelity of the testing materials, the more concrete the responses you will receive.
Conclusions
Peer Tutoring is a useful usability evaluation method when children are the end users of a product. By taking into account the cognitive skills of the participants and the natural way in which children learn, Peer Tutoring elicits more self-motivated user comments than other techniques, and makes it particularly easy to evaluate engagement. Because the tutor is testing the product by teaching it to his friend, we are able to assess learnability in a very natural and elegant way. If the tutor is not able to teach something to his friend, we can easily see where our software must be improved.
同伴辅导是 LEGO 集团已经在进行儿童可用性测试中成功使用的方法。在同伴辅导中，一名儿童充当辅导员，用自己的语言向其朋友介绍一款数字产品。因为是一名儿童教另一名儿童使用产品，并且另一名儿童是边学边用，所以观察人员就可以获得关于儿童如何看待和使用产品的宝贵信息。两名儿童之间作为朋友的社交因素让测试环境更自然，从而产生更丰富、更外显和真实的发现。
The full article is available only in English,또래 교사(Peer Tutoring)는 어린이와 함께 사용성 평가를 할 때 LEGO 그룹이 성공적으로 사용한 방법입니다. 또래 교사 방법을 통해 한 아이가 선생님 역할을 하며 그 아이 고유의 단어로 디지털 제품을 친구에게 설명합니다. 한 아이가 다른 아이에게 제품에 대해 가르치며, 배우는 동안 다른 아이는 제품을 사용합니다. 관찰자는 어린이들이 제품을 어떻게 이해하고 사용하는지에 관한 소중한 정보를 얻게 됩니다. 두 친구라는 사회적 배경이 테스트 상황을 더욱 자연스럽게 하여 더욱 풍부하고 외부적으로 소중한 연구 결과를 얻게 됩니다.
The full article is available only in English.A tutoria entre pares é um método que o grupo LEGO utilizou em testes de usabilidade com crianças. Na tutoria entre pares, uma criança atua como tutora e explica, em suas próprias palavras, um produto digital a seu amigo. À medida que uma criança ensina o produto à outra criança, e à medida que a outra criança usa o produto enquanto está sendo ensinada, os observadores recebem informações valiosas sobre como o produto é percebido e utilizado por crianças. O contexto social dos dois amigos faz com que a situação do teste seja mais natural, resultando em descobertas mais ricas e mais válidas externamente.
O artigo completo está disponível somente em inglês.LEGOグループは、子どもを対象にユーザビリティ・テストを行う際、友達同士での指導という方法を使って成功を収めてきた。友達同士での指導では、一人の子どもが指導者となり、自分の言葉を使って、友だちにデジタル機器について説明を行う。ある子どもが別の子どもに製品について教え、教えられた方の子どもは、教えられながら製品を使用し、観察者はそこから子どもたちがどのように製品を認識し、使用するのかという点について貴重な情報を得る。友だち同士という社会的状況がテストの場をより自然なものとし、内容が濃く、それ以外の場面でも通用する結果を得ることができる。
The full article is available only in English.La tutoría entre pares es un método que el grupo LEGO ha utilizado satisfactoriamente al realizar tests de usabilidad con niños. En la tutoría entre pares, un niño actúa como tutor y le explica, con sus propias palabras, un producto digital a su amigo. Mientras un niño le enseña el producto a otro y mientras ese otro niño utiliza el producto al tiempo que le enseñan, los observadores reciben información valiosa sobre cómo los niños perciben y utilizan el producto. El contexto social de los dos amigos hace que la situación de prueba sea más natural, con lo que se obtienen datos más significativos y válidos desde un punto de vista externo.
La versión completa de este artículo está sólo disponible en inglés.
