As many UX professionals know, unexpected issues seen in the context of actual deployment can trump an initial rosy impression about the usability of a system. These kinds of surprises can be especially dramatic when working in the area of ICT4D (Information and Communication Technology for Socio-Economic Development), where technology applications are designed to alleviate poverty among economically disadvantaged populations.
This article describes a case study of the deployment of a prototype low-cost digital slate for assisting the tracking of child malnutrition in rural India. The prototype runs on Windows CE and looks like a small clipboard with a calculator at the top (see Figure 1). It accepts handwritten input on ordinary paper notebooks placed on the A5 size-digitizing pad of the slate, and provides immediate electronic feedback on the 3.5 inch interactive touch screen display. The slate captures writing from a digital ballpoint pen as raw strokes, and the back of the pen serves as a stylus for touch screen input. The digital slate system allows for a fairly seamless transition from older, paper-based systems to digitally recorded information. In addition, the slate preserves the paper copies of all forms; this is particularly valuable in developing world contexts where paper records are still very important. We designed the system for people who have little or no experience working with digital systems beyond simple cellular phones or calculators.
Prior work, coupled with our initial controlled user trials, suggested that users strongly preferred the current design to other alternatives, and were eager to use it in the field. But in a follow-up three-month field trial with ten low-income, low-literate health workers in rural central India, a number of unexpected challenges were encountered that overwhelmed our initial optimism. This experience demonstrates that more attention needs to be paid to issues well beyond the simple usability of technological systems, to include broader socio-technical concerns that arise in real-world settings.
Previous Work
In previous explorations, we had tested the slate with low-income users in a rural microfinance setting. India’s extensive microfinance network brings formal savings and credit services to 86 million poor households. Yet, the inability to maintain high-quality records remains a persistent weakness in the smooth functioning of microfinance groups. We worked with a non-profit organization that facilitated microfinance loans among self-help groups. They faced a range of problems with their paper forms, including errors in arithmetic calculations and legibility issues during transcription, to a computerized management information system (MIS). Often, certain mandatory fields in the paper forms were left empty. Interruptions in the physical transport of the paper forms from the meeting to the transcription location and back caused further delays in resolving these errors. We studied these problems and built a financial record management application on our prototype. In a two-day supervised trial with 200 microfinance group members in rural eastern India, we saw that the use of the slate solution resulted in shorter data recording time, fewer incorrect entries, and more complete records, compared to the existing paper-based system. Furthermore, the slate device won strong approval from the microfinance groups and partner organization we were working with.
Adaptation to Health Context
With the success of this experience, we began looking for other potential applications for our prototype. We were approached by a non-governmental organization (NGO) that manages a malnutrition tracking and treatment program for about 65,000 children under the age of five across five districts in rural central India. The Real Medicine Foundation (RMF) was facing persistent problems with long delays in aggregating per-child data in their existing paper-based system. Local health workers record child-specific data in paper diaries during household visits, and this information is subsequently transcribed into ledger books that are eventually aggregated into a computerized MIS. It takes about a month for the information collected by health workers to make its way to decision makers, resulting in serious delays in remedial actions, such as providing emergency food rations and medical care.
We worked with RMF to build a .NET Compact Framework health data record management application on our slate prototype. The application’s design matched the format and workflow of the existing paper forms and diaries. All text labels and text field entries were in Hindi, the language spoken by the health workers. Based on the pen’s location on the digitizing pad, the application detected the cell of the paper notebook that was being filled. Each handwritten numeric digit printed on paper was simultaneously digitized, run through a digit recognizer, and placed in the corresponding field on the screen. We saved all the data on the slate’s micro SD card, which was transferred to the health worker’s phone and sent to the backend database via general packet radio service (GPRS) at the end of each day. The server was updated daily, and a summary of the data was made available to decision makers. Finally, as noted above, the slate seamlessly provided a paper trail and backup, which was seen as a critical feature to RMF.
A one-day training and evaluation session of the application was conducted with ten health workers, which was very encouraging. All of the health workers were women who were from the small town of Khandwa and surrounding villages in the district, with household incomes between USD $80-150 per month, and relatively little education. All had at least some exposure to technology (four of the women owned a mobile phone, and the rest had received calls on a family member’s mobile phone at least once).
The slate application was demonstrated to groups of five and had each worker practice individually for an hour. We were excited to see that workers quickly picked up the familiar pen and paper interaction and were very enthusiastic about it. Our partner NGO was equally pleased by our application and wanted to deploy the system. Following the success of the slate prototype in the microfinance context, this was the second feather in our cap and we were eager to see how our prototype could help fight child malnutrition in India!
Buoyed by the positive results, we deployed our prototypes with the same ten health workers for a field trial. For three months, the health workers were to visit individual rural households and use our system to record child-specific data. At least, that was the idea. Instead, the initial optimism we felt from the training was quickly overwhelmed by a number of challenges that rendered our prototype unusable.
Challenges Encountered
1. Application usability
Despite laboratory testing of the software, there was a limit to the real-world usage scenarios we could simulate in our lab. This was all the more challenging because the literacy level and information and communication technologies (ICT) experience of our end users was very different from the software testers. One worker from a remote village once accidently clicked on the Windows Media Player icon on the desktop instead of our application icon, and was not able to use the application for three weeks because no one in her village knew how to get back to the desktop screen. On a number of occasions, workers accidently closed the application without saving data they had entered. Workers often had problems using the stylus to double-click and ended up executing various functions that they were not trained to use. They landed on unfamiliar screens and did not know how to make their way back to the application. Given our limited time and resources in the field, we were not able to observe and design for such scenarios. Based on observations from the training session we did introduce some changes while in the field, but there were many issues we were unable to foresee.
2. Physical infrastructure
There were problems associated with the poor physical infrastructure of rural India. In some of the remote villages where the health workers lived, there were regular power outages of twenty hours throughout the day, with wildly varying voltage levels. The slate device required a power connection for around six hours to fully charge the battery, so devices were often not completely charged. On some occasions, the batteries died while using the device at patients’ houses, which seemed to embarrass the health workers.
3. Device management
According to the health workers, the odd size of the slate made it awkward to transport. Many workers traveled for work in very crowded public buses and found it difficult to manage the device in the rush; the only ICT they carried with them were mobile phones which easily fit into their handbags. Others traveled to work on the back of their husbands’ bicycles through muddy, uneven village roads. Bicycle crashes were very common, and workers were worried that they would break the device when they fell. Finally, our trial coincided with the monsoon season; in spite of multi-layered packing in plastic sheets, the health workers were concerned that water would seep through and spoil their devices during transport.
4. Human
Finally, there were a host of human-centered issues that worked against a successful trial. While our NGO partner helped us with many logistics in the field, they had very little technical competence. When health workers experienced technical glitches on the device, they would bring it to their sole point-of-contact, the program coordinator of the organization. However, the coordinator typically lacked the expertise and training to fix the glitches. While she could reach out to us over the phone for remote troubleshooting, she was reluctant to do this. We suspect that her lack of enthusiasm may have been partially due to the fact that she was not particularly vested in the project. We had not involved her in the initial training program, and so she had little or no ownership in the success of the slate intervention program.
A couple of the health workers tried seeking help from local mobile phone shop owners and photo studio owners with computers. However, due to lack of knowledge about the digital slate device, they were turned away from there as well.
Conclusion
The experience with our prototype in rural India exposed a number of broad challenges in deploying a promising technological system into a real-world setting. We believed our initial, limited-user trials had shown that our slate solution was an unqualified success. However, like many travelers before us, we found that the road to success is not so simple. During actual deployment of the system in the wild, we experienced a number of unprecedented challenges that marred our initial optimism. We conclude that there needs to be deep attention beyond simple usability to broader socio-technical issues that may arise in field deployment of technological systems.许多 UX 从业者都知道，在部署新设备或者服务的过程中遇到的意外状况可能会毁掉用户对系统可用性最初的乐观印象。在 ICT4D（面向社会经济发展的信息和通信技术）领域中工作时，这些意外可能更具有戏剧性。通过这篇文章我们描述了一个产品部署的案例。为了帮助跟踪印度农村儿童营养不良的状况，我们在印度农村发放和部署了一款低成本的数字书写板产品原型。我们的原型包括钢笔和普通纸张以及一个可以交互的触摸屏。用户可以通过这个原型系统实现从基于笔和纸张的传统系统到数字信息记录的相对平滑的过渡。这套系统是为没有数字系统（手机和计算器除外）使用经验或经验很少的人设计的。
我们以印度中部农村地区的 10 名低收入、低文化水平的卫生工作人员为样本，对该系统进行了最初的培训和评估。在此过程中，该系统看起来似乎大获成功。但是，在与相同用户进行为期三个月的后续现场实验时，我们遇到了大量未曾预料的难题，这打消了我们最初的乐观。这一经历表明，除了考虑可用性外，我们必须投入更大精力关注将这些系统应用于现实世界时会遇到的社会技术层面的问题。
文章全文为英文版많은 UX 전문가들이 알듯이, 실제 출시의 상황에서 볼 수 있는 예기치 않은 문제가 시스템의 사용성에 관한 초기의 낙관적인 느낌을 단번에 바꿀 수 있습니다. 이러한 예기치 않은 상황은 ICT4D(사회 경제 개발을 위한 정보통신기술) 지역에서 작업할 때 특별히 더 두드러집니다. 본 논문은 인도 농촌의 어린이 영양실조 추적을 지원하기 위한 저가 디지털 슬레이트 프로토타입 배포에 관한 사례 연구를 설명합니다. 우리의 프로토타입은 잉크펜과 보통 종이 그리고 디지털 정보 기록을 위한 더 오래된 종이 기반 시스템으로부터 상당히 매끄러운 전환을 허락하는 인터랙티브 터치 스크린을 사용합니다. 이는 간단한 휴대전화나 계산기 이상의 디지털 시스템을 이용한 경험이 적거나 없는 사람들을 위해 디자인되었습니다.
인도 중앙 지역의 저소득, 저학력 농촌 보건 요원 10명이 참가한 초기 훈련과 시스템 평가는 완전한 성공으로 간주되었습니다. 하지만 같은 사용자들이 참가한 3개월간의 현장 추적 실험에서는 우리의 초기 낙관주의를 압도해버린 예상치 못한 몇몇 어려움을 발견하게 되었습니다. 이 경험은 우리가 단순한 사용성을 훨씬 넘어서 이러한 시스템을 현실 세계의 무대로 배치하는 과정에서 발생하는 사회-기술적 우려 사항과 같은 문제에 매우 높은 관심을 기울여야 한다는 사실을 보여줍니다.
The full article is available only in English.Como muitos profissionais de experiência do usuário sabem, problemas inesperados vistos no contexto de implantação real podem exceder uma impressão otimista inicial sobre a usabilidade de um sistema. Essas surpresas podem ser especialmente dramáticas ao se trabalhar na área de ICT4D (Tecnologia da informação e comunicação para o desenvolvimento socioeconômico). Descrevemos um estudo de caso do desenvolvimento de um protótipo de tablet digital de baixo custo para auxiliar no monitoramento da desnutrição infantil na Índia rural. Nosso protótipo usa uma caneta esferográfica e papel normal acoplados a uma tela touch screen interativa, que permite uma transição razoavelmente contínua dos sistemas antigos baseados em papel para o registro de informações digitais. Ele foi projetado para pessoas com pouca ou nenhuma experiência em trabalhar com sistemas digitais além de simples telefones celulares ou calculadoras.
O artigo completo está disponível somente em inglês.多くのUXの専門家が知っているように、実際の導入の際に発生する予期していなかった問題というのは、システムのユーザビリティに関する当初の楽観的な見通しを一瞬にして暗転させてしまう場合がある。これらの予想外の出来事は、ICT4D（情報通信技術を生かした社会経済開発）の分野に携わる場合には、特に激しいものになりかねない。我々は、インドの農村地域で子どもの栄養不足の追跡に役立つ低コストなデジタル石板のプロトタイプを導入したケーススタディについて説明する。我々のプロトタイプはインクのペンと普通の紙をインタラクティブなタッチパネルと組み合わせたもので、従来の紙を使ったやり方からデジダルでの情報記録へのシームレスな移行を実現できるものである。これは、これまでにシンプルな携帯電話や電卓以外にはデジタルなシステムをほとんど、あるいはまったく使ったことのない人々に向けてデザインされたものだった。
初期段階でのシステムのトレーニングと評価は、低所得で、識字率が低い、インド中部の農村地域で働くヘルスワーカーの10名に対して行われ、文句なしの成功が得られたように思えた。しかし、同じユーザを使った3か月におよぶ追跡現場調査では、我々の当初の楽観的な見方を一転させるほどの予期しない沢山の問題が発生した。この経験は、我々が単なるユーザビリティにとどまらず、幅広く社会工学的な観点に多大な注意を払う必要があることを示すものである。
原文は英語だけになりますComo muchos profesionales especializados en UX saben, inconvenientes imprevistos observados en el contexto de una implementación real pueden ensombrecer una buena primera impresión sobre la usabilidad de un sistema. Estas sorpresas pueden ser especialmente decisivas cuando se trabaja en el área de ICT4D (Tecnologías de la Información y las Comunicaciones para el Desarrollo Socio-económico). Describimos un caso de estudio sobre el desarrollo del prototipo de una pizarra digital de bajo costo destinada al control de la desnutrición infantil en zonas rurales de la India. Nuestro prototipo utiliza un bolígrafo y papel normal junto a una pantalla táctil interactiva que permite una transición casi transparente de los sistemas más antiguos del papel a los de registro de información digital. Se diseñó para personas con poca o nula experiencia en el trabajo con sistemas digitales, más allá de simples teléfonos celulares o calculadoras.
La versión completa de este artículo está sólo disponible en inglés.
