As more companies look to ensure that their products can be used by people with disabilities, usability professionals are challenged to broaden their testing repertoire. An application or website may be designed and coded to meet accessibility standards and may also test adequately for general usability. But it still may not be usable by people with disabilities. You won’t know until you test your application with these users, too.

There are special considerations for user and task analysis, recruiting, and data analysis when testing users with disabilities. Of course, your test facility must be accessible and provide the required assistive technologies. But, usability testing with people with disabilities (PWDs) can be accomplished with the same “discount” methods that you may already be using to test people without disabilities.

The benefits of such testing extend beyond merely accommodating persons with disabilities. As people age, they often develop vision, hearing, dexterity, mobility, and cognitive issues. Any testing done with people with disabilities will benefit aging users as well.

Getting Started

As with usability, product development should address accessibility from the start, and not try to patch things up with some testing at the end of the development cycle.

Practitioners should start with design and coding to meet accessibility standards, particularly national laws, such as Section 508 in the U.S. (www.section508.gov/) and WCAG 2.0 (www.w3.org/TR/WCAG/) guidelines. There is really no point in usability testing until the basics are covered to provide, for example, keyboard-only usage, text equivalents for images, and captioning for videos.

Once fundamental accessibility guidelines are met, you should proceed as usual with testing by people without disabilities. If non-disabled users have problems using your product, these problems will be magnified for people with disabilities. Do users tend not to see links on the far right of your web application? Screen magnifier users may never find them (screen magnification means horizontal scrolling which means more work to use your application.) If the links are critical, consider moving them to the upper left.

User Profiles

The user profiles or personas you create for testing people without disabilities also apply to people with disabilities. However, you need to give some consideration to the types of disabilities you need to test for.

The broad categories of disabilities are:

• Vision (blindness, low vision, color blindness)
• Hearing (deaf or hard of hearing)
• Speech and voice-related
• Manual dexterity (for example: unable to use a mouse)
• Mobility (for example: wheelchair users)
• Cognitive disabilities (for example: dyslexia, autism).

Very likely, you can immediately eliminate some of these categories. For example, if your product does not present audio information or require users to speak, it probably poses no barriers to people with hearing and speech disabilities.

Both dexterity and cognitive disabilities are wide-ranging and difficult to define. Participants with these disabilities also tend to be difficult to recruit (more on recruiting later). But, if your user analysis reveals that people with a specific disability will be using your product, for instance, users with dyslexia, you should include this group in your testing.

Often, however, a general purpose application or website can be adequately covered by testing blind and low-vision persons who use screen reading and screen magnification assistive technologies (ATs). The JAWS screen reader from Freedom Scientific (www.freedomscientific.com/) and the ZoomText screen magnifier from Ai Squared (www.aisquared.com/) are the most widely used. While vision impairment is only one category of disability, it comprises a populous group with widely used ATs. As guerrilla tactics, testing by this group with those technologies will get you a good bang for your buck.

In addition, screen readers depend on keyboard access, so you can often extrapolate those results to people who use the keyboard only, or who use other keyboard-based assistive technologies such as joysticks or trackballs.

Task Analysis

As with user profiles, the critical tasks for your application will apply to users both with and without disabilities. However, assistive technologies such as screen readers and magnifiers add a level of complexity to the use of your application. This means more time is likely needed to perform tasks that can lead to greater user fatigue in test sessions.

As a rule of thumb, allow twice the time for task performance allocated to non-disabled users, but try to keep the entire session under two hours. Ideally you should use the same tasks for testing by users with and without disabilities to provide some basis for comparing results. But be ready to pare the task list down to fewer or simpler tasks. If possible, conduct a dry run with assistive technology users, especially with screen readers.

Recruiting: The Hard Part

The great advantage to guerrilla testing is that you can get results with three to five test participants per user profile. This is true for participants with disabilities, but you must also recruit for each type of disability; each category represents a different user group. So, if you have two user groups and you are testing your application with both blind and low-vision users, you need to recruit twelve to twenty participants. As always, schedule a few extra participants in case of last-minute cancellations, and keep a list of potential future participants.

In addition to recruiting more users, recruiting users with disabilities may require contacting organizations and agencies that provide training and other services to people with disabilities. You want to start making these contacts as early as possible, at least four weeks before you plan to begin testing.

If you are recruiting participants with dexterity or cognitive disabilities, begin establishing contacts with agencies even earlier—six to eight weeks before testing. It’s just more difficult to find enough participants with these disabilities.

To establish a relationship with an organization, school, or agency, you should plan to make an in-person visit with the director. These organizations are very protective of their clients and will want to make sure you will be as respectful as they are. That said, you may find that the organizations already have some experience with usability testing and will welcome the chance to work with you, both to test new software for accessibility and to provide an opportunity for their clients.

In the U.S., agencies are prevented by law from providing their clients’ contact information. However, they will likely agree to send an e-mail to their clients, asking volunteers to contact you. Offer to compose the e-mail so that you will be contacted by people who meet your user requirements. Be brief but specific about your requirements, and mention the compensation.

People with disabilities are generally under-employed and therefore likely to respond to opportunities that promise compensation. Interview them closely by telephone to verify that they meet your requirements and are experienced in using their assistive technology, as well as meeting the profile you require.

When scheduling the test sessions, allow extra time at both ends; transportation can be a big problem for people with disabilities. Many rely on special van services whose schedules may not be reliable. A thirty minute cushion, at both ends, is usually adequate.

If you work for a large company, you may be able to recruit participants internally through corporate channels. This may save you compensation expenses and you may be able to run the sessions at participants’ desks with their own computer setups. However, beware of using the same participants over and over for different testing projects. They may become more adept in running your tests, and less effective as participants.

Test Facility

Your test facility must be accessible. You will also need computers with appropriate AT set up on them.

When you start contacting organizations for recruiting, ask if they have a facility you can use. When you make an in-person visit, you can see if they have a suitable room. If you use an agency’s facility, it will already have accessible ramps, elevators, and other accommodations.

If they provide assistive technology training, they may be able to let you use a classroom, at a nominal rental fee, for testing, including desktop machines with the ATs already installed. Ready access to up-to-date ATs can be a real time and money saver.

Agencies are unlikely to have video capabilities, so you should plan to bring your own video equipment. Test your recording equipment before the test. You may want to use a video camera instead of, or in addition to, screen capture software, so you have a picture of the whole interaction.

Testing

Once you have the facility ready and the participants scheduled, the test sessions will be familiar. Even if the facility is accessible, if participants are not familiar with it you may want to post someone at the entrance to escort participants as needed. Provide a waiting area in case participants arrive early or are accompanied by another person.

The testing room should have wide, clear aisles to accommodate wheelchairs or service dogs. Give the participants time to check or make settings in the assistive technology they will be using.

Useful resources for running the test include:

• Conducting a test (www.uiaccess.com/accessucd/ut_conduct.html)
• Service dog etiquette (www.newhorizons
  servicedogs.org/service_dogs.shtml)
• Appropriate terminology (www.disapedia.com/index.php?title=Guide_to_Appropriate_Terms)

Data Analysis

Task time will be affected by the assistive technology used because it adds complexity and generally slows performance. Look at success rates, error rates, and satisfaction. Then look for patterns across the disability groups.

Conclusion

Testing your software using people with disabilities provides a richer understanding of software usability and leads to more usable and accessible software for all users—disabled, non-disabled, and seniors. While some testing tasks, primarily recruiting, will present challenges, the tactics of guerrilla or discount usability testing can be a useful addition to your testing repertoire