Our user experience (UX) team at Nationwide collaborated with an internal business team within Nationwide Financial on an ongoing project. Together, we adopted the Rapid Iterative Testing and Evaluation (RITE) methodology. The RITE methodology is credited to Michael Medlock who used this technique in association with game design. The Nationwide UX team has been experimenting with RITE for two years with a variety of financial services websites and applications.
Project Background
The Fund Management Redesign (FMR) project is an ongoing effort that has engaged the UX team in the past on various ethnographic studies concept testing and prototype testing. The project centered around pages on a 401(k) management site. A 401(k) plan is a type of employer-sponsored retirement account to which employees may contribute a portion their wages to withdraw at retirement. The money is placed into a variety of investment options and participants are typically allowed to make changes to the amount of money and into which funds they invest. The 401(k) management site in this project allowed participants to make these types of selections online.
As the combined team entered into a new redesign phase, the UX team suggested RITE as an approach to yield quick results at an early stage while allowing us to continually respond to feedback and make changes. The prototype tested consisted of low-fidelity, easily modifiable wireframes. We ran two participant sessions per day for five days, with a three-day gap after the third day. This allowed time for making quick changes to the prototype between tests, as the business and UX team reviewed findings on an ongoing basis.
In each testing session, the participant started at the landing page for a section of the site where they could make changes to their investments. This page had four main sub-sections, each offering a different method of instituting changes and requiring a different level of involvement on the part of the 401(k) plan participant. Each participant selected the method they wished and made changes to their portfolio. We then instructed them to do the same via the methods in each of the other three sub-sections. Throughout the process, we tracked their expectations, understanding, level of comfort, and ability to complete the tasks efficiently and correctly. We used six-point Likert scales (agree, strongly agree, etc.) administered after they explored each sub-section, asked probing questions, and ensured that the facilitator made careful observations. We continually made changes to content, images, layout, organization, and process flow to improve them.
Benefits of RITE
Three features of the RITE methodology helped us succeed on the FMR project:
- We created the prototype as an Adobe Acrobat (PDF) file with linked pages, exported from wireframes built using Adobe InDesign. Because no visual design or development work had yet to be done, changes to content, layout, and specific elements could be made directly by the UX lead assigned to the wireframes, who was present at all the testing sessions.
- We received feedback immediately because the UX lead and members of the business team were present for all sessions. On most days, we did a recap with the test facilitator so that all stakeholders were aware of what we learned in testing and could approve changes without having to wait for a report to be released. Since the prototype was altered quickly and often, each set of participants was able to offer unique feedback on content not previously viewed. This allowed the team to receive more feedback on more pages than they would have received from the same amount of traditional usability testing.
- The prototype evolved over time. The amount of feedback received, combined with how easily modifiable the prototype was, allowed for changes to be incorporated quickly, continually validated, and refined. Over the course of the ten participants, we developed and tested six different versions of the prototype. This allowed the prototype changes to follow a progression of continual improvement. The changes between the start and end prototype were significantly more substantial than we would typically have been able to be make in a single round of standard usability testing across the same number of participants. For example, Figure 1 shows the initial Change My Investments landing page, and Figure 2 the final version.
Due to the advantages of the RITE methodology, the project team was able to generate many actionable recommendations from the testing. These learnings were then applied as the project progressed.
Practical Considerations
- We found that the RITE methodology is best used early in the project lifecycle. It is most appropriate when dealing with early concepts that need to be vetted with users, as it can assist in quickly shaping designs.
- A benefit of RITE is that it reduces the risk of running later stage validation tests that identify the need for major changes. Risk is mitigated by early feedback obtained with low-fidelity concepts.
- RITE does not lessen the value of conducting validation testing later in the project lifecycle. Adding visual treatment to the wireframes and creating a much richer interactive prototype introduces elements that still need to be vetted with users. RITE alone is not sufficient for capturing all usability issues which may arise in the end experience.
- Usability professionals must insist on project team involvement during RITE sessions. By its very nature, the methodology requires input from all stakeholders on the changes made to the wireframes between sessions. Without business input, the UX team may violate rules or other restrictions. Without technical input, the team can add functionality that cannot be delivered on the target platform. To help offset the risk of project team members not attending the sessions, schedule the testing as early as possible and send out invitations to key players. At minimum, the project team members should attend the end-of-day recap, which typically lasts thirty minutes.
- The end-of-day recap sessions should occur as soon as possible after the last session of the day. It helps to have this meeting in a room with a whiteboard. If there are competing design approaches, low-fidelity alternatives can be mocked up (A/B testing) for evaluation in subsequent sessions.
- Experienced usability professionals are best qualified to employ the RITE methodology. Since changes to the wireframes are being made with feedback from one to three users, it is incumbent upon the UX members of the team to bring their experience to bear upon what is being learned. A junior or inexperienced UX professional may jump to conclusions that are not valid.
- Some project team members or stakeholders expect usability reports to contain a good amount of statistical analysis (for example, time on task, error rates, etc.). While these measures can be helpful in certain circumstances, the RITE approach being described here is ill-suited to provide quantitative report data. Since the heart of the approach is to marry the experience of the UX professionals on the team with what is being seen from testing a few participants, organizations that require a more quantitative approach may need to take additional steps. The usability lead on any RITE project needs to make the final call on when the team has received enough feedback to make a decision on when to change/update a design. There is a fine line to walk, but the lead can hold off on any tenuous decisions by having a design run through additional RITE sessions before moving forward.
- The UX lead on RITE projects needs to be aware that this technique will require much more time than a typical usability test. As the person responsible for creating and updating the wireframes, they need to be available to attend all sessions, recap meetings, and quickly make recommended changes to the prototype. The result is that the UX lead needs to be dedicated to the project full-time during the weeks of RITE sessions, and needs to put in overtime hours as needed. Overall, the UX lead should be prepared to spend up to twice as much time as he or she would running a standard usability test.
- The final report often takes on a “Tadpole to Frog” type of look. Progressions of key sections can be shown in wireframe format with explanation given for what changes were made and why. The report thus illustrates the series of stages as the prototype changed, much like the lifecycle of a frog illustrates its transformation from a tadpole.
- Since the prototype is constantly changing as lessons are learned, RITE presents a good opportunity to introduce additional modules or functionality during the course of testing. If the team is prepared to allow for such scope expansion, then the testing schedule should be extended accordingly to accommodate iteration on these new elements.
Conclusion
RITE is best suited for projects in an early concept phase when the project team is open to exploring new directions. The use of the RITE methodology allowed the UX team to examine several interface alternatives in a relatively short time, evolving a more usable final design more quickly than would have been possible using traditional usability testing methods.
All team members should be prepared to have fun. Our experience over the past two years is that almost every team member has thoroughly enjoyed themselves when being part of a RITE initiative. For many, this is one of the first times that they have been part of a participatory design process, and gaining feedback quickly on their ideas is very exciting.
Since changes to the prototype are taking place almost daily, the team needs to pull together during the week or weeks dedicated to the RITE sessions. Clients love being involved in the process and they tend to walk away from the project with a much greater understanding of the user-centered design process and the value of our work
