One of the latest techniques in the Human Computer Interaction (HCI) community is a data collection mechanism called cultural probes. These were introduced in 1999 by William Gaver, an academic based at the London Royal College of Art, in the publication Interactions.

Since then, cultural probes have been discussed, developed, and used extensively in academic HCI research. Although industry practitioners are starting to adopt cultural probes, there are still many questions around the usefulness and practical application of this technique. This is not a simple question and, as Gaver says, “The use of probes is geared toward design generation, not problem solution.”

What is a Probe?

So what exactly is a cultural probe and how would you explain it to your skeptical boss in the elevator? Gave describes cultural probes as “designed to prompt and elicit information from people about their lives and local culture.” A simple explanation is that it is a diary maintained by a user over several days or weeks. Typically, the diary includes photographs taken by the user, pictures cut from magazines, voice recordings, and written notes. The technique is also called “diary study,” “media diary,” and “photo collage.”

To create their diaries, users are given a “pack.” The contents depend upon the objectives of the data collection, the characteristics of the users, and the imagination of the designer, but common items are scrapbooks, pens, cameras, and diaries.

Some of the more innovative and interesting items that have been used in packs are catch phrases printed on sticky labels and simple digital memo-takers that are repackaged as dream recorders. These let users record ideas or potential requirements as they occur to them, such as when awakening from a vivid dream.

Users are briefed on how to use the packs and what information to record. They are asked to create the diary independently, in their own environments, with regular entries over an allocated time. The designer may contact or visit the users occasionally to monitor progress, answer questions, and keep the users focused.

Why would you use Probes?

All data collection techniques have advantages and disadvantages. To understand why probes might be useful, we need to consider the reliability of data constraints on time and money and the need to collect data for non-work or non-office activities—for example, the use of mobile devices for social, leisure, or on-the-road usage.

Reliability of data is a fundamental challenge for all designers. Every data collection technique risks unreliable data. For example, focus groups may drift so far from reality that the descriptions of actions bear little relation to actual behavior. In laboratory usability tests, the lack of real context may distort behavior. In contextual enquiry, it may simply be impossible to control observational conditions due to the noise of other activities unrelated to the project at hand.

Cultural probes are an additional tool with a new set of strengths and weaknesses. For example, the reliability of data may be enhanced by users being experts in their own lives. Cultural probes provide the opportunity for users to document their lives in their own context, with minimal interference from the designer. Probes also provide the opportunity to triangulate against another set of independent data (such as focus group, usability tests, or survey findings) and thus improve reliability.

Time and money constraints may be circumvented. Commercial organizations often expect HCI practitioners to somehow have an inbuilt and expert understanding of the target audience, and they may make little allowance for the time-consuming and expensive process of collecting and analyzing data to gain a significant understanding of the audience. Because cultural probes place the onus of the actual data collection on the user, practitioners can focus their effort on the analysis of data.

Access to interaction and behavior for non-work activities, or work performed outside the traditional workplace, is a new challenge for commercial HCI. The traditional work environment tends to be well-defined and accessible. Work within this environment is generally routine and structured. People are often able to describe their work, working environment, and patterns.

However, the contexts of both non-work and leisure time activities are much harder to define, and any data collection technique requiring the designer to be physically located with a user may disrupt or destroy the quality of the activity. Furthermore, the lack of routine and structure in these activities may greatly expand, or make uncertain, the amount of data collection required. This is one of the most significant advantages of cultural probes. Data can be collected easily despite the unstructured, non-routine, and easily disturbed context of the activity. Users frequently become very engaged in the studies, and often comment that keeping diaries and scrapbooks is fun.

Challenges using Probes

As with all data collection techniques, probes have challenges. The most significant of these is ensuring that users understand and maintain focus throughout the data collection process. This is not trivial, as probes are very much a hands-off data collection technique; the user is neither observed nor directly guided during the day-to-day creation of data.

On the other hand, the designer must be careful not to be too restrictive or prescriptive and thereby lose ideas, inspiration for design, and unexpected information about user behavior and interactions. The risk is that a lack of focus may lead to very noisy data that is difficult to analyze and does not provide useful input to a specific design. Probe studies should include an initial briefing session and at least one interview visit or telephone call to the user during the early part of the diary-keeping process. This is extremely valuable as many probe users need to be focused, re-focused, or simply taught how to use the pack materials.

Despite the reduction of time and effort required from researchers (since users collect their own data), probe studies consume elapsed time, which may be in short supply in tight project schedules. Therefore, probe studies should be run as early as possible in projects.

Researchers must also take care to gather an appropriate amount of data. The number of users, the length of the data collection period, the probe pack materials, and the requested frequency of diary entry must all be taken into account in the design of cultural probes. For studies in which I have been involved, about twenty to thirty minutes per day was asked of users. This may vary, with some days where less or no data is entered into the diary, and other days where more data is entered.

The length of the data collection period is also important. For example, one study involved seven weeks of data collection but five weeks would have been enough. Some of the users lost interest, finding that some of the data contained nothing new and there was simply too much data. At the other end of the scale, in an industry study for a specific product, the challenge was determining the minimum amount of time to make the study worthwhile. We tried seven days and found that to be just enough. This was mainly due to the simple and easily definable area of focus for the study.

Another influence on the quality of data is the level of interest and participation of the users, as they are not only the source, but also the transcribers of the data. Users may volunteer, ideally with the interest and personal motivation to support diary filling over an extended period. Alternatively, if you need to closely match a target audience persona, or have practical concerns such as time constraints, you may need to recruit and pay users. This is usually done through agencies. In this case, you must take care to maintain the users’ interest, and remuneration must be carefully considered.

Examining and Using Probe Data

Probe data is not straightforward to use. In the first place, probes can generate tremendous amounts of data, requiring significant time even for a first reading. The more complex question, however, concerns how to examine and analyze the data. Academic researchers have spent considerable effort trying to understand how to interpret the data from probes. Gaver comments in one study that, “Our probe results are impossible to analyze or even interpret clearly because they reflect too many layers of influence and constraint.” However, in another part of the same study he comments that, “Sometimes the trajectory from probes to designs is relatively straightforward, and design ideas can clearly be traced back to probe returns.”

There is currently no one standard method for using the data from probe studies to influence design. The most straightforward method is for the HCI practitioner to brainstorm design ideas. The data can provide a rich insight into the lives of users.

Probe data also generally provides a good understanding of a real-world domain. The data enriches the ideas of the designer or researcher and provides a supplementary influence rather than a direct prescription for design solutions. In particular, probe study data, together with interviews and debriefing, provides an excellent source of data from which personas can be constructed.

In one project, probe data was used to create scenarios which, in turn, were used as the basis for participatory design sessions. The data informed and enriched the scenarios, providing excellent context and motivations. The participatory design personnel included researchers involved in the study, HCI experts, and the probe users. It was found that the probe users, while having an intimate knowledge of their own data, did not have the HCI skills for designing, and that the best design concepts came from HCI experts who were familiar with both the probe data and the context of the project as a whole.

Probes for Industry

Cultural probes are a new technique worthy of attention from commercial practitioners. They are particularly useful for novel non-work design situations where user behavior is relatively unknown or difficult to access. Probe data can be very noisy, however, and the data at times may be incomplete, unclear, and biased. It is important to set expectations that the data will enrich the understanding of the target audience and support the creation of personas and scenarios; but while it is intended to support design solutions, it generally does not explicitly contain a design solution.

Perhaps expectations for the use of probe data are best summarized by Gaver, who stated, “Most of the time the relationships between probes and proposals are more complex and difficult to trace. Our design ideas are formed from a combination of conceptual interests, technological possibilities, imaginary scenarios and ideas for how to implement them. The probes are one influence in all this.”

Probes are perhaps best implemented in industry as a technique to supplement standard industry design processes such as initial analysis, feasibility, and early design work. To ensure that the focus of the project is maintained and the effort is contained, designers should take care to plan the materials in the probe pack, set the data collection time, and predetermine the type of data examination or analysis. Probes force the designer to think about the context and motivations of use, thus building a stronger understanding of the target audience and potentially leading to better and more informed designs.