The Evolution of UX Research: A Job Posting Analysis

Presumably, most people employed in UX only browse and read job postings while actively seeking or considering a new position. However, everyone can learn from the contents of postings regardless of their current employment situation. They provide answers to questions, such as:

  • “I want to become a UX researcher. What should I learn?”
  • “I would like to specialize in UX research. Which competencies should I master?”
  • “I want to expand my UX research skills. Where should I start?”

While the UX researcher is the focus of this article, the process of learning from job postings can apply to any UX position. Whether a student, an entry-level practitioner, or a seasoned expert, there are many insights you can gain.

In this article, we present the findings from a job postings analysis. The focus of our analysis was understanding the general expectations of UX researchers and, specifically, current mixed methods research expectations (for instance, qualitative versus quantitative). We wanted to know:

  • What do companies expect from a UX researcher?
  • How is the researcher’s role in product development described?
  • Which methods should UX researchers master?
  • What are the qualitative versus quantitative expectations of researchers?
  • What are the current trends in UX research?
  • How are the UX researcher expectations evolving?

We centered our efforts on the job postings of the top 15 “internet services and retail companies” according to Fortune.com. We began by searching for job openings in those companies using the search term “UX researcher.” All but one of the companies had open UX research-related positions, which is great news for UX researchers already in the field or those hoping to enter the field!

A variety of job titles were found across the 18 job postings we analyzed. Based on the titles alone, the postings could be grouped by the following focus areas:

  • User: UX researcher, user researcher, senior user experience researcher
  • Quantitative: quantitative user experience researcher, senior quantitative researcher
  • Design: (senior) design research engineer, UX designer researcher
  • Customer: customer research engineer

From Usability to UX

Has it always been like this? Definitely not. For those older owls in the field of UX, it may not come as a surprise that the concept of “UX research” started to appear in Internet searches around 2009; before that, “usability engineer” was the standard denomination. At that time, UX was limited to the end of the product development process, at the evaluation phase, when everything had been already conceptualized and implemented.

A comparison of the Google search data between the terms “UX researcher” and “usability engineer” from 2004 until now (see Figure 1) supports the view that we are trending away from a usability-only focus to a UX focus.

Google Trend graph showing differences between “UX Researcher” and “usability engineer” search terms

Figure 1. Screenshot of the search results for the search terms “UX researcher” and “usability engineer” in Google Trends.

This is in line with these common trends we identified from the job postings we analyzed:

  • Conduct research across all phases of product development. Almost all the job postings emphasized the expectation that the UX researcher would play a role in generative and evaluative phases. Companies expect those professionals to conduct foundational research activities and inform product development from the onset as well as directional research to guide everyday product development tasks.
    • “Research spans from generative research to design, launch, and measurement of product performance.” —UX researcher
    • “Your goal is to ensure that we can deliver just-in-time insights from generative, descriptive, or evaluative research within days, not weeks.” —Senior UX researcher
  • Run project from start to end. The UX researcher also must know how to run projects from start to end, from the initial research questions to the communication of the results.
    • “You will collaborate with business partners and user experience teams to outline research plans, execute research, consolidate findings, and present actionable insights.” —Senior UX researcher
    • “Drive research efforts on important projects, with impact on the business: planning, conducting, analyzing, or presenting user research studies.” —User researcher
  • Work autonomously and collaboratively. Almost all job postings stated the expectation that researchers work autonomously, always in collaboration, and be excellent communicators.
    • “The ability to manage ambiguity, work autonomously, and multi-task in an agile environment.” —UX researcher
    • “Work independently and prioritize time between multiple projects and be flexible by adapting to changing schedules and different projects, possibly with stakeholders in different locations and time zones.” —UX researcher

Qualitative vs. Quantitative Qualifications

Anyone working in the field of UX has heard or has been part of conversations regarding the number of participants to include in a study. Usually the conversations are focused on how much weight or relevancy qualitative results have. Traditionally, and still today, many non-UX professionals do not trust results from qualitative methods. Many think the word “data” refers only to numbers and that “data-driven design” means working solely with quantitative results.

In actuality, “data” is qualitative/quantitative agnostic. There are definitely different types of data, but they are equally valuable in “data-informed design.” In this sense, we like the differentiation between “thick data” and “big data” made by Tricia Wang in her article “Why Big Data Needs Thick Data.”

When UX started to grow as a discipline, the UX designer usually needed to operate as a “team of one” working on research, conceptualization, visual design, etc. This is still the case in many companies, and we still find companies that are searching for the “UX unicorn.” On the other hand, increasingly more and more companies understand the benefits of having specialized roles. A dedicated UX researcher spends his or her time on research activities and as a result can run more studies, spend more time on them, and focus on communicating the uncovered insights. This person is also a research expert and so has a broader knowledge of different methods and how and when to use them.

Indeed, when a company decides to allocate time and budget for a UX researcher role, it has decided that understanding the user or customer is essential for them, and that it requires individuals who are specialized. However, it is hard to find someone experienced with qualitative and quantitative methods. As a result, large companies are starting to differentiate and incorporate both roles.

In our analysis of job postings, we found three quantitative/qualitative research approaches:

  • Separate roles: Some companies had separate, specialized descriptions for qualitative researchers and quantitative researchers.
  • Combined role: Some companies expect the UX researcher to conduct qualitative and quantitative research.
    • “A healthy mix of qualitative and quantitative methodologies.” —UX researcher
    • “Solid knowledge of social science and design research methods and the ability to apply them to real-world practical problems by utilizing mixed methods.” —Principal UX researcher
  • Cross-team collaboration: Some companies describe the UX researcher as being responsible for only qualitative research, but are expected to collaborate with quantitative colleagues (for example, the data team) or vice versa.
    • “Define and measure quantitative UX goals and metrics in collaboration with designers, qualitative researchers, engineers, and program managers.” —Quantitative UX researcher
    • “Work cross-functionally with design, product management, data science, content strategy, engineering, and marketing” —UX researcher

Qualitative UX research is well established and backed up by the strong traditions of social sciences or psychology. Many people wrongly assume that anyone can run an in-depth interview or a user study: just make up a list of questions, pose them to some participants, and analyze the answers. There are, in actuality, many factors to take into account—defining the right research questions, selecting the most appropriate method(s), defining and recruiting the right participants, listening without asking leading questions and acknowledging our own biases, and analyzing the results. Arguably, it requires a certain level of expertise. In his book Interviewing Users, Steve Portigal states, “Doing this [interviewing users] well is hard and takes years of practice.”

Quantitative UX researchers are less common, and their role is often misunderstood with that of product analysts or product scientists—both use datasets to explain user’s behavior. However, as described by Mary Nguyen, et al., in “How Quantitative UX Research Differs from Data Analytics,” there are fundamental differences in these roles. Generally speaking, the product analyst is focused on explaining the “what” (metrics-focused) while the quantitative researcher is trying to infer the “why” (behavior-focused). In our job postings analysis, we noticed that quantitative researcher job postings emphasize working with research questions and not just metrics, which may be an attempt to distinguish the position from solely metrics-focused positions.

In the cases where the UX researcher is expected to conduct both qualitative and quantitative research, the job postings typically mention the requirement that the researcher use the same methods as the purely qualitative researcher plus some quantitative methods. However, the specialized quantitative researcher is expected to use more advanced analysis techniques and to have a broader expertise on quantitative methods such as creating statistical models and developing code for analysis of big data.

Other Qualifications and Expectations

What kind of background are UX researchers expected to have?

A bachelor’s or master’s degree in human-computer interaction was by far the most commonly requested background across the various job postings we analyzed. General psychology, cognitive psychology, or experimental psychology were also common. In the social sciences, anthropology, sociology, information science, or communication were acceptable backgrounds for some jobs. Lastly, some job postings also included mention of backgrounds in information architecture or human-centered design.

Despite appearing less often, computer science was an expected background of UX researchers in some postings. This was especially apparent for quantitative-focused researcher postings.

What methods are expected from a UX researcher?

We mentioned earlier how researchers need to work on and contribute to foundational and directional research. Besides that fundamental requirement, most job descriptions we analyzed did not include a comprehensive list of methods researchers need to be able to use. For the most part, the postings simply mentioned that researchers need to be experts in different types of methods, either qualitative, quantitative, or a combination of both.

Surveys and A/B testing were the exceptions, however, as they were directly listed in some job postings. Surveys are a quantitative method from a qualitative approach. A/B testing is an approach more linked to the way a company envisions product development than to a specialization expected from researchers. In this sense, some companies with multi-variate experimentation ingrained in their process expect all researchers to be comfortable with such a method.

With whom are UX researchers expected to collaborate?

Product, followed by design, are the two disciplines that most UX researcher positions refer to when describing collaboration requirements. Software engineers were also mentioned as collaborators. From the postings analyzed we clearly saw that the realm of UX research was product development—not strategy or operations.

Working in cross-functional teams, UX researchers are expected to understand the product roadmap and contribute to the development and innovation of products. Their expertise is expected to bring the “voice of the customer” to product design. With such a focus, researchers are either embedded in the product teams or as part of a dedicated research team that then collaborates in the product development. Ultimately, although not widely stated in the job postings analyzed, the deliverable of UX researchers is to have an impact on product development.

There are some companies in which marketing and/or data science are also stakeholders for, or partners with, UX researchers. In those cases, the collaboration appears to be associated with the company organization and structure as opposed to the methods (quantitative, qualitative, or mixed) the roles employ.

Conclusion: The Growing Trend of Mixed Methods

Following the general trend of increased specialization of UX roles, our analysis of 18 job postings suggests the trend is following suit for UX research. However, not everyone has caught up with the trend. In some companies, user research is just one of the activities that a UX designer has to do; the boundaries between product, UX research, and data science are blurry; the word “research” is associated with academia.

Nevertheless, with the lean startup movement and the increased interest in A/B testing—to cite just a couple of factors—formal research approaches are reaching the software development industry. Lean frameworks advocate for the application of the scientific method; from the hypothesis statement to its validation or refute.

The concept of mixed methods is also becoming widespread in research. Qualitative methods such as field studies, contextual inquiries, user interviews, user diaries, and the like have traditionally been in the suitcase of UXers. There were also more quantitative approaches, notably in usability, such as eye-tracking or the tracking of usage metrics such as performance.

Companies have realized that this dichotomy is an oversimplification of the problem space. This is especially true for those companies that are searching for quantitative versus qualitative specialists. They see how these two data sources are needed to inform product development, and, in order to make the most out of them, they look for specialists that master one or the other and then have them work together to increase their impact.

Our conclusion is that these seemingly opposing trends will persist for a while due to the different levels of research maturity in the market. UX teams of one, T-shaped UX, or UX unicorns are still in demand and will continue to be so. However, in our opinion, the demand for specialized UXers will keep growing. The increased understanding of UX by big companies is translated in the definition of job postings for “mixed” UX researchers or specialized quantitative or qualitative researchers. It is likely to be a slow trend in the same way that UX took awhile to reach companies. However, and luckily, the future looks bright for anyone wishing to work in UX research.

Garreta-Domingo, M., González Mosquera, A. (2018). The Evolution of UX Research: A Job Posting Analysis. User Experience Magazine, 18(3).
Retrieved from http://uxpamagazine.org/the-evolution-of-ux-research/

Leave a Reply