UX Debt in the Enterprise: A Practical Approach

In early 2014, we found ourselves working for an organization with a large product portfolio built up over decades of acquisitions. To create a cohesive branded experience, it became clear that many of the products we were suddenly supporting existed at very different levels of UX maturity. Our challenge was to create a comparable scorecard for an array of products across UX frameworks, usability maturity, and technology platforms. Our first, somewhat obvious step, was to leverage existing UX maturity models to build a custom framework for our organization, following the advice of Rich Buttiglieri in his presentation on maturity models at UXPA Boston 2012. The model we built focused on the following for each level:

  • What will the user see and experience?
  • How will the experience impact brand perception?
  • What level of UX process support is needed?
  • How would product management, as a whole, react towards the framework proposed?

Table 1. Measures of Experiential Maturity

What does this type of experience look like?
Ad-Hoc Fragmented & inconsistent experience

Disconnected from changing user needs

Measure satisfaction, but not experience quality

Intentional Conducts customer research, but it does not drive the experience

Culture uses data to drive experience decisions

By Design Uses structured research to understand needs by segment and designs directly address needs.

Specify the differentiated experience the organization wants to deliver

Integrated Identify operating model changes required to deliver that experience

Measure and reinforce desired behaviors across the ecosystem (internal and external)

UX is fully integrated with the product lifecycle

UX-Driven Engage customers, employees, etc. in collaborative innovation

Provide a platform to personalize and configure experiences

Find ways to better ways to engage users by testing designs with live traffic

What is the resulting brand perception?
Ad-Hoc “This product/service works as programmed.”
Intentional “This product/service is accurate & available.”
By Design “This product/service is super easy to use and works like I think.”
Integrated “This is a memorable product/service”
UX Driven “Wow, this is fantastic!”
How does the organization think about UX?
Ad-Hoc Unintentional design

“Users will be trained.” User will need training to use the product—usability is not self-evident.

Intentional Starting to use data in design decisions

Getting to know users better through structured data

By Design Activity-focused design

UI designed by UX professionals

Integrated User-centric design

“In the field to study users” and UX metrics also used for product planning.

UX Driven User-centric DNA

Data-driven UX is simply the way the organization learns and acts to transform its products/services.

What is the associated UX process maturity?
Ad-Hoc UX methods are not consistently applied and work is often outsourced.

Heuristic reviews and limited usability testing, usually at process end or conducted on products already in production.

Intentional Hire more staff to do more work; although volume increases, work remains more tactical.

Heuristic reviews & limited usability testing performed at the beginning and end of the process.

Mock-ups and prototypes used to iteratively improve designs

By Design Well-defined UX process, consistent quality and performance

Contextual analysis and greater focus on managing ‘usefulness’

Discovery research (e.g. personas) are regularly used to make design decisions.

Integrated User-centric design

“In the field to study users.”

UX metrics used for product planning

UX Driven Designs that perform in a predictable way

UX metrics are formalized to capture baseline vs. improvement

Competitive analysis, personas, and field research

Quantitative studies (baseline and comparative)

We began introducing our initial experience maturity model to leadership, both within the product organization and throughout IT. This proved invaluable in helping our internal stakeholders understand why our processes were a little different for each product. It also served as a communication tool to help our UXers understand how to relate to internal product management teams.

In addition to introducing our maturity model to key stakeholders, we also needed to provide senior leadership with an executive summary so they could make comparisons across products. This would allow them to gain insight across a large portfolio of varying user experiences and to identify where to prioritize UX in the organization.

We experimented with multiple methods to create this type of summary for products and kept returning to the idea of a UX debt scorecard. What if we could combine the concept of UX debt with our experiential maturity model? This could not only help us communicate across the organization on a level playing field, but would also provide us with a way to measure how products were decreasing their debt, thereby increasing their experience maturity.

In this article, we discuss UX debt as a way to measure how easy and intuitive a product is to use so we can use the UX debt score as a way to determine the organization’s level of experiential maturity. For example: when the Ad-Hoc UX Debt=0, the organization is ready move to Intentional UX. The same formula works for the move from each level, all the way to a UX-driven organization.

We are now calculating UX debt relative to the product’s current maturity level to provide senior leadership with a product-by-product comparative dashboard of UX debt specific to a given product’s maturity level.

Due to the organization’s numerous, diverse products, we created a calculation worksheet meant to consistently calculate UX debt across our portfolio. As we began to communicate this concept throughout the organization, it was apparent that this was an effective communication and planning tool across multiple stakeholders: sales, product teams, technology teams, and senior leadership.

Defining UX Debt

The term “technical debt” was coined by Ward Cunningham as a metaphor used to describe the increased cost of maintaining technology due to shortcuts taken during the development process. Joshua Kerievsky later extended the metaphor to user experience design using the term “User Experience (UX) Debt.” Kerievsky explained that, like technical debt, UX debt will eventually come due, usually in the form of less customer satisfaction and possible customer defects.

In an enterprise setting, studies show that the decision to accumulate technical debt is often made by non-technical stakeholders. In our current enterprise, we wondered if the same is true for UX debt? If we develop a practical way to measure UX debt for products in the enterprise, what would that look like and how would it be received?

From Theory to Practice

We were brought into a company that had previously tried to ”perform UX,” but teams were unable to make much visible progress across multiple diverse, distinct product lines. Non-UXers (largely product managers) were often in charge of making UX decisions. Across this wide variety of products, customer satisfaction scores were showing that user experience, along with a large UX debt, was hindering products’ usability and creating unhappy customers in the process.

How could a small UX team of five people begin to describe at a high level how more than 100 products compared to each other? How could we communicate to senior managers the amount of UX work that we might need to perform to improve them? We quickly shifted our conversations to whether the notion of UX debt could help not only prioritize our UX work, but could pave the way to communicate and develop a more data and user-centric approach to product development at an enterprise level. Our team was familiar with the term UX debt from a theoretical perspective, but we soon began to wonder if we could actually create a long-term UX debt calculation and track it over time for our products. We wanted to create a comparable scorecard across product lines, and to help explain to our senior leadership how much work needs to be done. This was the point where we began to develop the UX debt calculator spreadsheet.

Table 3. A summary of UX Debt by Maturity Level

Ad-Hoc Intentional By Design Integrated
How to calculate UX debt Heuristic
Heuristic analysis or usability data Usability data Usability data
Main type of UX debt to resolve Consistency


Baseline metrics

Meets user needs






Common areas to examine Map inconsistencies Analyze product against known user needs

Level of consistency, based on UX standards

Personalization to user needs

Alignment of current experience with organization’s visionary experience

Use of persuasive design elements

Desired user behaviors

The UX Debt Calculator

UX debt can be calculated with data from either heuristic evaluations or prior usability testing.

Calculating UX debt with heuristic data

We noticed that many products have little or no formative usability data available; products are mostly at the Ad-Hoc level of maturity. To calculate UX debt in these instances, we use heuristic analysis. The UX team performs a heuristic evaluation of the product using the following categories: Findable, Accessible, Clear & Communicative, Useful, Credible, Controllable, Learnable, Overall Aesthetics, Persuasive, and whether the product is currently tracked using our web analytics tool.

In the Template-Heuristics tab in the calculator, the categories correspond to a heuristic. As we complete the evaluation of a product or service, we update the value of the Assessment field for each measure. This field controls the Actual Score and Score Percentage. Possible values for the measure include:

  • Unusable: imperative to fix – Unusable
  • Major issue: important to fix with high priority – Critical
  • Minor issue: should be given low priority – Moderate
  • Cosmetic issue only – Minor
  • Meets Criteria No Issues (default value)

If the product has failed to take accessibility into account, we assign a severity of Major issue. In addition, if we do not currently track the product’s usage, do not complete formative and/or summative testing, and do not implement company branding, we assign a severity of Major issue. This helps drive a culture making product decisions based on user data and beginning to help “paydown” UX debt.

We use the fields Problems and Steps to Resolve to communicate what needs to be addressed in the product for it to move up in its UX maturity. Once all columns are completed, we calculate the current UX debt and any projected paydown.

Calculating debt with prior usability data

In our organization, usability tests were previously conducted on a few products at higher levels of UX maturity. Our initial proof of concept for our UX debt calculator was to translate these third-party studies into a standard debt score. This approach quickly became our preferred approach to calculating debt, as it takes direct user feedback into account. However, when there is no formative data (as was the case with many of our products at lower levels of UX maturity), we default to heuristic evaluation to calculate UX debt.

Screenshot of the author’s UX debt calculator

Figure 1. The UX debt calculator is a spreadsheet with measures and scores

Next Steps

This first phase of the UX debt calculator has been useful in helping us assess products. As a small UX team intending to create a branded experience across over 100 products, we must first determine where that product lies in our experiential maturity model and, second, calculate a debt score. As we began to use this calculator for many of our products, we discovered where our gaps lie in terms of user data, user experience, and customer satisfaction. From this vantage point, we can help product teams move the product forward in its UX maturity, as well as lower its long-standing UX debt.

As we move forward with these concepts across our organization, we expect that our company’s products will advance in their UX maturity and, consequently, pay down their debt. The calculator is a great tool for modeling this CX/UX transition in a product and reinforcing a user- and data-centric product culture.

Download the UX debt calculator

More Reading


Dunwoody, K., Teague Rector, S. (2015). UX Debt in the Enterprise: A Practical Approach. User Experience Magazine, 15(1).
Retrieved from http://uxpamagazine.org/ux-debt-in-the-enterprise/

2 Responses

  1. yet I never found any interesting article so much Interesting
    UX debt calculator has a great tool to help sell UX to management.

  2. Interesting article – I can see the UX debt calculator has a great tool to help sell UX to management and company leaders that don’t know what UX is or why it is necessary.

    Thanks for the information.