Marcus Andrey Study
Ler em português Library

Module 9 of 12 · 8 min read

UX Maturity & Measurement

How to baseline UX maturity, choose metrics goals-first, measure within data-protection rules, and show the ROI of UX for internal apps.

Why it matters

"A more consistent, data-informed UX practice" is one of the most common goals given to new UX leads (the Evidence-driven UX pillar in the course's example practice model). Without evidence, UX decisions become opinion contests, and the lead with the loudest voice wins. With evidence, you can prioritise, show progress and defend the team's work.

Expect these questions from your manager and stakeholders: "How mature is our UX? Which metrics should we track? Can we even track usage, given data protection?" Common pitfalls: tracking everything and answering nothing, measuring individuals instead of the system, and presenting a big ROI number nobody believes.


1. NN/g UX Maturity Model (6 stages)

Stage Name Signs
1 Absent UX ignored or nonexistent
2 Limited Sporadic, unreliable UX; depends on individuals
3 Emergent Functional and promising, but inconsistent and inefficient
4 Structured Semi-systematic, varies in effectiveness across teams
5 Integrated Comprehensive, effective, pervasive
6 User-driven Dedicated UX at all levels leads to deep insight and outcomes

NN/g assesses maturity across four factors:

  1. Strategy. Leadership, planning, prioritisation, resourcing.
  2. Culture. Awareness, knowledge, growth of UX in the organisation.
  3. Process. Methods, research, design approaches, systematisation.
  4. Outcomes. UX goals, measurement, impact on decisions.

Form a hypothesis, then verify it in your first 30 days. For example, in a fictional organisation where designers work on modern tools and libraries but vary by domain, measurement isn't systematic and designers are often isolated, a reasonable starting hypothesis would be Emergent–Structured (3–4). Treat it as a guess until the baseline confirms it.

Running a maturity baseline

  1. Use NN/g's free UX maturity self-assessment quiz as a starting point (see the Library).
  2. Ask each designer + one product manager per domain to complete it.
  3. Add 4–5 questions specific to your organisation (e.g., "Is there a UX request process?", "Are usage metrics available?").
  4. Visualise as a heatmap: domains × factors.
  5. Repeat every 6–12 months.

2. Choosing metrics: goals first

Goals–Signals–Metrics (from Google's HEART paper)

Goal:    Operators book shipments quickly and correctly
Signal:  Fewer corrections, faster completion, fewer abandoned bookings
Metric:  Median booking time; % bookings edited within 24h; abandonment rate

HEART framework (Rodden, Hutchinson & Fu, Google, 2010)

Dimension Internal-app adaptation Example metric
Happiness Satisfaction, perceived ease SUS / UMUX-Lite score per app; SEQ after tasks
Engagement Feature usage depth (less meaningful for mandatory tools) % users using bulk edit
Adoption New feature uptake, shadow-tool replacement % users of new filter after 30 days
Retention (Mandatory tools) → use "workaround avoidance" % tasks done in-tool vs Excel exports
Task success The most important for internal apps Completion rate, time on task, error rate

For internal tools, Task Success and Happiness matter most. Users have to use the tool, so engagement and retention say little.

Standard UX measures

Measure What Notes
Task success rate % users completing a task Simplest, most powerful
Time on task Duration of key tasks Use median; compare benchmarks
Error rate Errors per task / corrections Key for operational data quality
SEQ (Single Ease Question) 1–7 "How easy was this task?" Post-task, cheap
SUS (System Usability Scale) 10-item questionnaire, 0–100 ~68 = average; widely benchmarked
UMUX-Lite 2-item version, correlates with SUS Great for in-app surveys
Support tickets Volume/category per app Often already available, no tracking needed

3. Benchmarking programme (lightweight)

  1. Pick the top 5 tasks per key app (by frequency × importance).
  2. Measure baseline: task success, time, SEQ (moderated or unmoderated tests with 5–8 users, plus analytics where allowed).
  3. Add a SUS/UMUX-Lite survey per app (annually or bi-annually).
  4. Re-measure after major changes.
  5. Report in a before/after format, for example: "Task X: success 70% → 92%, median time 3:10 → 1:55." (Illustrative numbers; report your own.)

4. Analytics under data protection (GDPR & works councils)

For internal tools, analytics usually means tracking employees. In the EU, GDPR applies. In some countries, works councils also have a say: in Germany, for example, the works council (Betriebsrat) has co-determination rights over tools that could monitor employee performance, which usually means a formal agreement before you track. Raising these constraints yourself, before anyone else does, shows real maturity.

Principles

  • Purpose limitation. Measure the system's usability, not individuals' performance. Say this explicitly to stakeholders and the works council.
  • Data minimisation. Collect only events needed to answer defined questions.
  • Anonymise/pseudonymise. No user IDs where not needed; aggregate reporting; IP anonymisation.
  • Self-hosting / EU hosting. A self-hosted analytics tool (Matomo, for example) keeps data in-house, which is a common choice for GDPR.
  • Involve the data protection officer (DPO), legal and the works council early. Bring a clear tracking plan: events, purpose, retention, access.
  • Retention limits. E.g., raw events 6–13 months, aggregates longer.

Tracking plan template

Event Question it answers Properties (no PII) Retention Owner
booking_started Where do bookings begin? app, screen, entry_point 13 months UX
booking_completed Completion rate & time duration_bucket, steps_count 13 months UX
validation_error_shown Which fields cause errors? field_id, error_type 13 months UX

When analytics are not possible, use alternatives: moderated usability tests, diary studies, contextual inquiry (shadowing operators is gold for operational tools), support-ticket analysis, surveys, and server-side logs aggregated by IT.


5. Qual + quant: the balance

Quant tells you Qual tells you
What and how much Why and how
Where the problem is What the problem is
Whether a change worked Why it worked (or didn't)

6. ROI of UX for internal apps

  • Productivity: time saved × users × frequency × hourly cost.
  • Error reduction: errors avoided × cost per error (rework, claims, penalties).
  • Training: reduced onboarding time × new hires.
  • Support: tickets avoided × cost per ticket.
  • Development: less rework from late changes, faster delivery through the design system.

Be conservative and transparent about assumptions. Credibility matters more than a big number.


7. Measuring the UX practice itself

Area Metric
Maturity NN/g baseline score per domain (6–12 months)
Coverage % of domains with dedicated/shared UX
Process % projects with research before dev
Design system Adoption %, detach rate
Team health Health-check scores (module 04)
Stakeholder satisfaction Quarterly 3-question survey to product managers

Practice

Evidence to collect

Keep proof in your leadership log:

  • Maturity heatmap (even if estimated).
  • Metrics framework for one core app.
  • Any before/after numbers.

Read / watch

Library → Measurement: NN/g UX Maturity articles + quiz, HEART paper, Measuring the User Experience (Tullis & Albert), Quantifying the User Experience (Sauro & Lewis), MeasuringU blog, Matomo GDPR guide.


Talking about it

  • "How mature is our UX?"

    "I'd start with a maturity baseline across the four NN/g factors (strategy, culture, process and outcomes), per domain, because maturity probably varies a lot between domains. Then pick one or two factors to move a stage in the first year, rather than trying to fix everything."

  • "Which metrics should we track?" Start from goals, not tools: one Goals-Signals-Metrics example for a real task is more convincing than a list of KPIs.
  • Qual vs quant:

    "Analytics shows where users struggle; research explains why. I'd use quant to find and prioritise problems and to verify improvements, and qual to understand and solve them. For internal tools, a few hours of shadowing operators often teaches more than months of click data."

  • With the DPO and works council: lead with purpose limitation. "We want to measure the tool, not the people. Here's the tracking plan, here's what we will never collect."
  • In a promotion or hiring conversation: separate clearly what you implemented, what you proposed and what you measured. Honest scope is more credible than an inflated result.

Check yourself

Answer out loud, then open the section to compare.

  1. Name the six maturity stages and the four factors. → answer
  2. Give a Goals-Signals-Metrics example for a tool in your scope. → answer
  3. What does the works council need to see before you track usage? → answer


Connections

← Module 8: Design Systems & Standards Governance · UX Lead Course · Module 10: Stakeholders, Influence & Change →

Related

Hubs: Frameworks index · Design your UX practice

Saved only on this device.

↑ ↓ to move · Enter to open · Esc to close