Module 9 of 12 · 8 min read
UX Maturity & Measurement
How to baseline UX maturity, choose metrics goals-first, measure within data-protection rules, and show the ROI of UX for internal apps.
Why it matters
"A more consistent, data-informed UX practice" is one of the most common goals given to new UX leads (the Evidence-driven UX pillar in the course's example practice model). Without evidence, UX decisions become opinion contests, and the lead with the loudest voice wins. With evidence, you can prioritise, show progress and defend the team's work.
Expect these questions from your manager and stakeholders: "How mature is our UX? Which metrics should we track? Can we even track usage, given data protection?" Common pitfalls: tracking everything and answering nothing, measuring individuals instead of the system, and presenting a big ROI number nobody believes.
1. NN/g UX Maturity Model (6 stages)
| Stage | Name | Signs |
|---|---|---|
| 1 | Absent | UX ignored or nonexistent |
| 2 | Limited | Sporadic, unreliable UX; depends on individuals |
| 3 | Emergent | Functional and promising, but inconsistent and inefficient |
| 4 | Structured | Semi-systematic, varies in effectiveness across teams |
| 5 | Integrated | Comprehensive, effective, pervasive |
| 6 | User-driven | Dedicated UX at all levels leads to deep insight and outcomes |
NN/g assesses maturity across four factors:
- Strategy. Leadership, planning, prioritisation, resourcing.
- Culture. Awareness, knowledge, growth of UX in the organisation.
- Process. Methods, research, design approaches, systematisation.
- Outcomes. UX goals, measurement, impact on decisions.
Form a hypothesis, then verify it in your first 30 days. For example, in a fictional organisation where designers work on modern tools and libraries but vary by domain, measurement isn't systematic and designers are often isolated, a reasonable starting hypothesis would be Emergent–Structured (3–4). Treat it as a guess until the baseline confirms it.
Running a maturity baseline
- Use NN/g's free UX maturity self-assessment quiz as a starting point (see the Library).
- Ask each designer + one product manager per domain to complete it.
- Add 4–5 questions specific to your organisation (e.g., "Is there a UX request process?", "Are usage metrics available?").
- Visualise as a heatmap: domains × factors.
- Repeat every 6–12 months.
2. Choosing metrics: goals first
Goals–Signals–Metrics (from Google's HEART paper)
Goal: Operators book shipments quickly and correctly
Signal: Fewer corrections, faster completion, fewer abandoned bookings
Metric: Median booking time; % bookings edited within 24h; abandonment rate
HEART framework (Rodden, Hutchinson & Fu, Google, 2010)
| Dimension | Internal-app adaptation | Example metric |
|---|---|---|
| Happiness | Satisfaction, perceived ease | SUS / UMUX-Lite score per app; SEQ after tasks |
| Engagement | Feature usage depth (less meaningful for mandatory tools) | % users using bulk edit |
| Adoption | New feature uptake, shadow-tool replacement | % users of new filter after 30 days |
| Retention | (Mandatory tools) → use "workaround avoidance" | % tasks done in-tool vs Excel exports |
| Task success | The most important for internal apps | Completion rate, time on task, error rate |
For internal tools, Task Success and Happiness matter most. Users have to use the tool, so engagement and retention say little.
Standard UX measures
| Measure | What | Notes |
|---|---|---|
| Task success rate | % users completing a task | Simplest, most powerful |
| Time on task | Duration of key tasks | Use median; compare benchmarks |
| Error rate | Errors per task / corrections | Key for operational data quality |
| SEQ (Single Ease Question) | 1–7 "How easy was this task?" | Post-task, cheap |
| SUS (System Usability Scale) | 10-item questionnaire, 0–100 | ~68 = average; widely benchmarked |
| UMUX-Lite | 2-item version, correlates with SUS | Great for in-app surveys |
| Support tickets | Volume/category per app | Often already available, no tracking needed |
3. Benchmarking programme (lightweight)
- Pick the top 5 tasks per key app (by frequency × importance).
- Measure baseline: task success, time, SEQ (moderated or unmoderated tests with 5–8 users, plus analytics where allowed).
- Add a SUS/UMUX-Lite survey per app (annually or bi-annually).
- Re-measure after major changes.
- Report in a before/after format, for example: "Task X: success 70% → 92%, median time 3:10 → 1:55." (Illustrative numbers; report your own.)
4. Analytics under data protection (GDPR & works councils)
For internal tools, analytics usually means tracking employees. In the EU, GDPR applies. In some countries, works councils also have a say: in Germany, for example, the works council (Betriebsrat) has co-determination rights over tools that could monitor employee performance, which usually means a formal agreement before you track. Raising these constraints yourself, before anyone else does, shows real maturity.
Principles
- Purpose limitation. Measure the system's usability, not individuals' performance. Say this explicitly to stakeholders and the works council.
- Data minimisation. Collect only events needed to answer defined questions.
- Anonymise/pseudonymise. No user IDs where not needed; aggregate reporting; IP anonymisation.
- Self-hosting / EU hosting. A self-hosted analytics tool (Matomo, for example) keeps data in-house, which is a common choice for GDPR.
- Involve the data protection officer (DPO), legal and the works council early. Bring a clear tracking plan: events, purpose, retention, access.
- Retention limits. E.g., raw events 6–13 months, aggregates longer.
Tracking plan template
| Event | Question it answers | Properties (no PII) | Retention | Owner |
|---|---|---|---|---|
| booking_started | Where do bookings begin? | app, screen, entry_point | 13 months | UX |
| booking_completed | Completion rate & time | duration_bucket, steps_count | 13 months | UX |
| validation_error_shown | Which fields cause errors? | field_id, error_type | 13 months | UX |
When analytics are not possible, use alternatives: moderated usability tests, diary studies, contextual inquiry (shadowing operators is gold for operational tools), support-ticket analysis, surveys, and server-side logs aggregated by IT.
5. Qual + quant: the balance
| Quant tells you | Qual tells you |
|---|---|
| What and how much | Why and how |
| Where the problem is | What the problem is |
| Whether a change worked | Why it worked (or didn't) |
6. ROI of UX for internal apps
- Productivity: time saved × users × frequency × hourly cost.
- Error reduction: errors avoided × cost per error (rework, claims, penalties).
- Training: reduced onboarding time × new hires.
- Support: tickets avoided × cost per ticket.
- Development: less rework from late changes, faster delivery through the design system.
Be conservative and transparent about assumptions. Credibility matters more than a big number.
7. Measuring the UX practice itself
| Area | Metric |
|---|---|
| Maturity | NN/g baseline score per domain (6–12 months) |
| Coverage | % of domains with dedicated/shared UX |
| Process | % projects with research before dev |
| Design system | Adoption %, detach rate |
| Team health | Health-check scores (module 04) |
| Stakeholder satisfaction | Quarterly 3-question survey to product managers |
Practice
Evidence to collect
Keep proof in your leadership log:
- Maturity heatmap (even if estimated).
- Metrics framework for one core app.
- Any before/after numbers.
Read / watch
Library → Measurement: NN/g UX Maturity articles + quiz, HEART paper, Measuring the User Experience (Tullis & Albert), Quantifying the User Experience (Sauro & Lewis), MeasuringU blog, Matomo GDPR guide.
Talking about it
- "How mature is our UX?"
"I'd start with a maturity baseline across the four NN/g factors (strategy, culture, process and outcomes), per domain, because maturity probably varies a lot between domains. Then pick one or two factors to move a stage in the first year, rather than trying to fix everything."
- "Which metrics should we track?" Start from goals, not tools: one Goals-Signals-Metrics example for a real task is more convincing than a list of KPIs.
- Qual vs quant:
"Analytics shows where users struggle; research explains why. I'd use quant to find and prioritise problems and to verify improvements, and qual to understand and solve them. For internal tools, a few hours of shadowing operators often teaches more than months of click data."
- With the DPO and works council: lead with purpose limitation. "We want to measure the tool, not the people. Here's the tracking plan, here's what we will never collect."
- In a promotion or hiring conversation: separate clearly what you implemented, what you proposed and what you measured. Honest scope is more credible than an inflated result.
Check yourself
Answer out loud, then open the section to compare.
- Name the six maturity stages and the four factors. → answer
- Give a Goals-Signals-Metrics example for a tool in your scope. → answer
- What does the works council need to see before you track usage? → answer
Connections
← Module 8: Design Systems & Standards Governance · UX Lead Course · Module 10: Stakeholders, Influence & Change →
Related
- Module 6: UX Strategy & Vision: OKRs built on these metrics
- Module 5: Enterprise & Internal Applications: research methods when analytics isn't allowed
- Module 8: Design Systems & Standards Governance: design-system metrics