Framework · Measurement · 2 min read
UX benchmarking
Measuring the same top tasks with the same method over time, so you can show change, for example "success 70% → 92%".
- 1Top tasks
Pick the top 5 tasks per app by frequency × importance.
"Which tasks matter most?" - 2Baseline
Measure success, time and SEQ with 5–8 users.
Same script, same tasks - 3Survey
Add a SUS or UMUX-Lite survey per app, once or twice a year.
One score per app - 4Re-measure
Repeat the same protocol after major changes.
Change nothing in the method - 5Report
Show before and after, task by task.
"Success 70% → 92%"
Repeat after each major release: the last round becomes the new baseline.
Use it when
- You need before/after evidence that a redesign or a design system rollout improved an app.
- Leadership asks how UX quality is developing across your apps over the year.
- You want a baseline in your first months so later progress can be shown.
How to use it
- Pick the top tasks. Choose the top 5 tasks per key app, ranked by frequency × importance. Output: a task list per app with success criteria.
- Measure the baseline. Measure task success, time on task and SEQ in moderated or unmoderated tests with 5–8 users, plus analytics where allowed. Output: baseline values per task.
- Add a survey per app. Run a SUS or UMUX-Lite survey per app, annually or twice a year. Output: one usability score per app.
- Write down the protocol. Record the tasks, scenario wording, criteria, participant profile and setup, so the next round can repeat it exactly. Output: a protocol document.
- Re-measure after major changes. Run the same protocol after a major release or redesign. Output: a second set of values.
- Report before/after. Report per task, for example: "Task X: success 70% → 92%, median time 3:10 → 1:55." Give counts as well as percentages. Output: a one-page benchmark report.
Worked example
A benchmark for the booking app on an internal operations platform, before and after a form redesign (illustrative numbers).
| Task | Success before → after | Median time before → after | SEQ before → after |
|---|---|---|---|
| Create a booking | 70% → 92% | 3:10 → 1:55 | 4.8 → 6.1 |
| Correct a customs code | 5/8 → 7/8 | 2:30 → 1:40 | 3.9 → 5.4 |
| Export a monthly report | 7/8 → 7/8 | 1:20 → 1:15 | 5.1 → 5.3 |
The export task did not change, which is expected: it was not part of the redesign. That makes the other improvements more credible.
Watch out for
- Changing the method between rounds. New tasks, new wording or a different participant group make the comparison invalid.
- Too many tasks. Benchmarking every task makes rounds too expensive to repeat. Keep to the top 5.
- Small samples read as precise. With 5–8 users, differences are rough. Report counts and be careful with small changes.
- Only one round. A baseline without a re-measure shows nothing. Plan the second round when you plan the first.
Template
Copy this into your benchmark plan and keep it for every round.
App: Round: Date:
Participants (profile, number):
Method: moderated / unmoderated Setup:
| # | Task (scenario wording) | Success criteria | Success | Median time | SEQ |
|---|-------------------------|------------------|---------|-------------|-----|
| 1 | | | | | |
| 2 | | | | | |
| 3 | | | | | |
| 4 | | | | | |
| 5 | | | | | |
App survey: SUS / UMUX-Lite Score: Previous round:
Changes since last round:
- Measuring the User ExperienceBill Albert & Tom TullisBook · 3rd ed. 2022 (Amazon)
- Benchmarking UX: Tracking MetricsNN/g (external site)
- 7 Steps to Benchmark Your Product's UXNN/g (external site)
"See on Amazon" links are affiliate links. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.