Marcus Andrey Study
Ler em português Library

Framework · Measurement · 2 min read

UX benchmarking

Measuring the same top tasks with the same method over time, so you can show change, for example "success 70% → 92%".

  1. 1Top tasks

    Pick the top 5 tasks per app by frequency × importance.

    "Which tasks matter most?"
  2. 2Baseline

    Measure success, time and SEQ with 5–8 users.

    Same script, same tasks
  3. 3Survey

    Add a SUS or UMUX-Lite survey per app, once or twice a year.

    One score per app
  4. 4Re-measure

    Repeat the same protocol after major changes.

    Change nothing in the method
  5. 5Report

    Show before and after, task by task.

    "Success 70% → 92%"

Repeat after each major release: the last round becomes the new baseline.

A lightweight benchmark measures the same top tasks with the same method over time, so change becomes visible.

Use it when

  • You need before/after evidence that a redesign or a design system rollout improved an app.
  • Leadership asks how UX quality is developing across your apps over the year.
  • You want a baseline in your first months so later progress can be shown.

How to use it

  1. Pick the top tasks. Choose the top 5 tasks per key app, ranked by frequency × importance. Output: a task list per app with success criteria.
  2. Measure the baseline. Measure task success, time on task and SEQ in moderated or unmoderated tests with 5–8 users, plus analytics where allowed. Output: baseline values per task.
  3. Add a survey per app. Run a SUS or UMUX-Lite survey per app, annually or twice a year. Output: one usability score per app.
  4. Write down the protocol. Record the tasks, scenario wording, criteria, participant profile and setup, so the next round can repeat it exactly. Output: a protocol document.
  5. Re-measure after major changes. Run the same protocol after a major release or redesign. Output: a second set of values.
  6. Report before/after. Report per task, for example: "Task X: success 70% → 92%, median time 3:10 → 1:55." Give counts as well as percentages. Output: a one-page benchmark report.

Worked example

A benchmark for the booking app on an internal operations platform, before and after a form redesign (illustrative numbers).

Task Success before → after Median time before → after SEQ before → after
Create a booking 70% → 92% 3:10 → 1:55 4.8 → 6.1
Correct a customs code 5/8 → 7/8 2:30 → 1:40 3.9 → 5.4
Export a monthly report 7/8 → 7/8 1:20 → 1:15 5.1 → 5.3

The export task did not change, which is expected: it was not part of the redesign. That makes the other improvements more credible.

Watch out for

  • Changing the method between rounds. New tasks, new wording or a different participant group make the comparison invalid.
  • Too many tasks. Benchmarking every task makes rounds too expensive to repeat. Keep to the top 5.
  • Small samples read as precise. With 5–8 users, differences are rough. Report counts and be careful with small changes.
  • Only one round. A baseline without a re-measure shows nothing. Plan the second round when you plan the first.

Template

Copy this into your benchmark plan and keep it for every round.

App:                     Round:            Date:
Participants (profile, number):
Method: moderated / unmoderated     Setup:

| # | Task (scenario wording) | Success criteria | Success | Median time | SEQ |
|---|-------------------------|------------------|---------|-------------|-----|
| 1 |                         |                  |         |             |     |
| 2 |                         |                  |         |             |     |
| 3 |                         |                  |         |             |     |
| 4 |                         |                  |         |             |     |
| 5 |                         |                  |         |             |     |

App survey: SUS / UMUX-Lite    Score:      Previous round:
Changes since last round:

"See on Amazon" links are affiliate links. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.

Saved only on this device.

↑ ↓ to move · Enter to open · Esc to close