UX Foundations · Step 2 · 4 min read
Defining success
Decide how you'll know a design worked before you design it. Outcomes versus outputs, Google's HEART framework and Goals, Signals, Metrics, simple measures anyone can use (task success, time on task, SUS), and how to write success criteria into a brief.
"We launched it" is not success. A redesign can ship on time, look great, and change nothing for the people using it. Deciding upfront what success looks like (fewer people giving up, faster tasks, fewer support calls) keeps the team honest, focuses the design, and gives you something to show after launch besides screenshots.
What you'll be able to do after reading:
- Tell an outcome from an output
- Use HEART and Goals, Signals, Metrics to choose measures
- Measure task success, time on task and perceived usability
- Write success criteria into a design brief
1. Outcomes, not outputs
An output is what the team makes: a feature, a screen, a release. An outcome is a change in what people do. Josh Seiden's short book puts the idea in its subtitle: Why Customer Behavior Is the Key Metric for Business Success (Outcomes Over Output, 2019).
| Output | Outcome |
|---|---|
| New checkout page | More first-time buyers finish their purchase |
| Help centre redesign | Fewer people contact support about delivery dates |
| Onboarding tour | More new users complete their first task in week one |
2. HEART, and Goals, Signals, Metrics
Google researchers proposed HEART: "Happiness, Engagement, Adoption, Retention, and Task success", chosen through a simple process: goals, then signals, then metrics (Rodden, Hutchinson and Fu, CHI 2010).
How people feel about it.
Signal: satisfaction · Metric: SUSHow much and how often they use it.
Signal: repeat use · Metric: weekly active daysNew people starting to use it.
Signal: first use · Metric: share of new users who try itPeople coming back.
Signal: return · Metric: share still active after 30 daysPeople getting it done.
Signal: completion · Metric: success rate, time on taskYou rarely need all five. Pick the one or two that match your problem statement. See HEART framework and Goals-Signals-Metrics.
3. Simple measures anyone can use
- Task success: did people complete the task? Nielsen and Budiu call success rate "the simplest usability metric" (NN/g, Success Rate). See Task success and time on task.
- Time on task: how long it took, compared with the old design.
- Perceived usability (SUS): a ten-question survey scored 0 to 100. NN/g notes the score is "not equivalent to a percentage", and that the average is about 68 (NN/g, Beyond the NPS). See SUS.
- Behaviour after launch: drop-off, repeat use, support contacts, from analytics or support data.
A North Star metric (one number a whole product team rallies around) is a common industry term; it helps only if it measures an outcome for users.
4. Success criteria in the brief
Write them before design, next to the problem statement:
Problem: first-time buyers on phones give up at the delivery step
because they can't see the total cost.
Success: in a test with 5 first-time buyers, all can say the total
before the delivery step; after launch, drop-off at delivery
falls from 38% to under 25% within six weeks.
Specific, measurable, with a time frame, and linked to the problem.
5. Common mistakes
- Measuring outputs ("we shipped 12 features").
- Choosing metrics after launch, so any result looks good.
- Too many metrics, none linked to the problem.
- Treating a satisfaction score as proof that people can actually do the task.
6. Vocabulary
| Term | Meaning |
|---|---|
| Output | What the team makes |
| Outcome | A change in what people do |
| HEART | Happiness, Engagement, Adoption, Retention, Task success |
| Goals, Signals, Metrics | From what you want, to what would show it, to what you'll count |
| Task success rate | Share of people who complete a task |
| SUS | System Usability Scale: ten questions, a score from 0 to 100 |
7. Try it (no tools needed)
- Take the problem statement you wrote in Problem statements and "How might we".
- Write one outcome you want, using the HEART category that fits best.
- Write the signal that would show it and the metric you'd count, with a number and a date.
Reflection questions: Could your metric go up while users are worse off? What would you measure in a test with five people, before any launch?
8. If you already practise
- Add a "how we'll know" line to every ticket or brief, and review it in the retro after launch.
- Combine a behaviour metric with a perception one (task success with SUS) so neither misleads alone.
- Sauro and Lewis's Quantifying the User Experience is the standard guide to UX statistics.
Checklist
Learn more
- Frameworks: Rodden, Hutchinson and Fu, Measuring the User Experience on a Large Scale (HEART) (CHI 2010)
- NN/g: Success Rate: The Simplest Usability Metric · Beyond the NPS: Measuring Perceived Usability with the SUS, NASA-TLX, and the Single Ease Question (2018)
- SUS: MeasuringU, Measuring Usability with the System Usability Scale
Books, courses and videos to go deeper
- Outcomes Over OutputJosh SeidenBook · Sense & Respond Press, 2019 (Amazon)
- Quantifying the User ExperienceJeff Sauro and James R. LewisBook · 2nd edition, Morgan Kaufmann, 2016 (Amazon)
Book links go to Amazon. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.
Related
Problem statements and "How might we" · Journey maps · HEART framework · Goals-Signals-Metrics · Task success and time on task · SUS · Measuring AI features