Marcus Andrey Study
Ler em português Library

UX Foundations · Step 2 · 4 min read

Defining success

Decide how you'll know a design worked before you design it. Outcomes versus outputs, Google's HEART framework and Goals, Signals, Metrics, simple measures anyone can use (task success, time on task, SUS), and how to write success criteria into a brief.

"We launched it" is not success. A redesign can ship on time, look great, and change nothing for the people using it. Deciding upfront what success looks like (fewer people giving up, faster tasks, fewer support calls) keeps the team honest, focuses the design, and gives you something to show after launch besides screenshots.

What you'll be able to do after reading:

  • Tell an outcome from an output
  • Use HEART and Goals, Signals, Metrics to choose measures
  • Measure task success, time on task and perceived usability
  • Write success criteria into a design brief

1. Outcomes, not outputs

An output is what the team makes: a feature, a screen, a release. An outcome is a change in what people do. Josh Seiden's short book puts the idea in its subtitle: Why Customer Behavior Is the Key Metric for Business Success (Outcomes Over Output, 2019).

Output Outcome
New checkout page More first-time buyers finish their purchase
Help centre redesign Fewer people contact support about delivery dates
Onboarding tour More new users complete their first task in week one
A green tape measure with its yellow tape pulled out.
Decide what you will measure before you design, not after. Otherwise any result looks like success. Photo: Evan-Amos, Wikimedia Commons, public domain.

2. HEART, and Goals, Signals, Metrics

Google researchers proposed HEART: "Happiness, Engagement, Adoption, Retention, and Task success", chosen through a simple process: goals, then signals, then metrics (Rodden, Hutchinson and Fu, CHI 2010).

HHappiness

How people feel about it.

Signal: satisfaction · Metric: SUS
EEngagement

How much and how often they use it.

Signal: repeat use · Metric: weekly active days
AAdoption

New people starting to use it.

Signal: first use · Metric: share of new users who try it
RRetention

People coming back.

Signal: return · Metric: share still active after 30 days
TTask success

People getting it done.

Signal: completion · Metric: success rate, time on task
HEART (Google, 2010): for each category you pick, go from goal to signal to metric. Diagram: Marcus Andrey, CC BY 4.0.

You rarely need all five. Pick the one or two that match your problem statement. See HEART framework and Goals-Signals-Metrics.


3. Simple measures anyone can use

  • Task success: did people complete the task? Nielsen and Budiu call success rate "the simplest usability metric" (NN/g, Success Rate). See Task success and time on task.
  • Time on task: how long it took, compared with the old design.
  • Perceived usability (SUS): a ten-question survey scored 0 to 100. NN/g notes the score is "not equivalent to a percentage", and that the average is about 68 (NN/g, Beyond the NPS). See SUS.
  • Behaviour after launch: drop-off, repeat use, support contacts, from analytics or support data.

A North Star metric (one number a whole product team rallies around) is a common industry term; it helps only if it measures an outcome for users.


4. Success criteria in the brief

Write them before design, next to the problem statement:

Problem: first-time buyers on phones give up at the delivery step
         because they can't see the total cost.
Success: in a test with 5 first-time buyers, all can say the total
         before the delivery step; after launch, drop-off at delivery
         falls from 38% to under 25% within six weeks.

Specific, measurable, with a time frame, and linked to the problem.


5. Common mistakes

  • Measuring outputs ("we shipped 12 features").
  • Choosing metrics after launch, so any result looks good.
  • Too many metrics, none linked to the problem.
  • Treating a satisfaction score as proof that people can actually do the task.

6. Vocabulary

Term Meaning
Output What the team makes
Outcome A change in what people do
HEART Happiness, Engagement, Adoption, Retention, Task success
Goals, Signals, Metrics From what you want, to what would show it, to what you'll count
Task success rate Share of people who complete a task
SUS System Usability Scale: ten questions, a score from 0 to 100

7. Try it (no tools needed)

  1. Take the problem statement you wrote in Problem statements and "How might we".
  2. Write one outcome you want, using the HEART category that fits best.
  3. Write the signal that would show it and the metric you'd count, with a number and a date.

Reflection questions: Could your metric go up while users are worse off? What would you measure in a test with five people, before any launch?


8. If you already practise

  • Add a "how we'll know" line to every ticket or brief, and review it in the retro after launch.
  • Combine a behaviour metric with a perception one (task success with SUS) so neither misleads alone.
  • Sauro and Lewis's Quantifying the User Experience is the standard guide to UX statistics.

Checklist

Learn more

Books, courses and videos to go deeper

Book links go to Amazon. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.

Problem statements and "How might we" · Journey maps · HEART framework · Goals-Signals-Metrics · Task success and time on task · SUS · Measuring AI features

Saved only on this device.

↑ ↓ to move · Enter to open · Esc to close