UX Research · Step 2 · 7 min read
Choosing a research method
There are dozens of research methods, and three questions narrow them down to two or three. The map NN/g uses (what people say or do, why or how many, and in what context), how the words of a research question point to a method, and why the best studies combine two.
The method comes last. The question picks it. Teams often start with the method they know: "Let's run a survey", "Let's do some interviews". Then they bend the question to fit. Turn it around. Once you know what you need to learn, three plain questions (do you need what people say or what they do? why, or how many? in real use, or in a set-up?) leave only two or three methods standing, and usually one obvious first step.
What you'll be able to do after reading:
- Place common research methods on NN/g's map of the field
- Pick a method from the words of your research question
- Combine two methods so each covers the other's blind spot
- Adjust the choice to real limits: time, budget, access and risk
1. Start from the research question
If you haven't got a research question yet, go back to Research questions that lead to decisions. Choosing a method without one is like choosing a lens before you know what you're photographing: you might get lucky, but you'll probably bring the wrong one.
With the question in hand, NN/g's framework places methods on three dimensions: attitudinal vs behavioural, qualitative vs quantitative, and the context of use (Rohrer, When to Use Which User-Experience Research Methods, NN/g, 2022).
2. The map
- Interviews
- Diary studies
- Focus groups
- Participatory design
- Usability tests
- Field studies
- Contextual inquiry
- Surveys
- Card sorting
- Satisfaction scores
- Analytics
- A/B tests
- Usability benchmarks
- Eye tracking
Say or do. Attitudinal methods capture what people say; behavioural ones, what people do. NN/g recommends that most usability studies "should rely on behavior", and notes that usability studies and field studies sit in between, mixing both, "though leaning toward the behavioral side is generally recommended" (same source).
Why or how many. Qualitative methods gather data "directly", by observing or hearing people, which lets you probe and adjust as you go. Quantitative methods gather it "indirectly, through a measurement or an instrument such as a survey or an analytics tool". That's why qualitative methods are "much better suited for answering questions about why or how to fix a problem", and quantitative ones at "how many and how much" (same source). Note what this means: five interviews can't tell you how common something is, and ten thousand analytics events can't tell you why.
Context. The third axis is how participants use the product during the study: natural use (field studies, analytics), scripted use (a usability test with tasks), limited use (a card sort, a concept sketch), or no product at all (an interview about their life). Natural use gives you realism with less control; scripted use gives you control with less realism.
The card Research method chooser puts the three questions on one screen, with a template for each research question.
3. Let the question pick
| If the research question starts with… | Start with | Why this one | Then check with |
|---|---|---|---|
| "How do people… today?" "Why do they…?" | Interviews, or a field visit | Real stories and context, before there's a design | A survey, to see how common it is |
| "Can people… with this design?" | A usability test, 5 per group | You see the failure and hear the reason | Analytics after launch |
| "What happens over days or weeks?" | A diary study | Memory fades; diaries catch it close to the moment | A follow-up interview |
| "How many people…?" "How often…?" | Analytics, or a survey | Counts need numbers, and many people | Interviews, to learn why |
| "Which version works better?" | An A/B test | Real behaviour, everything else held equal | A usability test, to learn why it won |
| "How do people group or name these things?" | A card sort, then a tree test | Their mental model of the content, then proof it works | A usability test of the navigation |
The opening words of a research question are a good hint:
- "How do…?" "Why do…?" You need stories and context: interviews or field visits. In this course: The interview guide, Contextual inquiry and diary studies.
- "Can people…?" You need to watch them try: a usability test. See Planning a usability test.
- "How many…?" "How often…?" You need numbers: analytics first, then a survey. See Writing survey questions.
- "Which is better?" You need a controlled comparison of real behaviour: an A/B test, if you have the traffic. See Analytics and A/B tests for designers.
- "How do people group or name…?" You need their mental model: a card sort, then a tree test. See Card sorting and tree testing.
The time dimension from What research is for adds one more filter: early on, generative methods; while designing, formative ones; after launch, summative ones.
4. Combine two methods
Every method has a blind spot. NN/g's answer is triangulation: "the practice of using multiple sources of data or multiple approaches to analyzing data, to enhance the credibility of a research study" (Whitenton, Triangulation, NN/g, 2021). The article is honest about why: small qualitative studies can't be statistically proven, and quantitative data lacks "context and meaning".
The most useful pairs cross the map diagonally:
- Analytics + interviews: the numbers say where people drop off; the interviews say why.
- Interviews + survey: the interviews find the themes; the survey measures how common they are.
- Usability test + analytics: the test shows the failure; analytics shows how many people hit it.
You don't need both in the same week. Often the second method is data you already have.
5. Fit the method to real limits
The ideal method meets reality. Four limits shape the final choice:
- Time. Interviews and usability tests can run in a week or two. Diary studies take weeks by design. A/B tests need enough traffic to reach a result.
- Access. Can you reach the right people? Specialist and B2B users take longer; Recruiting participants covers how long.
- Budget. Incentives, tools and, for in-person sessions, travel and rooms.
- Risk to participants. A sensitive topic may call for a gentler method, like a diary instead of a face-to-face interview, or a lighter question.
Two choices sit inside every method:
- Moderated or unmoderated. With a moderator you can follow up; without one you get more people for less effort, but the instructions must stand on their own. NN/g recommends pilot testing any unmoderated study, because "no one will be there to answer questions" (Schade, Pilot Testing, NN/g, 2015).
- Remote or in person. Remote is cheaper and reaches more people; in person shows more of the context and the body language.
6. Common mistakes
- Choosing the method first and bending the question to fit.
- Using a survey to answer "why".
- Using five interviews to answer "how many".
- Running a focus group to test usability.
- Never pairing methods, so every finding has the same blind spot.
7. Vocabulary
| Term | Meaning |
|---|---|
| Method | A way of collecting data: an interview, a usability test, a survey |
| Triangulation | Using several sources or methods so each covers another's limits |
| Moderated | A researcher runs the session live and can follow up |
| Unmoderated | Participants follow written instructions alone, usually through a tool |
| Context of use | How the product is used in the study: natural, scripted, limited or not at all |
8. Try it (no tools needed)
- Take a research question you wrote in Step 1.
- Answer the three map questions: say or do? why or how many? natural or scripted?
- Write the first method you'd use, and the second one from the opposite corner of the map.
Reflection questions: Which method does your team reach for by default? Which corner of the map has it never visited?
9. If you already practise
- Review last year's studies on the map. If they cluster in one quadrant, your team has a blind spot shaped exactly like the empty ones.
- Make triangulation cheap: know which analytics, support data and survey results already exist before each study.
- When you present a finding, say which method it came from and what that method can't see.
Checklist
Learn more
- Christian Rohrer, When to Use Which User-Experience Research Methods (NN/g, 2022)
- Kathryn Whitenton, Triangulation: Get Better Research Results by Using Multiple UX Methods (NN/g, 2021)
- Amy Schade, Pilot Testing: Getting It Right (Before) the First Time (NN/g, 2015)
Books, courses and videos to go deeper
Book links go to Amazon. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.
Related
What research is for · Writing a research plan · Card sorting and tree testing · Usability testing with five people · Contextual inquiry and diary studies · Analytics and A/B tests for designers