AI for UX · For leads · 7 min read
Choosing and piloting AI tools
How a design team should choose an AI tool: start from the task, ask vendors the right questions about data, training, ownership and exit, run a short pilot that measures total time and quality, decide honestly, and roll out with training. Includes a worked example and a pilot plan you can copy.
A great demo is not a decision. Every AI tool looks good on the vendor's examples. What matters is whether it helps your designers on your work, with data you're allowed to use, at a cost that makes sense, and whether you can leave if it doesn't. Choosing a tool for a team is a small research project: a question, a pilot, evidence, a decision.
What you'll be able to do after reading:
- Start a tool decision from the task, not from the tool
- Ask vendors the questions that matter: data, training, ownership, security, cost, exit
- Run a pilot that measures total time and quality, not how impressive it looks
- Decide to keep, extend, set aside or revisit, and roll out with training
AI tools landscape for designers covers a fair trial for one designer. This lesson is the same idea for a whole team.
1. Why it's a team decision
- Most designers can't pick freely anyway. In NN/g's 2026 survey of UX practitioners, 59% could only use an approved set of tools, and the top barriers were output quality (54%) and privacy and security (50%) (NN/g, Oct 2026).
- Feeling faster isn't the same as trusting the work. In Figma's 2025 survey, 78% agreed that AI makes them more efficient, but only 32% said they can rely on its output (Figma, Apr 2025).
- AI amplifies what's already there. Google's DORA research describes AI's main role as "an amplifier, magnifying an organization's existing strengths and weaknesses" (DORA 2025). A tool won't fix a messy process; it speeds it up.
2. Start from the task
Write the decision as a task, not a product: "We want to cut the time from interview recording to shareable findings, without losing accuracy." Then shortlist two or three tools that could do it, including the tools you already pay for.
NN/g's PROVE framework gives a useful rule for comparing them: measure total time, not generation time. "A three-second generation can still create a thirty-minute editing job" (NN/g, Aug 2026).
3. Questions to ask every vendor
Where is it stored, for how long, who can see it?
Retention, region, deletionAre our files or prompts used to train models, on our plan?
Ask per plan, in writingWho owns what we make with it, and what we put in?
Licence, IP, indemnitySingle sign-on, admin controls, connectors, audit logs?
What IT security needs to approvePer seat, per use, or both? What happens at scale?
Include training and admin timeCan we export our work and delete our data if we leave?
Avoid lock-inThe plan changes the answer, not only the brand. For example, as of November 2026, Figma turns AI content training on by default for Starter and Professional teams and off for Organization and Enterprise plans (Figma Help). Anthropic says it doesn't use inputs and outputs from its commercial products to train models by default, with exceptions such as feedback people choose to send (Anthropic Privacy Center). Ask for your plan, in writing.
The UK government's AI Playbook gives good procurement advice for any organisation: work with commercial colleagues from the start, consider who owns the intellectual property, plan how to avoid vendor lock-in, and give suppliers dummy data where possible (GDS, Feb 2025). On ownership of AI output, see Copyright and AI-generated assets.
4. Run a pilot
- 1Task
One sentence: what should get better?
- 2Baseline
Time and quality today.
- 3Pilot
4 to 6 weeks, a mixed group, safe data.
- 4Measure
Total time, quality, errors, feelings.
- 5Decide
Keep, extend, set aside, revisit.
- Real tasks, safe data. Use real work, but data the tool is cleared for. Dummy or anonymised data first.
- A baseline. How long and how good is the task today? Without it, "it felt faster" is all you'll have.
- A small, mixed group. Enthusiasts and sceptics, juniors and seniors.
- Measure what matters: total time including editing, quality against a rubric (see Evals for designers), errors that reached a review, and how people felt.
- Expect weak comparisons. Even careful researchers struggle: in 2026, METR found many developers refused to work without AI, which made their comparison "an unreliable signal" (METR, Feb 2026). Design your pilot knowing that, and pair numbers with interviews.
5. Decide, honestly
PROVE ends with four outcomes: keep using it, run a time-boxed trial, set it aside, or revisit it later, and adds: "If you can't use a tool responsibly for the task you have in mind, don't force it into the workflow" (NN/g). A pilot that ends in "no" is a success: it saved the team from a bad rollout.
6. Roll out with training
If you keep it: write the playbook entry, train people, name an owner, and set a review date. See Guiding designers through AI adoption.
7. Worked example (fictional)
The task: a research team of four wants to synthesise interviews faster.
| Stage | What they did |
|---|---|
| Shortlist | Their approved general assistant (already paid for) vs a specialised research tool |
| Vendor questions | The specialised tool trained on customer data by default on the cheapest plan; the business plan didn't. Recordings were kept for 90 days unless deleted |
| Pilot | Four weeks, six past interviews already synthesised by hand (the baseline), anonymised; same rubric for both tools |
| Results | Both saved time on the first draft; the specialised tool's quotes were more accurate, but its findings missed two behaviours the humans had seen |
| Decision | Time-boxed trial of the specialised tool on the business plan, for one more quarter, with a human check of every finding |
8. Common mistakes
- Choosing from a demo or a social media thread.
- Comparing generation time, not total time including editing.
- Piloting with enthusiasts only.
- Forgetting the exit: how do you get your data out, and what happens to it after?
- Rolling out without training, owner or review date.
9. Vocabulary
| Term | Meaning |
|---|---|
| Pilot | A short, planned trial on real work, with a decision at the end |
| Baseline | How the task performs today, before the tool |
| Total time | Time to a usable result, including prompting, checking and editing |
| Vendor lock-in | When leaving a tool becomes too costly or hard |
| Content training | A vendor using your files or prompts to improve its models |
| AI literacy | The skills and knowledge to use AI well and know its risks |
10. Try it (no code needed)
- Pick one AI tool your team is curious about.
- Write the task it should help with, in one sentence.
- Answer the six vendor questions for your plan, from the vendor's own documentation. Note every question you couldn't answer.
- Sketch a four-week pilot: tasks, baseline, group, measures, and the result that would make you say no.
Reflection questions: Which tool did your team adopt without a pilot? What would you need to see to stop using it?
Checklist
Learn more
- NN/g: How to Decide When an AI Tool Is Worth Keeping (PROVE) (Aug 2026) · AI in UX: Work Is Faster, But Practitioners Have Mixed Feelings (Oct 2026)
- Procurement and risk: UK Government AI Playbook (2025) · NIST AI Risk Management Framework (voluntary; Govern, Map, Measure, Manage)
- Law: European Commission, AI literacy Q&A · US Copyright Office, Copyright and AI Part 2 (Jan 2025)
- Research: DORA 2025 · Figma's 2025 AI report · METR (2025)
Books, courses and videos to go deeper
- AI Snake OilArvind Narayanan and Sayash KapoorBook · Princeton University Press, 2024 · : how to tell real AI capabilities from hype (Amazon)
- AI Fluency: Framework & FoundationsAnthropic AcademyCourse · free online course (external site)
Book links go to Amazon. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.
Related
AI tools landscape for designers · Guiding designers through AI adoption · Measuring AI features · Evals for designers · AI governance and data protection · Copyright and AI-generated assets · Building an AI agent roadmap for your UX team