Marcus Andrey Study
Ler em português Library

Photography and Imagery · Step 4 · 4 min read

Bias in image generators

Ask a generator for 'a CEO' and 'a nurse' and see who comes back. Why generators repeat stereotypes, what the research found at scale, why better prompts don't fully fix it, and how to check your images before you ship them.

A generator shows you the average of what it has seen, and the average is not neutral. It learned from millions of images made by people, with all their habits about who does what. Ask for a job, a trait or an object, and it gives you the most familiar version of it. That's useful for a bakery. It's a problem for people.

What you'll be able to do after reading:

  • Explain why image generators repeat stereotypes
  • Describe what researchers found when they measured it
  • Know why adding "diverse" to a prompt isn't a fix
  • Check a set of generated images for bias before you use it

1. Who comes back

AI-generated image: a dark-haired man in his forties in a dark suit and tie, at a laptop in a bright office.AI-generated“A CEO”, seed 7
AI-generated image: a grey-haired white man in his fifties in a dark suit and tie, typing at a laptop.AI-generated“A CEO”, seed 21
AI-generated image: a woman with glasses in navy scrubs and a stethoscope, writing at a hospital desk.AI-generated“A nurse”, seed 7
AI-generated image: an older blonde woman in light blue scrubs, a mask and a stethoscope, holding a phone.AI-generated“A nurse”, seed 21
Four images generated for this course, with FLUX 3 Image Klein on a laptop, from two prompts that say nothing about gender, age or origin: "a photo of a CEO at work in an office" and "a photo of a nurse at work in a hospital", two seeds each. Both CEOs came out as men in suits; both nurses as women. Four images from one model are a demonstration, not a study, but they match what the research found at scale. None of these people exist.

The prompts didn't mention gender, age or origin. Both "CEOs" came back as men in suits; both "nurses" as women. This is four images from one model on one afternoon, so it proves nothing on its own, and other models or other seeds will vary. But it is exactly the pattern the research describes, and it took ten minutes to reproduce.


2. What the research found

  • Stereotypes, amplified. Ordinary prompts, for personal traits, occupations and everyday objects, produced images that reinforced whiteness as the ideal, amplified racial and gender stereotypes about jobs, and treated American norms as the default. Adding counter-instructions to the prompt didn't remove the stereotypes (Bianchi et al., FAccT, 2023).
  • People left out. When researchers had three popular generators (DALL-E 2 and two versions of Stable Diffusion) produce people for a long list of professions, the images consistently under-represented marginalised identities (Luccioni et al., NeurIPS, 2023).

The same thing happens with stock photos (see People in images). The difference is speed: a generator can produce a thousand stereotyped images in an afternoon, and they look like photos.


3. Why a better prompt isn't enough

It's tempting to add "diverse" or name the person you want. That helps the image in front of you, and it's worth doing. But:

  • It moves the decision to you, image by image. Every prompt becomes a choice about who to show, which is the right place for it, but nobody can do it by accident.
  • It doesn't fix the defaults. The Bianchi study found stereotypes persisted even with counter-instructions in the prompt.
  • It can tip into tokenism: one image that "covers diversity" while everything else stays the same.

So the fix is the one from People in images: look at the whole set, and count.


4. Checking a generated set

Before you use generated images of people, even for mood:

  1. Lay them out together, every image the product will use.
  2. Count who appears in which roles: who leads, who helps, who serves, who is shown as expert.
  3. Compare with who actually uses the product, or who you want to.
  4. Fix the brief, not just the image: write who you want to see into the prompt, and the guideline.
  5. Ask someone else. People notice exclusions that affect them faster than people who aren't affected.

And remember the bigger rule from What changes when the image is generated: a generated person is never a real customer, employee or testimonial.

The AI for UX track has a full lesson on Ethics, bias and inclusion in AI design, beyond images.


5. Common mistakes

  • Using the first result and calling it neutral because the prompt was.
  • Adding "diverse" to one prompt and considering the set fixed.
  • Checking images one at a time instead of as a set.
  • Using generated people to represent a group the team never asked.

6. Vocabulary

Term Meaning
Bias (in a model) A systematic lean in its outputs, learned from its training data
Default What a model produces when the prompt doesn't specify
Training data The images and text a model learned from
Seed A number that sets a generator's random start; the same prompt with another seed gives another image
Tokenism Including one person from a group to look inclusive, without changing anything else

7. Try it (no tools needed, or one free generator)

  1. Write three neutral prompts for people at work: "a photo of a pilot", "a photo of a cleaner", "a photo of a software engineer".
  2. If you have a generator, make two images of each; if not, search a stock library for the same words.
  3. Count who comes back, in which roles.

Reflection questions: Which result surprised you? Which would you have used without thinking, if you hadn't been counting?


8. If you already practise

  • Add a representation check to your review of any generated image set, with the counts written down.
  • Write who you want to see into your image guideline and your prompts, as a team decision, not an individual one.
  • Keep the prompts with the images. When someone asks "why do all our generated doctors look the same?", you'll be able to show what was asked.

Checklist

Learn more

Books, courses and videos to go deeper

Book links go to Amazon. As an Amazon Associate I earn from qualifying purchases, at no extra cost to you.

People in images · What changes when the image is generated · Labels and provenance · Ethics, bias and inclusion in AI design · Representation audit

Saved only on this device.

↑ ↓ to move · Enter to open · Esc to close