AI flattens identity into stereotypes
1 min read · June 12, 2026
New Power Labs
A 2026 study found AI-generated personas can overemphasize racial markers and reduce complex identities into stereotyped cues.
Researchers at Penn State's College of Information Sciences and Technology tested GPT-4o, Gemini, and DeepSeek by assigning them over 1,500 demographic personas, then asked each persona questions about its life and compared the responses to those of real people with similar backgrounds.
An AI persona built around a 50-year-old African American woman discussed gospel music, tough love, natural hair care, and social justice. Yet, the 141 real women with that background mentioned one or two of such topics at most, and instead focused on talking about work, parenting, volunteering, and health.
The chatbots produced responses that appeared rich and human-like, but in fact built on culturally coded language rather than lived experience. The researchers identified four harmful patterns: stereotyping, exoticism, erasure, and benevolent bias, the last of which uses polite or positive language to reduce and distort without triggering bias filters.
Organizations are increasingly using AI-generated persona simulations as stand-ins for real human participants in research. If the personas underlying are built on stereotypes rather than authentic experience, the errors shape the research findings, the products, and services, further exacerbating stereotypes and biases.
What these systems learned is how diverse communities have been described, not how they live. Representation without reality becomes caricature.
Narinder
New Power Labs
Like what you’re reading? Subscribe to get weekly Equity Shots in your inbox.
