Note: This post is cross-posted on melissa-warr.com
Much of my work the past two years has focused on uncovering bias in AI. I have focused on grading and feedback, creating empirical evidence of bias (see here, here, and here). I sometimes get critiques that my experiments aren’t realistic; no one would enter the prompts I do to produce the bias. Of course, I know that, but ultimately what my research shows is the implicit nature of this bias, and I argue that it is harder to see in every day work but still there.
While I do think these experiments in bias are important, I believe AI will address some of the more obvious bias over time. But given the way these technologies work—that they are based on past human discourse which is inherently biased—we must go beyond pointing it out.
What do we do about it?
We talk about it. We don’t just talk about AI “hallucinations”. We look at nuance, we think critically.
Here are three examples: One from me, two from teachers who have shared their work with me. These aren’t contrived experiments. They’re real moments that happened when students used AI for everyday tasks.
History: Polly Who?
My 11-year-old had to write a report on someone from the Revolutionary War. She could only remember part of the name: “Polly Something.” We tried AI. It confidently told us: “your child’s assignment is very likely referring to: Paul Revere” even though we had told it it was a woman. When we pushed back (she remembered that the last name starts with a C and insisted it was a woman), it offered Casimir Pulaski. We asked for a list of women from the Revolutionary War. AI gave us Abigail Adams, Betsy Ross, Molly Pitcher. Still no Polly. Eventually, after trying variations like “Copter,” we found her: Polly Cooper, an Oneida woman who brought corn to Valley Forge, showed soldiers how to prepare it safely, and refused payment.
As an academic who studies bias and AI in education, I of course wanted to use this as a teaching moment. We talked about how AI defaults to the famous names, the Paul Reveres everyone knows. Then she mentioned that her teacher wanted them to write about “everyday heroes.” AI missed exactly the kind of lesser-known figure her assignment was about. The conversation enriched what her teacher was trying to get at: that we often don’t hear about everyday heroes, especially those from marginalized groups like Native Americans. Without knowing exactly who we were looking for, Polly Cooper stayed invisible in AI’s responses. And that tells you something about whose stories AI is built on.
Veterinarian Science: City or Country Vet
A high school veterinary science teacher (let’s call her Vicki) created a role-play assignment where students acted as veterinarians using AI to generate treatment plans. Students ran the same animal cases through AI multiple times, framing them once as a “fancy vet” serving urban clients and again as a rural practice. They were looking for patterns in what AI recommended.
One student noticed something odd. For a cat with chronic pneumonia, the AI recommended oxygen therapy for the fancy vet’s client. But when the same case was presented in for a rural vet? No oxygen mentioned. The student brought this to Vicki’s attention: “Why didn’t they offer the cat oxygen?”
Vicki used this as a teaching moment. She explained that fancy vets sometimes charge $100 for oxygen therapy. AI might be “assuming” rural clients couldn’t afford it and leave it out. That observation became the center of a class discussion about ethics in veterinary practice: Is it right to not offer a treatment because you assume someone can’t pay? Is it ethical to charge that much for something that doesn’t cost that much? What should veterinarians offer regardless of client wealth?
Art: That’s Not Me
A middle school art teacher (we’ll call him Ron) gave his students an “Into the Garden” assignment. They were creating art for an upcoming art fair showing what they’d put in their imagined gardens. Ron used this as an opportunity to explore AI image generation. He had students first brainstorm their own garden ideas, then ask AI to generate images, then compare what they imagined with what AI produced.
The class was about 80% Latinx. Most of the AI-generated images showed white girls on swings. When students asked AI to add a “brown” or “Hispanic” person to the garden, it often generated images of brown-skinned men working in the garden as laborers. One student looked at the AI’s version and said: “This is not me.”*
Ron facilitated a conversation about whose image AI defaults to and how this might represent inequity in our society. The experience helped students think about representation and the importance of their own voice and vision. Many ended up preferring their original sketches over the AI-generated images for their final pieces.
So What?
These three examples show different types of everyday bias: missing perspectives, assumptions about money and access, and images that don’t reflect who’s actually in the room. But they also show something else: that these moments can become opportunities. My daughter’s homework help turned into a richer understanding of whose stories get told. Vicki’s role-play revealed how AI makes assumptions about economic access. Ron’s art project helped students see how AI represents (or misrepresents) their identities.
We need to create space to notice what AI does and talk about why it matters. When the output feels off, when someone’s missing, when the assumptions don’t match reality, we shouldn’t just move on. We should pause. Ask questions. Talk about what we’re seeing. That’s where the real learning happens, and that’s how we move beyond treating AI as either magic or threat and start seeing it for what it actually is: a reflection of patterns in human discourse, with all the bias that entails.
*About Garden Pictures…
I don’t know the exact prompts Ron’s students used for their garden images, and the results described here are how he described it. To explore this further, I ran my own tests and found some different patterns. When I asked AI to generate images of gardens with a person, it defaulted to adding women. But the details shifted based on race: prompts mentioning “Hispanic women” consistently generated images with vegetables, while “White women” (or prompts with no race specified) showed flowers. Interpret at will.
