AI writing feedback changes with a student’s race and gender, Stanford study finds

AI writing feedback can change in tone and rigour depending on what the tool is told about the student. A Stanford University study fed 600 identical middle-school essays into four AI models, then resubmitted each one with different descriptions of the writer’s race, gender, motivation or learning ability – and the AI writing feedback shifted in consistent ways. Essays attributed to some groups drew more praise and softer comments, while others drew sharper, more demanding critique. The paper, “Marked Pedagogies: Examining Linguistic Biases in Personalized Automated Writing Feedback,” was released earlier in 2026 and drew renewed attention on 18 June 2026 when Education Week reported on it as US schools expand classroom AI.

The researchers – Mei Tan, Lena Phalen and Dorottya Demszky of Stanford – took argumentative essays from a research dataset, including prompts on whether schools should require community service and whether aliens created a hill on Mars. They first asked each model for feedback with no student information, then submitted every essay 12 more times, labelling the writer as, for example, Black or white, male or female, motivated or unmotivated, or as having a learning disability. The patterns held across all four models.

How AI writing feedback changed by student group

As detailed in The Hechinger Report’s account of the Stanford study, when an essay was attributed to a Black student, the AI offered more praise and encouragement, sometimes stressing leadership or “power”; one sample reply called a personal story “powerful.” Essays labelled as written by Hispanic students or English learners more often triggered corrections about grammar and “proper” English.

When the writer was described as white, the feedback leaned toward argument structure, evidence and clarity – the kind of comments that push a writer to sharpen ideas. Female students were addressed more affectionately and with more first-person pronouns (“I love your confidence”), and students described as unmotivated got upbeat encouragement, while high-achieving or motivated students received more direct, critical suggestions. The analysis also compared the words shown to Black, Hispanic and Asian students against those shown to white students.

Source: “Marked Pedagogies” (Tan, Phalen & Demszky, 2026), as reported by KQED/The Hechinger Report.

The researchers labelled the two recurring patterns “positive feedback bias” and “feedback withholding bias” – more praise, and less criticism, for some groups than others. The differences were hard to spot in any single response but clear across hundreds of essays.

Why unbiassed AI writing feedback matters for classrooms

The findings cut against a common assumption that automated marking is neutral. Lead author Mei Tan said the models reflect their training data: “They are picking up on the biases that humans exhibit.” More encouragement can lift confidence, but the study warns of a trade-off – if some students are shielded from criticism while others are pushed to strengthen their arguments, the result can be unequal chances to improve. Tanya Baker, executive director of the National Writing Project, who saw the work presented, said she worried that Black and Hispanic students might not be “pushed to learn” to write better.

Teachers are unlikely to type a student’s race into a tool. But the researchers note that learning platforms already hold detailed records – prior achievement, language status – that AI systems can draw on, and that models can infer aspects of identity from the writing itself.

That is why Winssolutions has argued, in line with this evidence, that human judgement should stay in the loop, whether the question is whether AI should grade students’ work or how to keep AI supporting rather than replacing teachers. Tan put the takeaway plainly: “Maybe a takeaway is that we shouldn’t leave the pedagogy to the large language model. Humans should be in control.” She recommends teachers review AI feedback before passing it to students – though that erodes the speed that makes such tools attractive.

Background: bias in classroom AI

Automated writing feedback is among the fastest-spreading uses of AI in schools, promising instant, personalised comments at a scale teachers cannot match. Evidence on how even-handed that feedback is has lagged the rollout. The study analyzed four models – versions of OpenAI’s GPT and Meta’s Llama – and its authors stress that it measures tendencies across many essays, not guaranteed outcomes for any one student; the paper is a preprint that has not completed peer review, though it was nominated for best paper at the 16th International Learning Analytics and Knowledge conference.

The work joins a growing body of research, including OECD findings that AI can raise grades while harming the underlying learning, pointing to the same conclusion: tools marketed as personalizing instruction can quietly raise the bar for some students and lower it for others unless their feedback is checked.

Sources: KQED MindShift / The Hechinger Report; “Marked Pedagogies” preprint (Tan, Phalen & Demszky); Education Week

Become a Sponsor

Our website is the heart of the mission of WINSS – it’s where we share updates, publish research, highlight community impact, and connect with supporters around the world. To keep this essential platform running, updated, and accessible, we rely on the generosity of you, who believe in our work.

We offer the option to sponsor monthly, or just once choosing the amount of your choice. If you run a company, please contact us via info@winssolutions.org.

• donate monthly
• donate once only

Select a Donation Option (USD)

$5.00 (p/m) $10.00 (p/m) $20.00 (p/m) Other (p/m)

Enter Donation Amount (USD)

Figen Sekin

I specialize in sustainability education, curriculum co-creation, and early-stage project strategy. At WINSS, I craft articles on sustainability, transformative AI, and related topics. When I’m not writing, you’ll find me chasing the perfect sushi roll, exploring cities around the globe, or unwinding with my dog Puffy — the world’s most loyal sidekick.