New research suggests AI writing feedback tools can deliver sharply different coaching when the system is given information about a student’s race, gender, language, disability status, or achievement level. The findings come from work presented by doctoral researchers Mei Tan and Lena Phalen, associated with Stanford University’s Institute for Human-Centered AI. In the described study, researchers tested AI models including GPT-4o, GPT-3.5-Turbo, and Meta’s Llama variants, using 600 eighth-grade persuasive essays. When AI tools were prompted with background characteristics, the quality and depth of feedback shifted—raising concerns about whether “personalization” aligns with pedagogical goals or instead introduces biased or inconsistent instruction. The report notes that educators may assume modern language models provide accurate, fair differentiation, but the research indicates they may use proxies in ways that change student experience even with identical writing samples—an issue that matters for classroom and academic-integrity policy decisions.
Get the Daily Brief