University of Maryland Trial Finds AI Tutoring Access Lowered College Grades

A randomized trial of 2,379 undergraduates at the University of Maryland found that offering students access to an AI tutor during a semester lowered their final grades by 0.37 standard deviations and reduced their engagement with course materials, according to a working paper posted October 2026 by the Annenberg Institute at Brown University. The study, led by University of Maryland researchers Jing Liu and Tracy Sweet, is among the largest randomized evaluations yet of AI tutoring in a real college classroom rather than a lab setting.

How the AI tutoring in college courses trial was run

The trial, conducted across 34 course sections and 30 instructors in fall 2025, covered disciplines spanning STEM, humanities, social sciences, education and policy. Instructors were randomly assigned, within blocks of courses, to offer their students access to a Virtual Study Assistant (VSA) built on OpenAI’s GPT-4o model with retrieval-augmented generation grounded in each course’s own materials, the paper states. A second, “exact-match” analysis compared only courses where multiple instructors taught identical content, isolating the effect of tutor access from differences in course design.

In the exact-match sample, students offered the AI tutor scored 4.12 points lower on a 0-100 scale in final grades, a drop of 0.37 standard deviations, and their course letter grades fell by 0.13 standard deviations, the researchers reported. Engagement on the university’s learning management system fell more sharply: page views dropped 0.37 standard deviations, active days declined 0.51 standard deviations, and total participation fell 0.90 standard deviations among students with tutor access. Homework scores showed no measurable change.

Why access alone did not help, and who lost the most

Only 14.63% of students offered the tutor used it at all, and of those who did, 73.8% of requests asked for direct answers or explanations rather than practice or feedback, the paper found. Less than 1% of requests asked the tool to review the student’s own work. The researchers concluded that the tool most often substituted for study activities students would otherwise have done themselves, rather than adding new instruction on top of them. Lead researcher Jing Liu said the findings “do not establish that purpose-built AI tutoring tools are not beneficial,” adding that “when evaluating an AI tool’s impact on learning, we need to consider what it might replace,” according to EdTech Innovation Hub.

First-generation students experienced roughly twice the academic losses of continuing-generation students in the trial, the paper reported, raising equity concerns about rolling out general-purpose AI tutoring without structure. Most instructors in the trial left the tool in its default, direct-instruction configuration rather than switching it into a tutoring mode designed to withhold answers and prompt reasoning instead.

The result adds a cautionary data point to a mixed research record on AI tutoring. A two-year randomized trial of Khan Academy’s Khanmigo tool found only small maths gains, while a separate Stanford analysis found that students largely ignored AI tutors when simply given access without structured class time, echoing the Maryland trial’s low usage rate. By contrast, a controlled StudentBench evaluation found AI tutoring matched human tutoring on GRE-style learning gains under more structured conditions, and the OECD has separately warned that AI use can raise grades while weakening the underlying learning it is meant to support, a concern this trial’s low-structure design appears to confirm from the opposite direction: unstructured access that lowered grades rather than inflating them without learning gains.

Background: AI tutoring research in higher education

Universities and K-12 systems have rapidly adopted large language model tutoring tools since 2023, often without the randomized evidence that accompanies new instructional interventions in other fields. The Annenberg Institute at Brown University hosts EdWorkingPapers, a working-paper series used by education economists to circulate findings ahead of peer review, and this study is registered there as EdWorkingPaper No. 26-1598. The University of Maryland trial is among the first to randomize AI tutor access at the instructor level within real, graded college courses rather than in a lab or a single course, giving it broader relevance to how universities roll out generative AI tools across departments. The researchers frame their results as evidence that access to a capable AI tool is not sufficient on its own, and that how a tool is configured and integrated into coursework, not just its underlying model quality, determines whether it helps or displaces learning.

Sources: EdWorkingPapers (Annenberg Institute at Brown University); EdTech Innovation Hub

Featured image: photo by Ludovic Delot on Pexels (free Pexels license).

Become a Sponsor

Our website is the heart of the mission of WINSS – it’s where we share updates, publish research, highlight community impact, and connect with supporters around the world. To keep this essential platform running, updated, and accessible, we rely on the generosity of you, who believe in our work.

We offer the option to sponsor monthly, or just once choosing the amount of your choice. If you run a company, please contact us via info@winssolutions.org.

• donate monthly
• donate once only

Select a Donation Option (USD)

$5.00 (p/m) $10.00 (p/m) $20.00 (p/m) Other (p/m)

Enter Donation Amount (USD)

Figen Sekin

I specialize in sustainability education, curriculum co-creation, and early-stage project strategy. At WINSS, I craft articles on sustainability, transformative AI, and related topics. When I’m not writing, you’ll find me chasing the perfect sushi roll, exploring cities around the globe, or unwinding with my dog Puffy — the world’s most loyal sidekick.