What AI tutors can already do

Computer tutors came close to people before chatbots

In a review of experiments comparing human tutoring, computer tutoring and no tutoring, Kurt VanLehn found human tutoring had an effect size of 0.79 and step-by-step intelligent tutoring systems 0.76: nearly as effective tandfonline.com. A 2016 meta-analysis of 50 controlled evaluations found a median gain of 0.66 standard deviations over conventional instruction, the 50th to the 75th percentile journals.sagepub.com. Gains depended on how they were tested, and these systems predate today's chatbots.

Chatbot tutors in real classrooms

In a randomized experiment in a Harvard physics course, students using a GPT-4 tutor built on the same teaching practices as the in-class lessons had median learning gains over double those of the in-class group, in less time pmc.ncbi.nlm.nih.gov. One course, one university, short-term outcome.

In Nigeria, a six-week, teacher-supported after-school program using Microsoft Copilot raised English scores by 0.23 standard deviations in a World Bank randomized trial econpapers.repec.org.

In Ghana, students who used Rori, an AI math tutor delivered by chat, two 30-minute sessions a week for 8 months, had math growth with an effect size of 0.37 over control schools arxiv.org. That is a preprint, year 1 results, 11 schools.

In an online experiment with 274 adults, math hints written by ChatGPT produced learning gains that did not differ significantly from hints written by human tutors journals.plos.org. ChatGPT's help failed quality checks on 32% of problems before an error-reduction step.

When AI stops helping

Students who used a chatbot to replace learning tasks covered more topics but understood each one less; those who used it to add explanations understood more, and the gap between students with low and high prior knowledge widened arxiv.org. High school students who practiced math with an unrestricted chatbot scored 17% lower on their own exam than students who never had it, while a tutor version with guardrails avoided most of that drop econpapers.repec.org.

AI can teach the lesson. What it is for, and how it is used, is the question this issue puts on the table.

Sources

1. Kurt VanLehn, "The Relative Effectiveness of Human Tutoring, Intelligent Tutoring Systems, and Other Tutoring Systems," Educational Psychologist 46(4), 2011. tandfonline.com
2. James A. Kulik and J. D. Fletcher, "Effectiveness of Intelligent Tutoring Systems: A Meta-Analytic Review," Review of Educational Research 86(1), 2016. journals.sagepub.com
3. Kestin, Miller, Klales, Milbourne and Ponti, "AI tutoring outperforms in-class active learning," Scientific Reports 15:17458, June 3, 2025. pmc.ncbi.nlm.nih.gov
4. De Simone, Tiberti, Barron Rodriguez, Manolio, Mosuro and Dikoru, "From Chalkboards to Chatbots," World Bank Policy Research Working Paper 11125, May 2025 (abstract). econpapers.repec.org
5. Henkel, Horne-Robinson, Kozhakhmetova and Lee, "Effective and Scalable Math Support: Evidence on the Impact of an AI Tutor on Math Achievement in Ghana," arXiv:2402.09809, February 15, 2024 (preprint). arxiv.org
6. Zachary A. Pardos and Shreya Bhandari, "ChatGPT-generated help produces learning gains equivalent to human tutor-authored help on mathematics skills," PLOS ONE, May 24, 2024. journals.plos.org
7. Lehmann, Cornelius and Sting, "AI Meets the Classroom: When Do Large Language Models Harm Learning?" arXiv:2409.09047, 2025 (preprint). arxiv.org
8. Bastani, Bastani, Sungu, Ge, Kabakci and Mariman, "Generative AI without guardrails can harm learning," PNAS 122(26), 2025 (abstract). econpapers.repec.org

See an error? Report a correction.