The Rise of AI Tutors: What Works and What Doesn't

A 2025 Harvard study found that students learning with an AI tutor gained more than twice as much knowledge as those in an active learning classroom. A different study found that uncontrolled AI use led to a 20-40 percentage point decrease in knowledge retention. Same technology. Opposite outcomes.

That gap is the whole story of AI tutoring right now. The tool isn't the variable. The design is. After reviewing the latest research, we think the evidence is clear enough to draw firm lines between what works and what doesn't.

The Evidence for AI Tutoring

Harvard's Landmark Physics Study

The strongest evidence to date comes from a randomized controlled trial at Harvard (Kestin et al. 2025, published in Scientific Reports). Researchers divided 194 undergraduates into two groups: one learned physics concepts through a GPT-based AI tutor, the other through an active learning classroom (already considered a gold-standard teaching method).

The results were striking. Students using the AI tutor learned more than twice as much, with effect sizes ranging from 0.73 to 1.3 standard deviations. They also reported higher engagement (4.1 vs. 3.6 on a 5-point scale) and greater motivation (3.4 vs. 3.1). The AI tutor used Socratic pedagogy, guiding students through questions rather than handing over answers, which appears to be a critical factor in its effectiveness.

Large-Scale Validation

The Harvard study isn't an outlier. Multiple large-scale trials now corroborate the finding that well-designed AI tutoring works:

Carnegie Learning's MATHia underwent a U.S. Department of Education randomized controlled trial spanning 147 schools and more than 18,000 students. Students using MATHia showed nearly double the growth on standardized math assessments compared to control groups, with effect sizes of 0.21-0.38 standard deviations.

Squirrel AI, a Chinese adaptive learning platform, earned a Guinness World Record in December 2025 for conducting the largest AI-versus-traditional-teaching experiment ever: 1,662 students in a rigorous RCT. Their AI system produced greater math learning gains than expert human teachers.

Duolingo's Lilly AI tutor, operating across 40+ languages, has demonstrated a 30% improvement in learning outcomes through adaptive difficulty adjustment, with results comparable to traditional classroom instruction.

The Bloom Benchmark

In 1984, Benjamin Bloom claimed that one-on-one tutoring produced a two-standard-deviation improvement, placing the average tutored student at the 98th percentile. That figure has been critically reappraised since, and the consensus is that it was likely optimistic. Still, high-quality tutoring (human or software-based) clearly produces dramatic learning gains.

Where does AI stand? The Harvard study's effect sizes of 0.73-1.3 standard deviations represent a major step toward Bloom's benchmark. A broader meta-analysis of AI in education found an overall effect size of 0.86, comparable to or exceeding many traditional educational interventions. We're not consistently at two sigma, but the trajectory is clear.

The earlier Ma et al. 2014 meta-analysis, published in the Journal of Educational Psychology, synthesized 107 effect sizes across 14,321 participants. It found that intelligent tutoring systems outperformed large-group instruction (g = 0.42), performed roughly on par with small-group instruction (g = 0.05), and came close to human tutors (g = -0.11, a small advantage for humans). A 2025 study published in npj Science of Learning confirmed that ITS are now nearly as effective as human tutoring in STEM subjects, especially when combined with teacher guidance.

What AI Tutors Can't Do

The Emotional Gap

AI tutors can simulate patience, but they cannot replicate warmth. When a student is struggling not because they don't understand quadratic equations but because their parents are divorcing, an AI tutor has nothing useful to offer. Human teachers build relationships, read body language, and provide the kind of emotional scaffolding that no language model can authentically deliver.

This matters more than it might seem. Educational research consistently shows that the quality of teacher-student relationships is one of the strongest predictors of student outcomes, not just academically, but in terms of persistence, self-regulation, and long-term motivation.

Bias and Fairness Concerns

AI tutoring systems can inherit and amplify biases present in their training data. Stanford researchers found that AI systems falsely flagged essays written by ESL (English as a Second Language) students at higher rates. That should worry anyone building AI systems that make pedagogical decisions based on student writing.

There is also evidence that AI tutors may provide different levels of scaffolding based on demographic profiles, potentially offering less challenging material to students from underrepresented groups. This kind of scaffolding bias, even if unintentional, risks widening the very gaps these tools are supposed to close.

When AI Actually Hurts Learning

The most troubling finding in the literature comes from a 2025 study that measured what happens when students use AI without pedagogical guardrails. The result: a 20-40 percentage point decrease in knowledge compared to students who didn't use AI at all.

This doesn't contradict the Harvard results. It confirms that design matters enormously. The Harvard AI tutor was carefully constructed to use Socratic methods, withhold direct answers, and force students to think. Uncontrolled ChatGPT use, by contrast, lets students bypass the cognitive effort that produces actual learning.

The pattern mirrors what we see in the broader AI-in-education literature: AI as a tool for cognitive offloading harms learning; AI as a tool for cognitive engagement helps it. The technology is the same. The pedagogy makes the difference.

The Isolation Problem

Harvard's Academic Leaders Initiative has warned that AI is "on track to dramatically worsen student isolation." When AI tutors replace study groups, office hours, and peer collaboration, they transform learning from a fundamentally social activity into a solitary one.

This risk is compounded by students' own perceptions. According to a Coursera report from 2026, 4 in 5 students say AI has improved their academic performance, and Inside Higher Ed reports that 44% of students feel more confident when using AI tools. Students feel they're learning more, even when the evidence suggests the opposite may be true in unstructured settings. The gap between perceived and actual learning is itself a danger.

What Makes an AI Tutor Effective: The Socratic Method

A 2024 study comparing Socratic AI tutors with standard chatbots found that Socratic systems better supported reflection and critical thinking. Both Khanmigo and the Harvard GPT tutor used Socratic pedagogy, and both produced strong learning outcomes.

The Socratic approach works because it preserves what learning scientists call "desirable difficulty." The student still has to think. The AI doesn't remove the cognitive load. It redirects it, asking the right question at the right time to keep the student in what Vygotsky called the "zone of proximal development."

There is a practical limitation, however. Students can find Socratic tutors frustrating when guidance feels too vague or indirect. The best systems balance challenge with sufficient support to prevent discouragement, a calibration that remains more art than science.

The Equity Question

The cost difference between human and AI tutoring is staggering: $25-100 per hour for a human tutor versus $10-30 per month for unlimited AI tutoring access. Khanmigo offers individual access for $44 per year and is free for teachers, with over 380 district partnerships.

This price gap represents AI tutoring's greatest promise and its greatest risk simultaneously. On one hand, AI can deliver personalized instruction to students who could never afford a private tutor. On the other, the students who stand to benefit most (those in under-resourced schools and communities) are often the last to gain access to new technology. Without deliberate equity-focused policy, AI tutoring could widen achievement gaps rather than narrow them.

Khanmigo's growth curve illustrates the potential scale: from 68,000 users in 2023 to 700,000 in 2024 to 1.4 million in 2025. But reaching the students who need it most requires more than making the technology available. It requires infrastructure, training, and integration into school systems that are often resistant to change.

Principles That Hold Up Across the Research

The research points to several clear principles for effective AI tutoring:

1. Pedagogy over technology. The AI model matters less than the instructional design. A well-designed Socratic tutor on a modest model will outperform an unstructured interaction with the most powerful AI available.

2. Teacher integration, not replacement. The 2025 npj Science of Learning study found that AI tutoring was most effective when combined with teacher guidance. AI handles the repetitive, patient work of practice and feedback; teachers provide the human judgment, emotional support, and relationship-building that AI cannot.

3. Structured access, not open-ended chat. The 20-40% knowledge decrease from uncontrolled use is a clear signal. Students need AI tools that are designed for learning: tools that ask questions, withhold answers, and verify understanding rather than optimizing for user satisfaction.

4. Process visibility. When students know their learning process is visible (not just their final answers) they're more likely to engage authentically with AI tutoring rather than using it as a shortcut.

AI tutoring has crossed an important threshold. The evidence now shows it can match or exceed traditional instruction, but only when designed with sound pedagogy, integrated with human teaching, and structured to preserve cognitive effort. The same technology that doubled learning gains at Harvard reduced knowledge by 20-40% when used without guardrails.

We think the deployment question will define the next decade of education: do we give students tools that help them think more deeply, or tools that let them stop thinking altogether? The research is pretty clear about which direction works.

Limudi combines AI-guided learning with process-level visibility, so teachers can see how students actually work: every revision, every pause, every moment of genuine effort. See what that looks like.