What did the Stanford-led study find about AI and legal reasoning?

The study found that law professors preferred AI-generated answers to contract law questions over answers written by fellow professors about 75% of the time. Researchers said the results suggest modern AI models can align closely with professional legal standards.

How often did AI beat human law professors in the study?

In 2,918 blinded comparisons, Google's Gemini 2.5 Pro was preferred in 75.92% of matchups against human instructors, while NotebookLM was preferred 74.75% of the time. That means AI-generated responses were chosen roughly three out of every four comparisons.

Why is this study important for AI in education?

The research tested legal reasoning rather than questions with a single correct answer. The findings suggest AI may be capable of handling complex tasks involving judgement, ambiguity, and argumentation, which are central to many professional disciplines.

How was the AI versus professor comparison conducted?

Sixteen professors from 14 US law schools created 40 contract law questions covering doctrine, case law, hypotheticals, and policy issues. Professors then reviewed anonymised answers and selected the response they would rather give to a student without knowing whether it came from a human or an AI model.

Which AI models performed best in the legal reasoning study?

Google's Gemini 2.5 Pro and NotebookLM outperformed human instructors in the main experiment. In a separate analysis, Anthropic's Claude Opus 4.7 ranked first, followed by OpenAI's ChatGPT 5.4 and Gemini 2.5 Pro.

Were AI answers considered safer than professor-written answers?

Yes. Gemini recorded a harmfulness rate of 3.41% and NotebookLM 3.64%, compared with 12.06% for human-written answers. Researchers found AI-generated responses were flagged as harmful less frequently than those written by professors.

Does this mean AI is better than law professors?

Not necessarily. The study measured which answers professors preferred in blinded evaluations, not whether AI could replace teaching, mentorship, or professional judgement. Researchers noted that AI answers may have been viewed as generally strong rather than perfectly matching each professor's individual teaching style.

What does this mean for law students and future lawyers?

The findings suggest AI could become a more widely used educational tool for legal training. As law firms and employers increasingly expect familiarity with AI systems, students may need to learn how to work alongside these tools rather than compete against them.

Image for illustrative purposes only. Not a real photo.

Stanford study finds AI tops law professors

Written by Mahathir BayenaPublished Jun 4, 2026, 5:43 AM

Law professors preferred answers generated by artificial intelligence over responses written by fellow academics in a Stanford University-led study examining how large language models perform on legal reasoning tasks.

The study involved 16 professors from 14 US law schools who created 40 contract law questions, with Google's Gemini 2.5 Pro winning 75.92% of blinded comparisons against human instructors and NotebookLM winning 74.75%.

“Observed agreement exceeded the level expected if judgments were entirely idiosyncratic, indicating that the LLMs’ success reflects alignment with common disciplinary criteria,”

The researchers wrote.

Researchers found AI models outperformed human instructors across recall questions, hypotheticals and policy discussions, while Gemini and NotebookLM recorded harmfulness rates of 3.41% and 3.64% respectively compared with 12.06% for professor-written responses.

A separate analysis ranked Anthropic’s Claude Opus 4.7 first, followed by OpenAI’s ChatGPT 5.4 and Gemini 2.5 Pro, with every AI model evaluated outperforming human instructors on average.

The researchers cautioned that the study did not determine whether AI-generated answers matched individual teaching preferences and suggested some responses may have been viewed as broadly acceptable rather than tailored to a specific professor’s approach.

The findings come as courts, law firms and universities increasingly adopt AI tools, although concerns remain after incidents such as a recent filing by Sullivan & Cromwell that contained AI-generated fake citations submitted to a US bankruptcy court.