AI models are becoming powerful research tools, but a new evaluation reveals a striking divide in mathematical reasoning. Proprietary systems from frontier labs are pulling far ahead of open-weight models when they face research-level mathematics, turning a once-hidden gap into a measurable result.
The First Proof project released its third batch of results on October 14, 2026, with a clear goal: objectively evaluate the mathematical reasoning of publicly available AI models. This round shows that the strongest systems can solve problems in a manner acceptable for peer review, while the best open-weight model solved none.
A Benchmark Built for Real Mathematical Research
First Proof focuses on research mathematics rather than routine exercises or familiar textbook examples. That distinction matters because models have become very good at proving well-defined statements, yet success on those tasks does not capture the full challenge of research-level work.
Mohammed Abouzaid, a mathematician at Stanford University and member of the First Proof team, described the result in direct terms: “The gap between the open-weight models and frontier models is very large in research mathematics—much larger than in other domains.”
The evaluation included OpenAI’s GPT 6 Astra and Anthropic’s Claude Opus 5.5. Google did not participate. ProofCouncil, an aggregate model, delivered the strongest performance by solving nine out of the 10 research problems in a manner acceptable for peer review.
That score marks a major jump from earlier rounds. First Proof released its first-round results in February 2026, when the top model got just a few problems correct. In June 2026, the winner got six or seven problems correct. The third batch pushed the leading result to nine.
Open-Weight Models Face a Huge Challenge
The open-weight results tell a very different story. Kimi K3, the leading open-weight model in the evaluation, solved exactly zero problems. That result places it far behind ProofCouncil and shows how large the divide remains between publicly available model weights and proprietary systems from frontier labs.
Lauren Williams, a mathematician at Harvard University and member of the First Proof team, urged the field to respond: “The open models are really, really far behind, so we would urge the community to invest a lot into improving them.”
The gap is not a small difference in polish or speed. It reaches the core ability to produce mathematical arguments that meet a research standard. ProofCouncil solved nine problems, while Kimi K3 solved none, creating a sharp contrast in the same evaluation.
These results also add detail to a much larger debate about AI and mathematics. OpenAI’s solution of the Navier-Stokes problem last month, in September 2026, convinced many that large language models were on the verge of superseding human mathematicians. Yet OpenAI’s new model attempted thousands of problems, most of which it did not solve, and outside mathematicians still cannot access that model.
That combination creates a powerful tension. A model can produce a breakthrough that captures public attention, but a broad evaluation can still reveal major limits across thousands of attempts. First Proof’s results shift the focus from one celebrated solution to performance across a set of research problems.
What Comes Next for AI Mathematical Reasoning?
The benchmark may also be approaching a turning point. Models have become very good at proving well-defined statements, so testing frontier models on that task might not remain relevant in the future. If systems continue improving, evaluators will need to measure forms of mathematical reasoning that demand more than proof completion.
For now, the results establish a clear hierarchy. ProofCouncil reached nine out of 10 acceptable solutions, earlier leaders reached only a few and then six or seven, and Kimi K3 reached zero. Those figures make the progress easy to track while exposing the distance between frontier models and open-weight systems.
First Proof is turning AI benchmarking into a visible part of mathematical research. Its third batch does not settle whether AI will supersede human mathematicians, but it does show where current systems stand: frontier models can produce research-level work, while open-weight models have a long climb ahead.
The next question is no longer whether AI can solve difficult mathematical problems. It is how broadly those abilities can spread, how openly they can be tested, and whether open models can close the gap.
Based on
