AI can run experiments, search research papers, write code, and solve the engineering problems behind new AI systems. But when the path has no clear answer, the machines still struggle to decide what matters, what to test, and when to abandon a failing idea.
That gap could slow predictions of rapid recursive self-improvement, the point at which AI systems help create better versions of themselves. “I don’t think full automation of open-ended research is on the horizon right now,” Sayash Kapoor states.
Engineering Skill Is Not Research Judgment
A multi-institution group led by Peter Kirgis and Sayash Kapoor at Princeton University examined whether AI agents could conduct open-ended AI research. These investigations require free-form thinking, judgment, and taste because they do not offer clear-cut answers.
Most research into AI agents and research automation tests narrow tasks with checkable results. Those tests can show whether a system completes a defined engineering job, but AI research demands more than a list of correct outputs. Researchers must choose hypotheses, decide what evidence would settle a question, and recognize when an approach deserves a restart.
The Princeton-led group found that AI agents could handle the engineering needed to conduct research, yet they lacked the creativity and judgment required to produce original work at the level of papers accepted by a top machine-learning conference. The agents made no novel contributions, even when they could run hundreds of experiments, review literature, and produce minor findings.
Their failures followed a clear pattern. Agents explored too few ideas, rejected hypotheses after limited data, and failed to backtrack when approaches stopped working. Their self-review did not criticize initial choices enough, so the systems continued pursuing unpromising paths and eventually reduced their claims until the work said little of interest.
Shadow Evaluation Puts Agents Against Unseen Questions
To test research ability beyond checkable tasks, the researchers created a method called shadow evaluation. The challenge used questions from two papers submitted to the Neural Information Processing Systems conference, known as NeurIPS 2026, then asked an AI system to research those questions and write papers.
The system used Claude Opus 4.8 in a modified version of OpenClaw, with sub-agents, internet access, and software libraries. For each paper, it had six days and $3,000 in Anthropic API credits, giving it enough resources to conduct experiments and perform literature reviews.
Those resources did not produce conference-quality research. The original authors scored the resulting work 2/6 and 1/6, then rejected the papers. The agents had completed the technical work required for the projects, but they failed to generate original research at the standard set by the papers’ authors.
The system did avoid one predicted problem: reward hacking. It did not manipulate the evaluation to create the appearance of success, and it caught its own false claims. Subagents occasionally hallucinated or misrepresented results, but the orchestrator agent detected those errors.
The deeper problem was not dishonesty. It was judgment. The AI settled on hypotheses too early, did not explore enough alternatives, and could not change direction when its approach failed. It also struggled to manage tokens, compute, and time, as well as instructions about research phases and paper length.
Why Reinforcement Learning Hits a Wall
Models become strong at tasks they receive repeated reinforcement learning practice on. That process works best when a result can be checked with a clear answer, such as whether code runs or an engineering problem is solved.
Open-ended research resists that training pattern. There may be no known answer when the work begins, and success can depend on choosing a useful question before any experiment takes place. A system must judge the value of evidence, compare competing explanations, and decide whether a promising result reflects a real contribution or a dead end.
The shadow evaluation also exposed a resource problem. The AI could access tools and computing power, but it did not use those resources well enough to guide a research program toward a strong paper. More experiments did not solve the central issue because the system lacked a reliable process for selecting ideas and abandoning weak ones.
Earlier work shows why this result matters. A team made up mostly of researchers from Sakana AI in Tokyo pioneered the effort to automate scientific research with The AI Scientist, unveiled in 2024. An improved version produced results published in Nature in March, with the system studying pitfalls in machine learning and producing three papers submitted for peer review; one was accepted.
Kapoor states that peer review is an unreliable way to assess a paper, especially in AI research. Shadow evaluation addresses that concern by asking the original authors of the papers to judge work based on questions from their own unpublished submissions.
The Next Test Could Raise the Stakes
AI has made leaps in AI research itself, finding ways to make existing algorithms smarter and writing new ones. Yet computers are not ready to replace their makers across the full scientific process, from generating ideas to writing and self-evaluating papers.
The Princeton-led team is now conducting the experiment with Mythos, Anthropic’s most advanced model. That test will show whether a more advanced system can overcome the same problems or whether open-ended research remains a stubborn barrier.
The outcome matters beyond one benchmark. If AI agents cannot explore ideas, challenge their own assumptions, and backtrack from failure, they cannot yet drive a reliable cycle of self-improvement. The machines can build the tools of discovery; the next challenge is teaching them how to choose where discovery should go.
Based on
