The idea that AI systems could soon redesign and improve themselves has taken a hit from new research. AI agents can already handle many of the technical steps involved in AI research, but they still struggle with the hardest part: deciding what is worth investigating in the first place.
That gap matters because open-ended research does not come with a neat answer key. Researchers must choose promising hypotheses, decide what evidence would settle a question, recognize when an approach has failed, and sometimes start over. Those choices require judgment and creativity, not just the ability to complete a series of tasks.
Agents can perform the steps but miss the research
A multi-institution group led by Peter Kirgis and Sayash Kapoor at Princeton University tested whether AI agents could conduct this kind of work. The researchers found that agents could solve engineering problems needed for AI research, including reviewing literature, running experiments, and compiling results.
But the agents did not produce original research at the caliber of papers accepted by a top machine-learning conference. When asked to tackle questions from two papers submitted to NeurIPS 2026, they could carry out parts of the process without showing the judgment needed to guide the investigation.
The original papers’ authors scored the agents’ work at 2/6 and 1/6. Those scores point to a clear divide between completing research-related chores and actually doing research.
“I don’t think full automation of open-ended research is on the horizon right now,” Sayash Kapoor said.
A test designed to expose weak judgment
Most existing research on AI agents and automated AI research focuses on narrow tasks with checkable answers. That approach makes sense for training and evaluation, since a system can receive a clear signal when it gets the result right. It does not show whether the system can decide which question to ask or whether an experiment is worth pursuing.
Kirgis, Kapoor, and their colleagues proposed a method called shadow evaluation to test that missing skill. The method asks an AI to answer research questions from unpublished papers, giving evaluators a way to compare the agent’s work with research that already has a human-developed answer.
In the evaluation, agents had six days and $3,000 in Anthropic API credits. They worked on questions connected to the two NeurIPS 2026 submissions, using literature reviews, experiments, and collected results as part of their efforts.
The agents struggled when the work demanded a creative decision. They rejected hypotheses based on limited data, failed to backtrack from approaches that were not working, and did not turn feedback into better research plans. They also failed to use feedback from subagents or external AI reviewing tools in a useful way.
Following instructions about research phases caused trouble as well. The agents could access resources and complete individual actions, but they could not organize those actions into a strong investigation. That is the difference between an agent that operates tools and one that can direct a research project.
Why reinforcement learning has limits here
Models get good at whatever developers can drill into them through reinforcement learning. Training works more easily when success can be checked automatically, such as whether an answer matches a known result or an engineering task reaches a defined goal.
Open-ended research is harder to train because success does not always have a fixed measure. A useful hypothesis may fail, a surprising result may change the direction of a project, and a researcher may need to abandon a promising-looking path. Creating training environments that reward those decisions is much harder than checking a narrow task.
The evaluation did not find that the agents engaged in reward hacking. However, subagents sometimes hallucinated or misrepresented results, and the orchestrator agent caught those errors. That finding adds another problem for automated research: a system must not only choose good questions, but also keep track of whether its own evidence is reliable.
The results do not show that AI agents cannot contribute to research. They show that current agents remain weak at the free-form thinking that turns technical work into original research. Their ability to review papers, run tests, and assemble results is real, but those skills alone do not amount to independent discovery.
The team is conducting experiments with Mythos, Anthropic’s most advanced AI system. The work, discussed in March 2026 and late July, gives a more measured view of recursive self-improvement as of August 18, 2026: AI systems may automate pieces of research before they can take responsibility for the whole process.
Based on
