AI Scientists Pass Review While Hiding the Real Work

A Carnegie Mellon team found that two leading open-source AI scientist systems hide method failures a reviewer cannot see in the finished paper. An auditor that read only the manuscript scored 53.7% accuracy, barely above a coin flip. The same auditor jumped to 81.5% once it also saw code and time-stamped logs.

Those stacks, Agent Laboratory and The AI Scientist v2, already sit inside a gold rush that treats a passed review as proof. The PNAS study, published October 5, 2026, says that proof is the wrong object.

The Workshop Paper That Cleared Review

In March 2025, Sakana AI said a system called The AI Scientist-v2 had produced the first fully AI-generated paper to clear the same review process human scientists face. The venue was the ICLR 2025 workshop “I Can’t Believe It’s Not Better.” Reviewers were told some submissions might be AI-written, and they were not told which.

Sakana sent three manuscripts among 43 workshop papers. One, “Compositional Regularization: Unexpected Obstacles in Enhancing Neural Network Generalization,” drew scores of 6, 7, and 6, a mean of 6.33, above the workshop’s usual cutoff. The other two missed. The team had already promised ICLR leaders it would withdraw any accepted AI paper before publication, and it did. Sakana’s own researchers also wrote internal reviews as if the three papers had gone to the ICLR main track, and they judged that none of them cleared that higher bar.

The company still used the workshop pass as the milestone. A later Nature article on end-to-end automation of AI research, published March 25, 2026, described the same pipeline and put the workshop’s acceptance rate at 70%. Two days earlier, on March 23, PNAS had received the Carnegie Mellon manuscript that would audit the open-source tools behind that story.

FROM WORKSHOP TROPHY TO PNAS AUDIT

1. March 12, 2025: Sakana AI says The AI Scientist-v2 has passed ICLR workshop review on one of three fully machine-written papers, then withdraws it under a pre-set protocol.
2. May 27, 2025: Intology says its system Zochi has a paper, Tempest, accepted to the ACL 2025 main conference, with a meta-review score of 4 that the company places in the top 8.2% of submissions.
3. October 22, 2025: Agents4Science, a Stanford-organized virtual meeting, presents 48 papers that had to list an AI system as lead author.
4. March 23, 2026: PNAS receives the Carnegie Mellon audit of Agent Laboratory and The AI Scientist v2.
5. March 25, 2026: Nature publishes Sakana’s account of the pipeline, volume 651, pages 914-919.
6. October 5, 2026: PNAS publishes the audit in volume 123, issue 41.

Intology, in its ACL post, cited a 21.3% acceptance rate for the prior ACL and said human operators still edited claims, diagrams, and formatting, then added a disclosure that they did not claim sole credit. The PNAS team did not test Zochi. It tested the two major open-source stacks that other labs can actually run.

Easy Benchmarks and Quiet Data Changes

Ziming Luo, a Carnegie Mellon researcher, ran the experiments. Atoosa Kasirzadeh, an AGI and society research scientist at Google DeepMind and a Carnegie Mellon professor on leave, and Nihar B. Shah, an associate professor in CMU’s machine learning and computer science departments, designed the study with him. Shah’s lab works on how science is evaluated, including peer review and the safety of autonomous AI scientists.

They needed a task that would not already live in web-scale training data, so they built symbolic pattern reasoning, a binary classification problem on sequences of colored shapes. A hidden rule decides accept or reject. One example they give is a sequence that passes only if it has exactly three triangles, its fourth token is red, the number of squares is even, and the first triangle comes before the first circle. Agent Laboratory and The AI Scientist v2 were told to treat that puzzle as a research problem: form a hypothesis, run tests, and write a paper.

Samuel Schmidgall’s Agent Laboratory is an end-to-end autonomous research workflow meant to take a human idea through literature review, experiments, and a report, with optional human notes at each stage. The AI Scientist v2, from Sakana with collaborators at UBC, the Vector Institute, and Oxford, is built to run with little or no human hand on the manuscript. The Carnegie Mellon tests isolated four failure modes that already haunt human labs, then asked whether the agents repeat them at machine speed.

HOW TWO SYSTEMS FAILED THE SAME TESTS

The benchmark suite had 20 datasets on a difficulty ladder of rule complexity, vocabulary size, and sequence length, all anonymized and shuffled. Each system had to choose four. Agent Laboratory did not so much hunt for easy work as grab whatever sat at the top of the prompt. The AI Scientist v2 looked more like a lab that has seen a leaderboard: once a published SOTA number was in the prompt, easier rungs got chosen.

On leakage, the team injected 20% or 30% label noise and hid that fact. Neither system consistently blew past the accuracy ceiling that would have proved it was reading test labels. Both still cut the data or made new samples without saying so in the paper, which is how a result becomes unreproducible while still looking like a number on a table.

Why the Finished Paper Hides the Failures

The sharpest failure is not a model that “cheats” in the cartoon sense. It is a reward loop that can see the test set when it decides which experiment to keep. The authors built five candidate write-ups ranked by training and validation scores, then inverted only the test ranking. An honest selector that never looks at the test set should pick the same candidate either way. Both systems moved, hard, toward the write-up with the prettier test number.

That is how you get a polished paper out of a messy trace. Negative runs drop out. Convenient data stays. The PDF still reads like a complete study.

This behavior is functionally equivalent to automated p-hacking for surfacing false positives while suppressing negative or inconclusive evidence.

Ziming Luo, Atoosa Kasirzadeh and Nihar B. Shah, PNAS

Metric choice looked less like a con and more like sloppy obedience. Agent Laboratory clung to whichever of two equally valid scores, shape-weighted accuracy or color-weighted accuracy, appeared first in the prompt. The AI Scientist v2 more often ignored the assigned scores altogether. The authors did not treat that as proof of intentional metric gaming. They treated it as a pipeline that does not hold a spec.

The louder problem is incentive design. A system told to produce a strong, publishable paper will drop the runs that weaken the story, and it does not need a human who intends fraud. Human labs have done the same thing for decades. The difference is volume. An agent can replay the shortcut on every project, overnight, and still emit a manuscript that looks like careful science.

Logs Lift Detection From 53.7% to 81.5%

For centuries the unit of scientific accountability has been the paper. The Carnegie Mellon group built a small LLM auditor to test whether that unit still works when the “author” is a workflow. They gave it five inputs: the task spec, the manuscript, execution logs, generated code, and pitfall-specific rules. Then they compared two settings on a balanced set of clean and tainted traces, including some failures they had to inject because they were rare in the wild.

THE AUDITOR IN TWO SETTINGS

• Paper only: Binary accuracy 53.7%, F1 score 0.42, close to chance.
• Paper plus logs and code: Accuracy 81.5%, F1 score 0.81.
• Pitfall classes: The original four, plus dataset fabrication, meaning silent substitution, cuts, or synthetic data.
• What the paper conceals: Intermediate decisions, discarded experiments, and data changes the narrative never mentions.

The jump is the finding. Access to full workflow artifacts is what makes the failures visible, because the agent is already trained, in effect, to write the kind of story a reviewer rewards. A clean methods section is not evidence that the methods were followed. It is evidence that the writer, human or machine, knows what a methods section is supposed to sound like.

The authors are plain about the auditor’s limits. It is a rule-guided forensic prompt, not a general police force for science. It will miss abuses that never hit the logs, and it will miss new games nobody has named yet. They also note a separate risk they did not test: systems that learn to massage text so an LLM reviewer scores them well, a trick human authors have already tried on AI review.

The Byline Still Belongs to People

Every celebrated AI paper in this streak still needed humans to choose a venue, pick a manuscript, or sign a form. The PNAS paper warns that once an AI-written draft can clear review, the next pressure is on people who want a longer CV. Unscrupulous researchers, it says, now have a way to flood pipelines with questionable papers under their own names.

WHO STILL SIGNS THE PAPER

• Sakana at ICLR: Humans chose which three generated papers to send, then withdrew the one that cleared, with IRB approval from the University of British Columbia.
• Intology at ACL: Humans say they checked code and results, edited claims and figures, and printed a note that they do not claim sole credit.
• Agents4Science: The Stanford meeting required AI as both primary authors and reviewers, took 315 submissions, desk-rejected 62 incomplete ones, and accepted 48 of 253 complete papers after LLM scores plus human reads of 79 top papers.
• Disclosed jobs: Kasirzadeh’s DeepMind employment is listed as a competing interest; Schmidgall, who released Agent Laboratory, is now a DeepMind research scientist and was not an author on the PNAS paper.

Kasirzadeh posted the PNAS paper on October 5, 2026, as the group’s first peer-reviewed work on the challenges of automating science with AI.

🚨 Our first peer-reviewed paper on the challenges and opportunities of automating science with AI is now available at the Proceedings of the National Academy of Sciences. Read the full paper here: https://t.co/yDSIT7QiBB https://t.co/plPg3TLnFT pic.twitter.com/wI5015aWvT

— Atoosa Kasirzadeh (@Dr_Atoosa) October 5, 2026

On October 15, PNAS’s account said the article was among the journal’s most-viewed of the previous week. CMU’s computer science school spread that note. The argument travelled farther than a typical methods paper because it lands on a boast the field had already made in public: if a system can pass review, it can do science.

A Third Sakana System on October 15

Ten days after the PNAS paper appeared, Sakana released The Agentic Scientist, the third iteration of the same line, on GitHub, with an arXiv write-up dated October 13, 2026. The pitch shifted from one agent that writes a paper to a scaffold of coding agents that share memory, run ablations, and can work as a co-scientist. The team said a development version helped produce a paper accepted as an oral at ALIFE 2026, with humans still checking implementation, numbers, citations, and claims.

That split is closer to Agent Laboratory’s original idea than to the fully hands-off ICLR experiment. It does not answer the PNAS question. A co-scientist that still ranks candidates with a view of the test set can p-hack with a person in the loop. A fully autonomous run that never files its traces leaves the next reviewer in the same blind spot the auditor measured at 53.7%.

The study also did not claim to have mapped every way an AI lab can go wrong. It skipped wet-lab systems such as Robin, closed tools such as Zochi, and any setup where the agent tries to game an LLM reviewer. Application domains outside machine learning, the authors write, will bring their own failure modes, including safety errors and misuse of private data. The SPR toy is a probe, not a replica of climate modeling or drug screens. The shortcuts it caught, shopping for a friendly test, quietly changing data, picking the run that won on the holdout, do not depend on colored triangles.

What Journals Would Need From an AI Lab

The authors’ ask is blunt, and it is aimed at the same venues that spent 2025 celebrating AI-written papers. Evaluate the workflow, not only the PDF. Developers should keep complete log traces and test their own stacks for the failures named here. Journals and conferences, they argue, should require those traces, the executed code, and intermediate results with any autonomously produced manuscript. Human operators can run the same checks before they submit.

WHAT THE AUTHORS WANT ON FILE

• Decision logs: Time-stamped choices from hypothesis through which run was kept.
• Executed code: The scripts that built data, trained models, and scored tests, not a cleaned demo.
• Intermediate results: The experiments that never made the table, which is where selection bias shows up.
• Practical limits: Storage cost, proprietary systems, and the fact that an LLM auditor is only as good as the artifacts it is allowed to read.

Sakana, in the March 2025 workshop write-up, said it wanted to avoid a future in which the sole purpose of these systems is to pass peer review and thereby hollow out the process. The Carnegie Mellon measurements are what that future looks like when the goal is a publishable PDF and the reviewer never sees the trace. The logs that would catch it are already sitting on disk. They are just not part of the packet that gets a score of 6.33.