AI can make research easier to challenge. Determining whether a challenge is right still requires transparent methods and human judgment.
What to Know
- AI tools can help check research code, data, citations, and modeling choices.
- An anonymous audit has challenged work involving economists Emmanuel Saez and Gabriel Zucman.
- Its disputed findings are not independently established corrections.
- A Harvard study found AI-generated manuscript reviews scored highly against expert reviews in its test.
- Policymakers should demand reproducible evidence from both a study and its critics.
A published study can influence a tax proposal long before another researcher has time to check its calculations. AI is changing that timetable. Given a paper and its supporting files, a tool can help rerun calculations, compare cited sources with claims, and flag assumptions for closer examination.
A recent anonymous AI-assisted review of work involving Saez and Zucman illustrates both the promise and the risk. Economist Tyler Cowen publicized the audit amid debate over wealth taxes. Its authors distinguish alleged errors from debatable modeling choices and acknowledge that no finding overturned a work’s core result outright. Its claims still require independent examination.
Faster Scrutiny, Not an Automatic Verdict
Researchers traditionally needed substantial time and specialized expertise to reproduce another team’s calculations. An AI-assisted review can examine a paper alongside its appendices, code, and available data, then identify calculations or citations worth checking. That makes an initial audit faster and gives researchers outside a journal’s usual network another way to question influential work.
Speed does not make every flagged problem an error. A calculation that fails to match an author’s published data is different from an alternative assumption that produces a different estimate. The Saez–Zucman audit itself separates what it calls defects from contested choices. It also acknowledges limits, including confidential data it could not inspect. Readers should treat its classifications as claims to test, not a final verdict on the researchers or their policy conclusions.

The audit presents its own breakdown of claimed defects and contested choices; those classifications have not been independently established, via Saez–Zucman AI-assisted audit.
A separate study by Harvard researchers Gary King and Steven Worthington compared AI-generated feedback with unusually strong human reviews of manuscripts. AI reviews scored better under the study’s measures. That finding suggests AI could help authors discover weaknesses earlier and make useful feedback available to more researchers.

AI reviews received stronger ratings in this manuscript-review experiment; the figure does not establish that either reviewer reproduced the papers’ findings, via Gary King and Steven Worthington.
Manuscript feedback is not the same as independently reproducing a study’s results. A persuasive review may identify a questionable method without proving that a calculation is wrong. Human experts still need to examine the data, assess whether alternative assumptions are reasonable, and decide which findings withstand scrutiny.
Peer Review Beyond the Journal
Traditional peer review usually happens before publication and depends on a limited number of invited specialists. AI-assisted checks can continue after publication, when other researchers, journalists, or public agencies want to examine a claim. More people can raise specific questions without waiting for a journal to organize a formal review.
Wider participation also creates a new quality-control problem. An automated critique can sound confident while misreading a method or counting the same disputed assumption across several papers. Publishing the audit’s prompts, code, and checking steps would let others test the critique rather than asking them to trust its conclusions.
Better access to review should therefore mean more opportunities to investigate evidence, not a race to declare winners. Journals and researchers still have a role in correcting genuine mistakes and explaining why reasonable analyses sometimes reach different results.
What Policymakers Should Ask
Before relying on an AI audit, officials should ask whether its methods, calculations, and source materials can be inspected. They should separate arithmetic mistakes from reasonable disagreements about assumptions and seek responses from the original researchers. Applying comparable checks to studies on competing sides of a policy debate would make the findings more useful.
Wrap Up
AI can lower the time and effort needed to question research that shapes public policy. Faster checks may bring overlooked mistakes to light and give independent researchers a stronger voice.
An AI-generated criticism is not proof that a published conclusion is false. Its calculations, assumptions, and access to underlying data need scrutiny too.
Public trust will grow when original authors and their critics both show their work. Policymakers can then weigh verified findings rather than the reputation of a researcher or the speed of a machine.