97

I’ve submitted a paper to a reputable journal and received two reviews asking for a revision. One of the reviews is on-topic and helpful. The other, however, I suspect was generated by an AI based on the abstract only. I don’t want to debate the decision (the second reviewer is fair), but I wonder if I should mention my suspicions to the journal editor.

Should I? I doubt I can prove the use of AI. All I have is a strong suspicion. Plus, I don’t want to hurt my article. Yet, using AI for generating (not merely styling) reviews seems like such a bad practice that maybe it’s worth letting the editor know.

My suspicions are based on several things:

  • First, the style is off. The review consists only of very long questions like:

    In terms of the paper's stated goal of contributing to the literature, how well does it provide [contribution mentioned in abstract] for [use-case mentioned in abstract]?

    There are no suggestions, no feedback, and no statements. It’s only broad questions suspiciously rephrasing each line of the abstract (and heavily re-using entire phrases from it).

  • Second, the list of suggested articles is strange and includes existing papers (only names) from irrelevant fields.

  • Third, out of curiosity, I've run the text via several online AI-detection tools, and all of them conclude it’s AI (which I am aware is not a valid proof on its own).

Wrzlprmft
  • 61,194
  • 18
  • 189
  • 288
pintor
  • 823
  • 1
  • 2
  • 8
  • 7
    To clarify some points, have you gone back and glanced over the entire review? Have you attempted to check into any of the suggested articles that, at surface-level, don't make sense? More or less, have you done due diligence to assert your suspicion? In my opinion, you should be treating this as if you're about to accuse the reviewer of misconduct. – David S Nov 30 '23 at 00:34
  • 4
    Anecdotally, I've found that many people are using AI as a super spell checker, running whatever they wrote through AI to make the writing better. Assuming good faith, that might not necessarily mean the reviewer is asking AI to review. – Passer By Nov 30 '23 at 04:38
  • 7
    @DavidS Yes. My suspicions are based on the entire review, not some selected sentences. I've checked that all articles exist, then quickly went through 3 out of 6, but I don't see how they can be relevant. If there is a higher wisdom in those links, I don't see it. As for accusations, I don't know what kind of due diligence I'm supposed to do. All I have is a suspicion and no fail-proof method to prove or disprove Al-use. – pintor Nov 30 '23 at 09:15
  • 3
    @PasserBy I don't mind Al being used for styling, grammar checking (I'm surely guilty of that), and even expanding ideas into a text (within reason). But this review doesn't read like Al-edited. My feeling is it was written entirely by Al with minimal human involvement. However, again, I have no proof. – pintor Nov 30 '23 at 09:25
  • Apart from any other consideration, I think it's worth considering the apparent reviewer's motive. Specifically, was he being paid even a nominal fee for doing the work, or in some way accumulating credit for advice given and (gratefully) received? – Mark Morgan Lloyd Dec 01 '23 at 07:16

5 Answers5

119

I think you should contact the editor with basically the same content as your post here: explain why you think the review could be AI generated, including the limitations of each point of evidence. I think most editors see their job as letting reviewers guide their decision rather than reviewing the reviews, so I would start from the assumption that the editor would not have considered checking the reviews for suspicion of gen AI.

Explain why you think this is concerning, which I think at minimum would include the integrity of the peer review process (you and your colleagues are expecting papers to be reviewed by humans, not AI) and the protection of your intellectual property (having an AI review your paper involves inserting your work into a 3rd party tool which may incorporate it into the training data and is a definite breach of the confidentiality that reviewers are sworn to). I would check the journal's website to see if they have an explicit statement about use of AI in peer review; if so, you can refer to that as well. I think you can do this all in one sentence to not appear to lecture the editor.

Finally, make clear that you're not protesting the decision for a revision and plan to work on revising and resubmitting your paper. I'd include a note thanking the editor and the reviewers for their time.

From there, let the editor decide what should be done next; it's their call, your only duty here is to raise their awareness. It's possible they'll want to bring in another reviewer. It's also possible they'll decide it's okay. At that point you'll have to decide whether you're satisfied with their effort and whether that affects your opinion of the journal as an appropriate place to submit your work.

Bryan Krause
  • 114,149
  • 27
  • 331
  • 420
  • 28
    Yes (YESSS!), editors should be made aware of this so that appropriate policies can be developed. It goes beyond the concerns of a particular author or paper and requires deep thought. – Buffy Nov 29 '23 at 16:31
  • 4
    Thank you! I've checked peer-reviewing guidelines, but it says nothing about explicitly prohibiting Al-use for reviewers. At least not at first glance. Though, implicitly, it can be tied to copyrights (as you mentioned). However, I found that the journal has an ethics team. Do you think it's better to ask them instead of the editor? – pintor Nov 29 '23 at 16:59
  • 21
    @pintor Not too surprising, this genAI stuff is still pretty new and publishing moves slowly, and I agree with you that it's likely already covered by standard confidentiality policies even if not explicit. I'd start by working with the editor. You might possibly ask the editor if this is something the ethics team could help out with, but I'm guessing they exist more as a check on the editorial team when editors run amok (or, cynically, they just exist to quash ethical complaints). Going to them directly may be seen as an accusation against the editor, which you don't intend to make. – Bryan Krause Nov 29 '23 at 17:02
  • 3
    While I agree with the first, third, and fourth paragraph, I feel your recommendations in the second one might be overdoing it. It feels like lecturing the editor on something they should know. I would only go to these length if the editor doesn’t see a problem with an AI-generated review. – Wrzlprmft Nov 30 '23 at 07:41
  • 1
    @Wrzlprmft They certainly will have to be mindful about the phrasing, though second paragraph has a very important point. Depending on the AI tool (allegedly) used, users consent to the input becoming a part of the datapool. Therefore, even if the paper has no rules regarding AI use, confidentiality still might have been breached. From my point of view, that is the top possible problem and should be highlighted accordingly. – vspmis Nov 30 '23 at 10:58
  • The IP/confidentiality concerns depend a lot on what AI tool they are allegedly using and how they are using it. Some companies allow at least paying users to opt out of any submitted data being used to train anything, and there are now open source models that can run on a home computer while offering roughly the response quality that ChatGPT offered at launch (for instance Mistral-7B, Starling-7B). I would focus completely on the quality of the reviews being abysmally low, suggesting that the reviewer did no mental work when they reviewed. – Polytropos Nov 30 '23 at 11:41
  • 1
    @Polytropos that's absolutely true, it depends on the AI tool. Though there's a difference between a lazy review and a lazy review that not only appears OK at a glance but also the reviewer possibly disregarded confidentiality. It's technically not against any rules to give a barely useful review, though a valid concern and a reviewer can grow. It's a different level of disregard for scientific integrity to possibly expose someone's IP in the name of laziness while parading as a sincere reviewer and therefore should be handled differently. – vspmis Nov 30 '23 at 12:54
  • 1
    @Wrzlprmft I added a note that I think this can be done in a single sentence, which I do not think will come off as lecturing. I think it's there more to reinforce that OP is concerned about the general problem of AI reviews rather than griping about this specific review. – Bryan Krause Nov 30 '23 at 13:38
  • 1
    @Polytropos Whatever agreements the user has made with the AI tool, they have not made those agreements with the journal or the author. It's not up to the reviewer to decide what is an appropriate use of the author's property, they've been given the paper to review confidentially. They don't get to decide that sharing with a given AI tool is okay because that AI tool promised to be confidential about it, too, any more than you get to decide to share someone's secret with anyone you want as long as they promise they won't tell. – Bryan Krause Nov 30 '23 at 16:25
  • @BryanKrause This line of reasoning seems weird to me. There are a number of pieces of software a reviewer might use that involve license agreements that journals and authors never had any input on, including things as commonplace as PDF readers, printer drivers and virus scanners, but also things like cloud storage and VPN services, that more clearly involve third-parties. In principle, there is even the agreement with the ISP or public wifi used to download the manuscript in the first place. If the risk of confidentiality breaches is not a factor, then where should the line be drawn? – Anyon Nov 30 '23 at 18:00
  • 1
    @Anyon That's a good point, though many journals already avoid things like sending manuscripts via email to keep out some of those third parties. PDF readers and printer drivers don't typically send content outside of someone's local computer/network though I suppose they could be manipulated to do so. I guess I may be especially suspicious of AI companies given the loose opinions about data ownership they have expressed including the idea that training any data they can access through the internet is fair use. – Bryan Krause Nov 30 '23 at 18:15
  • @BryanKrause: OP thinks the review was based on the abstract, which is often shared without confidentiality (e.g. subreviewer asks, maybe even via gmail, which in your security model means trusting Alphabet). Also, under your view, would using local models be fine as nothing leaves that computer? I think even without training opt-out, I would disagree that this is much less of a breach of ethics than copy/pasting GPT-4 output. On the other hand, if GPT-5 produced actually good reviews, I think I would support its use in peer review. It would reduce the problem of poor human reviews. – Polytropos Dec 01 '23 at 00:50
  • 1
    @Polytropos ChatGPT is incapable of understanding so it's incapable of reviewing a paper. Copyediting is not peer review. Word salad is not peer review. – Bryan Krause Dec 01 '23 at 00:55
  • @BryanKrause Capabilities of future models are hard to predict. I would hard disagree that GPT-4 output is, for the most part, word salad, but agree that the capabilities to perform reliable, meaningful peer review of scientific papers are not there as yet. But even now, there is likely a nontrivial time limit (maybe 20 minutes in CompSci?) where a well-prompted GPT-4 will on average produce a more informative review of a paper than an expert, if we assume that reviewer expertise and paper topic are not a total match. The question how much that will grow near future is totally empirical. – Polytropos Dec 01 '23 at 05:16
  • All this hoping the editor is not some AI algorithm itself ;) – EarlGrey Dec 01 '23 at 11:38
  • @Polytropos Anyone who is using an artificial time limit like that to compare to an expert review is measuring the wrong thing. ChatGPT is not capable of peer review, full stop. – Bryan Krause Dec 01 '23 at 12:49
  • @BryanKrause Once the time limit gets into the order of days, it will at least for some disciplines be the right thing to think about. In my field, I don't think reviewers spend more than a working day or two reviewing each paper on average (we publish in conferences, so there is a hard time limit on reviews anyway). – Polytropos Dec 01 '23 at 23:34
8

The Committee on Publication Ethics and related organizations are developing reporting and investigational best practices for this (COPE). It may be considered misconduct according to the policies of the journal, and if not disclosed, it most likely is and should be investigated by the editors. If they do not take your report seriously, I would question the integrity of the journal--see related material on predatory publishing.

In addition to reporting this to your editor for investigation, as @Bryan Krause stated above (+1 to everything he said), I would familiarize yourself with the work of the committee on this topic, as they are monitoring these issues closely and developing best practices and even formal guidelines on handling investigations.

COPE also provide significant resources to editors who may actually be less versed in dealing with these issues themselves, so it couldn't hurt to share that information with your editors if they need a bit of guidance.

COPE. 2023. Where next in Peer Review? Part II. https://publicationethics.org/news/where-next-peer-review-part-1.

Eric
  • 81
  • 4
5

I have been a journal editor for 20 years, and I would not pass on this review to the author. Definitely alert the journal.

Prof S
  • 59
  • 2
2

Edit in response to discussions here.

I think my answer is wrong. @BryanKrause and @Buffy and @leonos are right. The OP should contact the editor immediately.

Leaving this post up so the back and forth is visible.

Interesting puzzle.

Since the (possibly AI prepared) review is generally positive, treat it as if it were legitimate. Since the reviewer asks for no changes you need not argue about whether they are appropriate. If any of the suggested references are in fact relevant, include them.

After the paper is accepted you could contact the editor to voice your suspicions, with whatever supporting evidence you have.

Edit after reading @BryanKrause's answer.

Clearly you will revise your manuscript in response to the good suggestions from the unquestionably legitimate review. If you think the work would benefit from a second good review then you should tell the editor now about your suspicions.

Community
  • 1
Ethan Bolker
  • 35,457
  • 1
  • 82
  • 127
  • 14
    This suggestion feels unethical. Only raise the problem if the review was negative? Either using “AI” to write a review is a problem, or it is not, this should be independent of the outcome of the artificial review. –  Nov 29 '23 at 16:04
  • 3
    @leonos Perhaps. My thought was that the ethics of the situation did not require the OP to risk losing the chance to publish by possible antagonizing the editor. There are plenty of lazy referees out there. I hope the OP will report back here when the dust has settled. – Ethan Bolker Nov 29 '23 at 16:09
  • 10
    in my opinion, this is a clear ethical issue and the authors should send a clear and immediate message that it is not acceptable - particularly because the journal might not care that much. Imagine if journals could just dispense from finding reviewers and artificially generate reviews at no additional time cost…a gold mine for predatory journals. It’s quite concerning that the editor did not even notice or care about that report in the first place. –  Nov 29 '23 at 16:34
  • 2
    @leonos You're right. See my edit. – Ethan Bolker Nov 29 '23 at 17:32
  • 5
    Ethan - I saw it! Thanks for the edit and for being open to changing your mind! –  Nov 29 '23 at 17:38
-8

I suspect one of the journal reviews I got is Al-generated I wonder if I should mention my suspicions to the journal editor.

I don't think that's a relevant criterion. Whether the review is useful is a more important criterion.

You could focus on the lack of constructive feedback or how the comments seemed to closely align with your abstract without offering substantial insights or suggestions for improvement.

Expressing gratitude for the helpful feedback from one reviewer while politely pointing out the absence of actionable feedback from the other might prompt the editor to consider the nature of the review.

Wrzlprmft
  • 61,194
  • 18
  • 189
  • 288
Julien Reszka
  • 133
  • 2
  • 12
    Strongly disagree; it's highly relevant, for the reasons I point out in my answer. – Bryan Krause Nov 29 '23 at 18:49
  • 21
    In addition, the peer-reviewing system relies on the implicit trust that the editor will choose reviewers that are knowledgeable experts of the relevant field. At this stage, the language models are not knowledgeable, experts, or even concerned about truth at all. Using one to write a peer-review completely misses the point of peer-review which is to promote scientific accuracy. –  Nov 29 '23 at 20:13
  • 6
    I don't disagree with the notion that an AI reviewer could feasibly give useful feedback on a paper. But no matter how you spin, it is not peer review - a journal claiming their articles are peer reviewed when they are not is a common fraud tactic of predatory journals. Not good company to be among. – Nuclear Hoagie Nov 30 '23 at 14:26
  • 1
    @leonos I agree, and the "at this stage" is important. – Volker Siegel Dec 02 '23 at 01:42