50

  1. A journal article I submitted to a highly reputable journal has been returned with a number of revisions requested, most of which are useful and helpful and will improve the quality of the article. However, two of the three reviewers have also suggested articles that should be added as references to my paper. These articles are not appropriate to reference in the paper; I have tried at length to find relevance but cannot. I assume that the reviewers are authors of these papers and wish to increase their citations.

  2. The email from the (unnamed) journal editor asking for revisions is pro forma and includes a line asking for any inappropriate requests for citations to be referred to the editor.

  3. Analysis of the authorship of the requested articles suggests that the two reviewers in question are affiliated with the same institution. Two members of the editorial board of the journal are at the same institution, so it is at least possible and perhaps likely that the editor shares an affiliation with the 2 reviewers.

My supervisor (and co-author) suggests I try and find the suggested paper that is least inappropriate and reference it in our article ‒ "throw them a bone."

I feel that we should address the other revision requests comprehensively but decline to reference the articles, giving our reasons.

Ethics vs Pragmatism, yes, but I also want to get the article published and this may not be the hill to die on. Will my approach mean rejection? Do I have any recourse if it does? Should I call this out to the Editor in Chief?

I'm interested in other people's experiences.

kyrill
  • 103
  • 2
doctorer
  • 983
  • 7
  • 15
  • 51
    You could ask the editor to ask the reviewers to clarify the relevance of the suggested references. – Andrés E. Caicedo Mar 28 '19 at 00:43
  • 1
    Closely related (especially F'x's answer): https://academia.stackexchange.com/questions/11923/can-a-reviewer-suggest-references – Allure Mar 28 '19 at 07:48
  • 1
    Even more closely related: https://academia.stackexchange.com/questions/99745/how-to-deal-with-an-unreasonable-reviewer-asking-to-cite-irrelevant-articles? – Allure Mar 28 '19 at 07:49
  • I have voted to close as "strongly depends on individual factors" since you seem to already understand the underlying principles and it really comes down to: what is your personal choice? – user2390246 Mar 28 '19 at 10:38
  • Thanks for those links - interesting, but a different question, in my view, due to the possibility that the editor and 2 reviewers are from the institution. – doctorer Mar 28 '19 at 11:12

4 Answers4

53

If you think you see this often, imagine how much more often journal editors see it.

So sticking to ethics is fine. Journal editors see this often enough to know when to reject a review because of it. You are not generally under threat of rejection if you decline to cite a reference. The worst that can happen is that the reviewer rejects your article, but since they've already recommended revision the first time, the editor is more equipped to discern if the rejection is unfair. Remember that if the reviewer says "reject because they didn't cite XYZ", the editor (who is able to see the reviewer's identity) is very much able to see if XYZ is also written by the reviewer. In your case you even have an editor who said to refer any inappropriate citation requests to them.

A word of caution: there's no guarantee that the requested citations are articles by the reviewers. There's a lot of diversity in what reviews look like, and it's possible the reviewer did not write those articles. Don't leap to conclusions. Stick to the facts ("we do not think these articles are relevant") and don't allege collusion (such as how the reviewers & editors are from the same institution - you simply don't know).

Allure
  • 127,528
  • 50
  • 325
  • 493
  • 38
    As a matter of fact, it's generally not helping your professionally or mentally to try and figure out who reviewers are. Nothing good can come of having this knowledge. So try and avoid the thought of wanting to figure out. – Wolfgang Bangerth Mar 28 '19 at 02:27
  • Agreed that I "simply don't know". But when the facts are unclear, as in this case, is it not prudent to consider all possibilities before decing on a course of action? Collusion is at least possible and may impact on how the editor would deal with my rejection of the suggestions. Also, as stated in the question, the request to refer the citation is from a pro-forma Journal email, not directly from the editor in question. – doctorer Mar 28 '19 at 02:29
  • @doctorer Perhaps, but I think you're reading too much into the situation, and thinking it's much more likely than it actually is. For comparison: if you write an email to your supervisor but he doesn't respond after a day, it is technically possible that he's been kidnapped and you should alert the police at once - but if you actually do that, you're likely overreacting. The same applies here; you are inferring that the reviewers & editors are from the same institution based on extremely weak evidence. – Allure Mar 28 '19 at 02:58
  • 2
    I strongly suspect that citation cartels are significantly more frequent than academic kidnappings, and would be interested to see evidence to the contrary. – doctorer Mar 28 '19 at 03:14
  • 1
    @Allure - I also realise I misconstrued your last paragraph, thinking you were saying I shouldn't suggest on this forum that there may be collusion. But clearly you meant that I shouldn't make an accusation to the Editor, which I 100% agree with. – doctorer Mar 28 '19 at 05:50
  • Does it matter if the reviewers and editors are from the same institution? If the citations being forced and the editor are from the same institution, there's potential for conflict of interest -- and the authors know the identity and affiliation of the editor. – Ben Voigt Mar 28 '19 at 05:50
  • @doctorer no, I'm saying that you should not speculate about the identity of the reviewers at all. You simply have no way of knowing how likely your guess is. In fact based on my experience as an editor, I would hazard that there's a >50% chance that your guess is incorrect (of course this number might also be incorrect, since it can vary by field). Besides, what is the point of speculating? As Wolfgang Bangerth wrote, nothing good can come out of it. Just don't do it. Googling for the affiliations of your hypothetical reviewers is already problematic; you now know things that don't help you. – Allure Mar 28 '19 at 07:08
  • @BenVoigt it's certainly conceivable but is it likely? There'd need to be a lot of coincidences: the reviewer needs to recommend his own irrelevant papers (consider that the reviewer knows the editor personally: would you do that to one of your friends?); the editor needs to not disregard the review; the editor needs to think that it's important that a colleague gets more citations (which doesn't provide the editor any direct benefit); the editor-in-chief needs to not intervene ... Absent evidence to the contrary, one should assume that the editor is a person of integrity. – Allure Mar 28 '19 at 07:14
  • @allure I'm not disputing those odds. I'm not particularly interested in the identity of the reviewer and I don't doubt that most reviewers act with integrity. However, I think it is a strange coincidence that 2 of the reviewers recommended similar articles by authors from the same institution, none of which had any discernible relevance to the article under review. It's a strange coincidence that throws up possibilities, and I would be foolish to ignore those possibilities. – doctorer Mar 28 '19 at 11:06
  • @doctorer I think you're foolish to pay attention to those possibilities, which as you say, are just possibilities. That's all. – Allure Mar 28 '19 at 11:08
  • @allure Strangely, I've always considered an interest in possibilities to be an asset in academia. ;-) – doctorer Mar 28 '19 at 11:14
  • @doctorer you might want to look into the possibility that, right now, your house is burning down then. Yes, the example is silly, but it's just to illustrate that an interest in possibilities isn't necessarily an asset. – Allure Mar 28 '19 at 11:26
  • Yup @allure, silly. But if my best advice was "I'd hazard a guess that there's a > 50% chance your house is not burning down", I'd probably want to investigate the possibility. – doctorer Mar 28 '19 at 11:34
  • @doctorer well, what you're doing isn't investigating the possibility. You're speculating. If investigating is what you want to do, you're not going to find answers here, because the only people who know the identity of the reviewers are the staff of the journal. You'll have to ask them. – Allure Mar 28 '19 at 11:39
  • @allure I'm simply trying to determine the best course of action, given a range of possibilities, and asking others how they have acted in similar circumstances and what the consequences were. I don't quite understand why you object so strongly to to this. I should ignore all possible outcomes unless they are certain? – doctorer Mar 28 '19 at 11:59
  • 2
    @doctorer because 1) you're likely to be wrong and 2) you have nothing to gain. Again, as Wolfgang Bangerth said in the first comment: "As a matter of fact, it's generally not helping your professionally or mentally to try and figure out who reviewers are. Nothing good can come of having this knowledge. So try and avoid the thought of wanting to figure out." – Allure Mar 28 '19 at 12:12
  • 2
    This paper claims a roughly 10% accuracy rate in authors properly identifying their reviewers (although my library doesn't have access so I can't read the whole thing) https://www.atsjournals.org/doi/abs/10.1164/rccm.201711-2257LE?journalCode=ajrccm – llama Mar 29 '19 at 20:40
  • Agreed - As I've said, I wouldn't try to identify my reviewers. – doctorer Mar 30 '19 at 00:05
46

(1) Your co-authors are correct in that you should strive to minimise friction in the review process. (2) Commenters here are correct that you should stick to the facts of the situation and not risk attempting to attribute speculative motivations, or identities, to the reviewers. (3) You are correct that you shouldn't reference papers that have no apparent relevance to your work.

So consequences that arise from the points above are:

(1) Don't even consider referring the issue to the editor. Firstly, you can deal with this solely within the bounds of the normal response to the reviews. Secondly, is it remotely possible that there is actually some relevance to the papers that the reviewers see but you don't? Imagine the small possibility that they are right and you are wrong, and the damage to your reputation and theirs by escalating this.

(2) Make no statements and take no actions other than to respond to the issue that you believe that the papers aren't relevant. Don't get clouded by things that you can't absolutely know, like intentions and identities.

(3) In your response, simply state that you haven't added the references, and DO NOT add any other claims about the reviewers' motivations. You just want this to be as friction-free as possible. To avoid conflict in a subsequent round of review, you can gently put the ball back in the reviewers' court. e.g. state something like this:

We thank the reviewer for the suggested reference. Unfortunately, on close reading we could not determine exactly which part of our argument it supported. Therefore, we have not added it to the manuscript at this stage, but would welcome any specific guidance as to how it could be incorporated.

This avoids you being confrontational, yet without "giving them a bone". If they want to push things, then they now have to jump through a hoop to make the link for you. I suspect the issue will just get dropped. But it also leaves the door open for you in the small chance that the reviewers have actually made an insight or connection that you have failed to see.

Finally, as there are major revisions requested, deal with those fully and constructively and in detail. The couple of sentences devoted to this issue will not seem important to the editor in that context.

Michael MacAskill
  • 637
  • 6
  • 12
7

Unfortunately, I also made the experience that reviewers often try to recommend their articles for citation. Often this allows me actually to identify who the reviewers are based on the suggestions for reviewers I made when submitting the manuscript. Therefore, I think this is no good practice at all, as it undermines the actual review process (but also promotes citation cartels).

The question is then rather to me, would a non-citation of their articles be a reason for a major revision. To my experience, suggested editing of the references is normally not more than a minor revision, so the reviewers are not asked anymore for their agreement and it is up to the editor to publish your article based on the minor revisions you made. If you explain to him/her the suggested references are not related to your article, after checking it thoroughly and you don't know where to cite and how to explain them in the manuscript, it is up to him/her to leave them out.

Ethics vs. pragmatism, well, throw a coin or think about how much harm citing their articles implicates (if you don't have to highlight them with another sentence in the manuscript and can add them to a group citation [1,...,4]) for your article and the scientific community. If they are not linked by any interdiscplinary, theoretical or experimental distant context, then the reviewers will also have a difficult argument to explain, why they should be cited. But don't start to poker with all of them in the review process.

Stewart Macdonald
  • 105
  • 4
user48953094
  • 4,346
  • 14
  • 25
  • Thank you. Although the addition of references is only a minor revision, we have been asked to make major revisions, so it is at least possible that the revised paper will be returned to teh reviewers. Moreover, my concern is that the editor is also part of the "citation cartel".... – doctorer Mar 28 '19 at 02:06
3

There is no true motivations to insert inappropriate references in the paper, moreover being the reference system the only mechanism able to make the better works emerge.

Do not do that.

If you have a lot of time before the deadline, write to the reviewers cited asking some help in order to have evidence of the relevance, because you are not able to find it. You should write this in some plain manner, nothing alarmed or worried or whatever. And watch what answer returns to you.

If the deadline is too close, simply omit the references not truly relevant, writing to the reviewer something about that, argumenting that properly and having the organizer in CC.

Do not abandon ethics, because without it the Academia is something unuseful and very similar to mafia or similar organizations.

Rick Park
  • 302
  • 2
  • 6