59

I received a paper to review that I can only describe as very bad. Not only are there structural issues that anyone that has previously published a paper in a respectable journal should know to avoid (like providing sources for images that are clearly not the authors' own) and instances of (self) plagiarism. Most of the papers of the authors themselves that are cited in the paper stem from one very questionable journal.

Much worse, though, is the fact that the paper is (IMHO) religous agenda badly disguised as a scientific paper: the authors claim miracles to be the cause of effects they are describing, they selectively dismiss results that dispute their theories and claim in the last paragraph that they have thus proven that Jesus has risen from the grave.

I am honestly really shocked that this was not desk rejected but made it to peer review, but here we are. I now wonder if it is OK to write a review along the lines of "Your paper is bad and you should feel bad" because that really is my honest opinion? Or should I rather skip the review report altogether and just write to the editor that I think this should not be published?

Anyon
  • 26,132
  • 8
  • 87
  • 116
Sursula
  • 20,540
  • 8
  • 62
  • 121
  • 66
    Did you receive it on April 1st? – TonyK Apr 17 '23 at 23:43
  • 13
    @TonyK I wish, but no. – Sursula Apr 18 '23 at 02:44
  • Comments have been moved to chat; please do not continue the discussion here. Before posting a comment below this one, please review the purposes of comments. Comments that do not request clarification or suggest improvements usually belong as an answer, on [meta], or in [chat]. Comments continuing discussion may be removed. – cag51 Apr 18 '23 at 19:25
  • "Do I have to be nice?" is a bit imprecise; What kind of answer would accurately answer your question? You don't have to be even minimally polite to anyone in life, but there might be consequences of varying degree. Are you asking about those? – Philip Apr 18 '23 at 20:55
  • Strange question. If you do not want to review a paper, don't. Being nasty won't correct that, in fact, if you are nasty, the Editor may not consider your review and may not forward it to the authors. If you cannot be objective and neutral in tone, then you cannot communicate effectively, and you are then upsetting yourself and possibly others to no effect other than harming your own reputation, which should not be chosen as an option. – Carl Apr 27 '23 at 19:03

13 Answers13

108

Why waste your time writing a lengthy review? Just note to the editor that the paper should have been desk rejected. Identify one or two fundamental issues --these could be high level.

There are many authors who have no idea what research is nor the resources to carry out high quality research. There is no point 'scolding' them.

Prof. Santa Claus
  • 1
  • 4
  • 22
  • 29
  • 134
    Review the paper, but not the person who wrote it. – Buffy Apr 17 '23 at 10:53
  • 5
    @VitaminE From what I can gather from their citations, at least one of the authors is a professor at a EU university - they should have both ressources and an idea how science works. – Sursula Apr 17 '23 at 11:16
  • Some people, even professors, are so wrapped up in their religious identity that their "belief" colors everything they do. They can't even leave it alone and be objective otherwise, but have a need to proselytize at every opportunity. I've known such people, though not many. They are unlikely to ever change and won't be swayed by any argument, even when their behavior is negatively affecting their career. – Buffy Apr 17 '23 at 13:05
  • 1
    @Sursula In any case, really point out to the editor that the paper should have been desk rejected. Too many times these days some editors fail in their responsibilities. – Massimo Ortolano Apr 17 '23 at 18:43
  • 14
    @Sursula Unless it's spelled out, you don't know in what manner the professor at an EU university contributed to the paper. Perhaps they provided general guidance and the other authors included them as a courtesy. Perhaps the professor hasn't even read the paper. – shoover Apr 17 '23 at 19:06
  • 4
    @shoover is that better or worse, though? – henning Apr 17 '23 at 19:28
  • 2
    @shoover Or perhaps their inclusion is simply fraudulent. – user71659 Apr 17 '23 at 19:46
  • 10
    @shoover I suppose you could use something like this strategy: if they stand by the paper, they'll be appropriately insulted, if their name was added surreptitiously they'll be appropriately informed (to be clear, I'm not actually suggesting this, and a reviewer should not communicate to the authors except through an editor). – Bryan Krause Apr 17 '23 at 19:55
  • 1
    @shoover If they haven't read the paper, they should not be listed as author (and should not sign off on being listed as such). Authorship isn't just about credit, it implies taking responsibility for at least some of the content. – GB supports the mod strike Apr 18 '23 at 00:06
  • 3
    +1, an actual review I've seen ended with "I have many minor comments. But, I think the points mentioned above are enough to reject the paper" -- which could also be applicable here. I would also definitely write in the confidential to editor section that the paper should have been desk rejected, since it prods them to pay more attention. – Allure Apr 18 '23 at 00:58
  • @Sursula quite frequently the professor is only mentioned as a courtesy because he's the department head or otherwise the boss of the actual author. – jwenting Apr 21 '23 at 08:19
52

No, you don't have to be nice. You have to be professional, but this does not always equate with being nice. Generally, there are three kinds of papers that I get to review.

  1. Very good papers. I enjoy them. Very rare. Minor revision, and review is short.
  2. Mediocre to good papers. Lots of work, perhaps good idea, but then something is lacking. Major revision or reject, review has to be detailed to sort good and bad aspects out.
  3. Bad paper. Don't enjoy them, but not a lot of work. Reject, and just a few arguments why it was rejected. Reviewer does not need to list ALL the reasons why. It is up to author to reach certain scientific paper minimum, and it is not reviewer's job to teach authors how to write papers properly.

Of course, everyone expects that papers (3) would be desk rejected, but this is not always possible (someone mentioned religious aspect of the paper, which may frighten the editor into getting reviewer's opinion). Of course, with (3) also goes sometimes the explanation to the editor why the paper is bad, and since it is not visible by authors you can be as nasty as you want there. Just don't send this to the authors my mistake.

xmp125a
  • 3,915
  • 1
  • 16
  • 18
  • 22
    Very good papers: I do write lengthy reviews for them sometimes. It helps counteract ill-informed, and -argued short rejection reviews where clearly one has not read or understood the papers. – Captain Emacs Apr 18 '23 at 15:08
  • 3
    The authors may be friends/colleagues/head of dept of the editor. And they are for sure human persons (although they may try very hard to hide this fact :) ). No need to be nasty (cfr. https://www.ldoceonline.com/dictionary/nasty#:~:text=From%20Longman%20Dictionary%20of%20Contemporary,nasty%20streak%20in%20her%20character ). – EarlGrey Apr 18 '23 at 15:21
  • Just don't send this to the authors by mistake ... he he he. Can't help but laugh. But hey, very important. – semmyk-research Apr 18 '23 at 20:36
  • 2
    I don't see anything in here that I would consider "not-nice" -- i.e., anything that's mutually exclusive with being polite. – Philip Apr 18 '23 at 20:50
  • @Philip, think harsh love, telling someone their paper is bad because of a list of reasons could be viewed as not-nice depending on the person, but it can be done in a polite way. – N A McMahon Apr 18 '23 at 22:33
  • 2
    @NAMcMahon the OP says they want to tell the authors "your paper is bad, and you should feel bad". I don't interpret this answer as giving moral permission for this. – Ian Sudbery Apr 19 '23 at 07:39
  • ''Of course, with (3) also goes sometimes the explanation to the editor why the paper is bad, and since it is not visible by authors you can be as nasty as you want there. Just don't send this to the authors my mistake''. You do realise authors can obtain these comments if they wish? – Tom Apr 19 '23 at 07:55
  • I think there is still room for being nice. – JosephDoggie Apr 19 '23 at 22:08
  • 1
    @CaptainEmacs And what a (rare) pleasure that is. :D – Karl Apr 20 '23 at 18:07
  • @Tom That may differ, but at least where I've reviewed til now, there was space to enter comments specifically only for the editors eyes. Can you elaborate how the author should get access to those? – Karl Apr 20 '23 at 18:14
  • @Karl Data Protection Act or something like that? If they ask to see the comments which were only for the editors to try and find out if anything strange was said which led to the rejection, I think they can get access to them legally. – Tom Apr 20 '23 at 19:33
  • @Tom Very unlikely. By submitting your manuscript, you agree to the procedure set up by the journal. How would you convince a judge to seize that private communication? You'd have to construct a criminal case. – Karl Apr 20 '23 at 19:57
  • @Tom I never heard about authors being able to view confidential communication directly, let alone getting reviewers' identity. As an editor, I would fight this tooth and nail to prevent this and would consider resigning if somebody would overrule me. Yes, you can be nasty in the comments to the editor, 1) it defuses the anger so the reply to authors can be nicer 2) to educate the editor about the concept of "desk reject" (I am a bit sarcastic here). – xmp125a Apr 21 '23 at 13:09
  • @CaptainEmacs I agree about trying to help the authors this way, but my time is limited... – xmp125a Apr 21 '23 at 13:11
  • 1
    ''it defuses the anger so the reply to authors can be nicer'' That's a really sad reason for being nasty in editor comments, it's like saying I'll kick the dog and take out my anger on it so that I can be nicer to people. If a person gets this angry about a bad paper, they shouldn't be a referee in the first place. – Tom Apr 21 '23 at 14:06
  • @Tom I am editor too. Editor who does not use desk reject instrument should be properly schooled. It really damages the scientific community because reviewers are incredibly scarce commodity right now. Editors should not waste reviewers time on obviously bad papers. – xmp125a Apr 22 '23 at 15:32
  • @xmp125a Sure. I do not always do that, but sometimes my enthusiasm gets the better of me. And then it turns out the lengthy review was indeed needed, so that's nice. – Captain Emacs Apr 23 '23 at 00:27
  • I once was sent by mistake the comments to the editors of one of the reviewers. It was along the lines "this is absolute crap, I am so annoyed about how crappy this work is that I can't even start mentioning why", and that was the whole review (no comments to the authors at all). The other two reviewers thought more like mediocre-to-good paper (one major and one minor revision). The one calling our paper crap was being criticized i our paper and we wonder if the editor did slip on purpouse... – Lucas Apr 27 '23 at 08:19
28

You do not have to be nice, nor do you have to be nasty.

Just be concise, and everyone's life will be easier, even the authors' (it is up to them to blame the short review on their approach or on being "offended" by reviewers' suggestions to the editor).

terdon
  • 2,451
  • 19
  • 20
EarlGrey
  • 17,959
  • 2
  • 28
  • 79
19

If you do not think a paper is publishable, then reject it (it is part of the assignment to filter out the bad ones). If it is really bad, you should be able to form a short, objective critique of the manuscript in your answer, and you do not need to be rude.

If the manuscript cannot support its results with arguments, the manuscript is so poorly written/structured that is is a problem to follow its argument, if there are several big logical mistakes etc these are well defined, objective reasons while you reject a paper.

Greg
  • 2,999
  • 16
  • 21
11

If the paper is truly as you describe it, I would just write the editor who asked me to review it and tell them "This should be desk rejected." I would still write and formally submit a review, consisting of only one page with major comments. Then I'd send my review off in a day. But no, no need to be unkind.

Jared Greathouse
  • 1,967
  • 1
  • 12
6

If it's really very bad you don't need to write a whole dissertation about how horrible it is. The editor will probably see it. Just clearly, succinctly and objectively state the biggest problems and it will get rejected. Don't waste your life on bad papers, instead worry about good papers.

gomennathan
  • 756
  • 1
  • 8
6

The reason there's an option for reject is to allow reviewers to filter out bad papers.

As reviewers, we are duty bound to engage the manuscript beyond

  • 'who' the author(s) is/are
  • what our 'personal' feeling is
  • what our individual subjective view is

Rather, we ought to engage

  • the academic value of the manuscript
  • the contribution to knowledge (this can be a lengthy debate in itself)
  • the academic content
  • the validity, reliability, rigour
  • the manuscript vis-a-vis its methodological approach (philosophy, methods) and not our fixated methodology.
    Say a manuscript is philosophically underpinned by critical realism and follow pluralism (multimethods) instead of quantitative or mixed-methods.
    Are we to castigate the authors or shoot down the paper, bearing in mind the methodological choice of the manuscript and rationale.
  • the manuscript objectively (be objective and put our research 'bias' aside for a moment) That's a matter for another day.

Of course, there're other metrics of review. I'm just listing some. By and large, when a manuscript has scaled the editorial gatekeeping, we ought to focus on the review and contribute our quota to knowledge. We need not become a second layer editor. Let's review

So for a bad paper, how bad is bad. Philosophically speaking, isn't bad, in this context of post editor, subjective?

I'll just list the defects from a academic review standpoint. I need not list them all: just some highlights that'll

  • show that the manuscript is off-track (bad as in bad)
  • give indication of defects that might prompt/assist in making a reasonable author in (re)channelling their research endeavour/energy.
    Be objective and succinct.

PS: Some bad papers have gone up to become frontiers

[Note that in saying we must fix our eyes on the manuscript and get on with the review, I am not saying we must not write that note to the editor. We can and we should. That's why, as reviewers, we're empowered with that "confidential note to the editor". Nonetheless, let's get on with the review and contribute to our discipline]

semmyk-research
  • 3,560
  • 1
  • 3
  • 23
  • even the academic value shouldn't really be a factor. If the paper doesn't really add much but is well written there's no reason to call it a bad paper, it may merely be addressing a very narrow niche or be intended as an addition to a prior, larger work or a prelude to a following larger work. – jwenting Apr 21 '23 at 08:21
5

Given the topic of the paper, it's entirely possible that the editor did not feel comfortable with a desk reject because of (or in order to avoid the appearance of) religious bias.

As such, I would say that your best service to the editor, the journal, the scientific community at large, and maybe even the authors themselves, is to compose a review along the lines of what you posted in the question.

As long as you keep your comments to the content and references of the paper and refrain from ad hominem (and from what I read here, this is well within your capabilities), I would say that that's your duty.

João Mendes
  • 792
  • 5
  • 14
5

You should be professional and respectful. Someone spent time writing the article and thinking about it (although it may be very bad) and there is never any justification for being nasty, apathetic, moody or cantankerous, regardless of age or power level in the academic system.

Just give a short, professional explanation of what you think the main problems are and why it should be rejected, then send that to the editor and move on.

Tom
  • 3,733
  • 1
  • 11
  • 29
3

Reading between the lines, it sounds like your motive for being anything other than nice might be somehow to "talk sense" to the authors or at least to take an abstract stand by decrying behavior you consider unacademic.

If that really is your motive, you'll get closer to your goal by being polite but firm. If you veer off into ad hominems or screeds about religion in public life, you could be reinforcing their preconception that "Christians can't get a fair hearing in academia," even as unwarranted as that view is here. Imagine them telling everyone at their church about that one time they tried submitting to a journal in your field and got berated. Imagine some young congregant, a future superstar in your discipline, listening to them and becoming dissuaded from academic research. Maybe that doesn't matter to you, but maybe it does.

TBH your question doesn't make sense to me. If you're asking for someone's permission to not be nice, it implies you want to not be nice. ("You should feel bad"? I know that's not literal, but it still says a lot.) If you can explain why you would prefer not to be nice, you can probably answer yourself better than we can.

Philip
  • 996
  • 4
  • 15
1

Academia is, thankfully, not the Stack Exchange network: you don't have to pretend to be nice when it's unwarranted. Papers that completely fail the most basic tests of academic rigour are not just a waste of your valuable time, they show a fundamental incompetence and disrespect for intellectual honesty, and ultimately demonstrate that the author(s) of said paper need a wake-up call.

A simple rejection is unlikely to trigger that wake-up call, but a harshly-worded one might.

Ian Kemp
  • 758
  • 4
  • 10
  • Intuiting from the OP, I'd put it at about 90% probability that the authors know exactly what they're doing and are trying to prove a point. Perhaps I'm wrong and it's actually 10%. Regardless, if that is their motive, your strategy is to give them exactly what they are seeking. (See also findings that presenting facts only further entrenches political partisans' errors.) OTOH if they're sincere and unaware, it's hard to see how a "harshly-worded" response accomplishes anything that "polite-but-firm" wouldn't. – Philip Apr 21 '23 at 19:57
  • Proving what point, exactly? That claptrap isn't accepted in academia? – Ian Kemp Apr 21 '23 at 19:59
0

I now wonder if it is OK to write a review along the lines of "Your paper is bad and you should feel bad" because that really is my honest opinion?

You certainly shouldn't write that, even if it is your honest opinion. As is pointed out in other answers, many people have no real idea what it takes to create a proper scholarly paper, and if they lack the skills to do this then it ought not be a cause for them to feel bad about themselves. As you point out, this should have been rejected at desk review. If you wish to respond with a substantive review for the authors (as opposed to just going back to the editor and telling them to reject), keep it professional and comment in a sober and constructive manner about the deficiencies of the paper that prevent it from being acceptable. Do not make extraneous comments about the author or their (presumed) motivation, or assert that they should feel bad about the quality of the work.

Ben
  • 68,453
  • 9
  • 142
  • 263
-5

and claim in the last paragraph that they have thus proven that Jesus has risen from the grave.

The optimistic view is that the author is (apparently) trying to be academically rigorous. If you reject it out of hand, it will reinforce the writer's belief that the peer review process is nothing but a gatekeeper for approved (secular, in this case) ideas.

You should explicitly state why everything up to the conclusion is bad. The only thing I'd mention about the conclusion is that no conclusion can be drawn from such a flawed set of "facts".

EDITS:

  • A commenter thinks that I advocate debating with the author. But since peer reviewers don't meet the author, and the author doesn't know who reviewed it, there can't be a debate.
  • Another commenter does think that the only gates which exist are those of methodologically and theoretically sound science. That is manifestly not true, even though academics would like to believe it so.
RonJohn
  • 194
  • 5
  • 7
    It's not necessarily worth your time to argue with those that cannot be convinced otherwise, particularly those who value faith in the face of criticism and label such efforts as the work of evil. Sure, if you want to/enjoy that debate, go ahead, but I wouldn't say it's required academic service. – Bryan Krause Apr 19 '23 at 20:49
  • @BryanKrause arguments are two-sides. Statements are one-sides. – RonJohn Apr 20 '23 at 03:36
  • 2
    It doesn't matter whether or not the reviewer reinforces some flawed beliefs of the writer. What matters is that the reviewer holds up scientific standards and, yes, acts as "gatekeeper for approved" (i.e. methodologically and theoretically sound) science. – henning Apr 20 '23 at 08:09
  • @henning you'd like to believe that methodologically and theoretically sound science are the only gates which exist. They are not. – RonJohn Apr 20 '23 at 12:55
  • @RonJohn That is not a correct statement about the meaning of the word argue/argument. I refer to the time spent writing what you call a "statement" and I argue it is likely not worth the reviewer's time. – Bryan Krause Apr 20 '23 at 13:03
  • @BryanKrause you can take the time to advance the cause of truth and rationalism, or you can sit in an ivory tower and act smug towards religious weirdos. One is the right thing to do, and the other is not. – RonJohn Apr 20 '23 at 13:07
  • Are you sure you mean "rationalism"? In my world, rationalism is certainly not the primary guiding principle and I don't know why I'd be pressed to advocate it. I do not think it is smug to recognize when someone is arguing from an incompatible belief system and that no argument is possible to sway them. – Bryan Krause Apr 20 '23 at 13:21