15

While I clearly encounter the fact that the preferred type/types of peer-review varies/vary across research disciplines, like pointed out in this question, I wonder which disciplines prefer which type/types and maybe even the underlying reasons.

Is there any kind of a peer-review map showing which disciplines prefer which type/types of peer-review and maybe even why?

FuzzyLeapfrog
  • 4,850
  • 1
  • 20
  • 44
  • 5
    I'm adding a CW answer so people can add their disciplines in the appropriate entry, and we can create such a map here. – Kimball Feb 21 '19 at 14:02

2 Answers2

11

Community answer:

Single blind

In single blind reviews, the authors are not given the identity of the reviewers, but the reviewers are given the identity of the authors. The editors handling the submission know the identities of everyone involved in the process, and the authors may or may not be given the identity of the editor handling their submission.

  • Pure Math
  • Physics (single blind is natural given the lively preprint culture)
  • Some parts of Computer Science:
    • Algorithms
    • Logic
    • Verification
  • Atmospheric Science / Meteorology (typically Open Single Blind, i.e. reviewer reports and author responses are openly available)
  • Library and information science
  • Life/biomedical Sciences (this is currently the most common form, although some parts of the field are slowly moving to open non-blind review, where everyone knows everyone's identity, reviews are signed and made public).

(if your discipline primarily uses single blind reviews, add your discipline here)

Double blind

In double blind reviews the authors are not given the identity of the reviewers, and the reviewers are not given the identities of the authors. The editors handling the submission know the identities of everyone involved in the process, and the authors may or may not be given the identity of the editor handling their submission.

  • Some parts of Computer Science:
    • Cryptography
    • Natural Language Processing
  • Economics (although preprints common in practice)
  • Public Policy/Administration
  • Psychology
  • Operations Research
  • Forecasting

(if your discipline primarily uses double blind reviews, add your discipline here)

Triple blind

This is a system where the authors are not given the identity of reviewers, the reviewers are not given the identity of the authors, and the editors do not know the identity of the author and only find out if/when a paper is accepted (partial after-decision unblinding).

(if your discipline primarily uses triple blind reviews, add your discipline here)

Hybrid

Some disciplines use a hybrid system that is not easily classified as one of the above. If your discipline fits this description, note it below with a short explanation.

  • Human Computer Interaction (HCI) - examples: CHI and UIST
    • 2-3 "outside" reviewers review a submission double-blind. 1-2 reviews are single-blind reviews or meta-reviews by the equivalent of editors (members of a program committee), which both summarize the previously obtained external reviews and include the reviewers own judgement and discretion in assigning scores of their own. A light anonymization process is used, so blindness at any stage varies by submission. Final decisions are made collectively by a larger program committee, based heavily on the judgements of the editors as informed by external reviews, often using a score-cutoff (submissions receiving less than a certain score will not be discussed by the committee and are rejected automatically).

(if your discipline primarily uses such a hybrid system, add your discipline here)

Note that in all fields, how strict the blindness of submissions must be varies. Sometimes the submission must be scrubbed of any hint of who the authors might be, up to and including censoring citations that might suggest identity, rewording phrasing to speak of work in the third person, etc. Some venues within fields may even check to ensure that a search for the title of the submission does not return identifying results from previous talks or pre-print copies. Others take a light approach, and allow authors a great deal of discretion in choosing how heavily to censor their submission. Blindness does not mean that one party absolutely cannot figure out or determine who the authors might be, only that the identity of a party is not made intentionally explicit.

Shepherding

The Patterns Community in CS and other fields (including Poetry, actually) assign a shepherd to each paper. The shepherd works with the authors through several versions of the paper, seeking improvements. The shepherd reads a version and make suggestions. The authors produce a new version in light of the advice. Usually after about three rounds, the shepherd gives a recommendation to the conference committee.

If the paper is accepted by its shepherd and the committee then the resulting version of the paper will go to the conference, and be workshopped by peers of the authors in a session in which the authors take notes but, in the classic version, don't participate. A workshop isn't a presentation by the authors. The authors then write a final version of the paper that will appear in the proceedings.

Rejected papers might need to go through the process again.

There has been some evolution of the concept, but it is still used. If you read a paper in the proceedings of a patterns conference or in many books by community members it has gone through this process.

AFAIK, this process was first developed in the poets community where I assume it is still used.

Ian Sudbery
  • 38,074
  • 2
  • 86
  • 132
Kimball
  • 32,248
  • 7
  • 64
  • 169
  • Isn't submitting a preprint pretty much "zero blind". in condensed matter physics, which is by far most of the literature in this field, every serious and reasonal journal has double blind, some conference proceedings may prefer single blind, but this is rather the exclusion of the rule. The publishing culture in particle/astro-physics is very different to CDM physics, but there is no point in publishing their results in journals behind a paywall, CDM results are very industry-related, which explains this situation a bit – user48953094 Feb 21 '19 at 14:17
  • The availability of a pre-print is more like a single-blind peer-review, isn't it? The reviewers of the journal, where the pre-print is submitted to, are till anonymous. – FuzzyLeapfrog Feb 21 '19 at 14:19
  • Is there anyone reviewing on, e.g. arxiv.org, which is dominantley used in physics? That's not what I heared and physicists mostly upload there if they find no good double-blind journal or the review takes over a year in nature/science... – user48953094 Feb 21 '19 at 14:31
  • Most use cases I know are publishing the pre-print on arXiv upon or slightly before submitting the manuscript to a journal, where the journal peer-review process happens. Comments and therefore public peer-review on arXiv is independent of the journal peer-review. – FuzzyLeapfrog Feb 21 '19 at 14:41
  • 3
    @MichaelSchmidt There's a very perfunctory check on arXiv, but no review. People generally post it there, then later submit to a journal, say one of the Physical Review ones, which tend to use single blind review. As in, identity of reviewer is not known to authors. Things get less clearcut when you approach the boundaries to e.g. engineering / materials science / chemistry, but in my experience single blind certainly is the standard. And I'm in condensed matter physics (CMP). – Anyon Feb 21 '19 at 14:42
  • I've published and reviewed with AIP, APS, Elesevier and have always experienced double-blind, PRL you get sometimes even triple. I think this and the answer to this question depends stronly, if the results can be proven easily (which is in CDM often the case) and Industry has to pick up the results if usable, while in psychology, medicine reproduction rate of published results is lower than 30%. And therefore, the top journals for letter and articles (no Review articels) often have at least double-blind, because the industry will test good results, imho. – user48953094 Feb 21 '19 at 14:55
  • 1
    @MichaelSchmidt What? How would triple blind even make sense? Are you maybe talking about the number of review rounds rather than "degree of blindness"? I currently have a paper in front of me that I'm reviewing for PRL, and it clearly includes the author names => it is a single blind review process. – Anyon Feb 21 '19 at 15:01
  • then you are correct and can delete my misconception/comments, though for me the number of reviewers is the most important measure, if the authors name are given or not I actually don't care ;-) – user48953094 Feb 21 '19 at 15:10
  • 5
    I believe Triple blind would be where the editors also do not know author identity until after completing their work. – Dawn Feb 21 '19 at 15:21
  • I have a little concern with the categories, in that as noted by @Anyon single-blind can mean "the reviewers don't know who the author is", or it can mean "the author doesn't know who the reviewers are". Any thoughts on if this should be clarified or stated explicitly in the answer? – BrianH Feb 21 '19 at 16:27
  • 1
    @BrianH If there is a field where "reviewers don't know who the author is", but the authors know who the reviewers are, then I think the two categories should be distinguished. (I don't know of one myself.) If not, as highlighted by the discussion in the comments, it could still be useful to define the terms in the answer or question. – Anyon Feb 21 '19 at 16:36
  • @Anyon That's a good point, I can't think of one - now that you mention it, I suppose I was actually thinking of single vs double-blind when I said it. So that's probably a moot point, but defining the terms explicitly would certainly help reduce future confusion. – BrianH Feb 21 '19 at 16:46
  • 1
    @MichaelSchmidt I've seen journals in the biological sciences that do double-blind peer review, in that the work is anonymized before sent to reviewers, but allow preprints. Obviously the reviewers could search for the preprint or may have already come across it at the time of review, but the former would be a bit of an ethical breach and as far as the latter, it's just as likely they'd have seen an earlier stage of the work at a poster at an earlier conference. The journals know that double blind is sort of just an aspiration. – Bryan Krause Feb 21 '19 at 16:47
  • 4
    I've added term definitions, as well as a note about the strictness of blindness as this varies quite a bit between fields and is not covered by the blindness categories alone. Please do edit if anything added is not more or less universally correct across fields. – BrianH Feb 21 '19 at 17:04
  • In psych, I have never heard of not knowing the identity of the handling editor. Also, I likewise don't know of any double-blind peer review in psych – Azor Ahai -him- Feb 08 '24 at 17:29
2

I am a mathematician at a small liberal arts college, who has published in a number of fields including biology, economics, criminology, epidemiology, and math/stats education. I became aware of this question about a month ago. Since then, I've talked to colleagues from a number of departments not listed in the answer above, and can therefore add several new academic disciplines to this classification. I also spoke to colleagues at Ohio State University. I asked colleagues what was most prevalent in terms of review type, and included details below if it seemed to be a tie, or that the culture was transitioning.

Single Blind:

  • Geoscience, but there's a culture of reviewers opting to reveal their identities, and shift to Open after writing their review. Sometimes the reviews are published online by the journal alongside the articles.

  • Geochemistry: same as above.

  • Environmental Science: same as above.

  • Some parts of Computer Science, including Machine Learning, which has a culture of putting preprints on arXiv. Note, however that some double-blind journals allow that.

  • Evolutionary Biology

  • Organismal Biology

  • Ecology

  • Chemistry (see also this link)

  • Electrical engineering

  • Medical

Double Blind:

  • Anthropology (speaking of journal articles not books)

  • Asian Studies

  • Marketing

  • Computer Science Education (like SIGCSE)

  • Math Education (like PRIMUS)

  • Stats / Data Science Education (like, the Annual Review of Statistics)

  • Agricultural Economics, though, like economics, a few highly ranked journals are single blind.

  • Political Science

  • English literature (speaking of journals not books)

  • German literature

  • German studies

  • Musicology

  • Music theory

  • History

  • Criminology (like the journal Crime Science)

  • Epidemiology

  • Psychology (especially higher tier journals), though I was told that in recent years, reviewers sign their names around 50% of the time.

  • Physical Science

  • Modern languages

  • Business

Open:

  • Law

See also this question for some info about Astronomy and Astrophysics, and this one for some info about statistics.

David White
  • 669
  • 2
  • 11
  • This is awesome! Thanks for the contribution. You might consider merging this into the above answer, so it's all in one place -- that's what we mean by "community wiki" or community answer, everyone is welcome to edit. – cag51 Mar 12 '24 at 21:13
  • @cag51 You're welcome. I did consider that, but decided to write a separate answer instead, since the source of the info (hallway conversations with liberal arts professors) was relevant, and because I don't think the existence of a CW answer should preclude people from adding their own answers, or force them to only add into the existing CW answer. – David White Mar 12 '24 at 21:15
  • @DavidWhite Rather the contrary, that's exactly what it's for. – Azor Ahai -him- Mar 13 '24 at 00:38
  • 1
    I guess we'll have to agree to disagree then, about forcing people to add to others' cw answers vs allowing them to add their own answers. I swear, every time i venture to academia.se from mathoverflow, I have weird interactions (and often, bad experiences) that make me not want to come back. Including a deleted question a month ago on the topic of this answer. I came back to write this, because I think it's important to get this info out there, but I can't see myself being a regular in this community, given the unwelcoming nature I've experienced multiple times. – David White Mar 13 '24 at 03:42