Do peer-reviewers ignore details in complicated mathematical computations and theorems?

Question

Upon reading many papers in my field, which is Mathematics. I have come across many complicated theorems and proofs in which long overwhelming computations have been done. While paying attention to the peer-review time, I have noticed that many of those papers didn't spend a long time as they should, which is a deduction I based upon the complication of the proofs present in the papers.

This made me ask the question. Do experienced peer-reviewers occasionally ignore verifying "the math", and rather focus on the validity of the techniques used in the proofs and numerical simulations (if applicable)? Since experienced peer-reviewers are very busy academics, I guess it wouldn't be possible for them to verify each point in the proof. However, by not verifying each point of the paper, including the details in the proofs, the peer-review validity becomes questionable.

Answer 1

Preliminary remark

What is considered a "complicated computation" can depend quite heavily on the field, and also on the individual mathematician. But even in fields or papers that don't rely so much on lenghty computations, there might still be lengthy technical arguments, to which the same question applies.

It think it is worthwhile to add the following point to the answers already given.

About checking the correctness of proofs and computations

In my experience many people who start out in mathematical research have a somewhat inaccurate impression of how more experienced people (well, many of them, I guess, though probably not all) tend to check the correctness of mathematical proofs:

Going through every single line of a proof and a computation is not only lenghty and cumbersome, it also tends to be extremely (and by this I mean really extremely) unreliable. Human minds are not computers and are highly succeptible to various types of mistakes - in particular if we don't produce an argument ourself, but just read an argument that somebody else has written down. Without a very good intuitive understanding of a proof, it is just too easy to believe that "every step follows from the previous step" - and to overlook a missing assumption in some step, or a wrongly applied theorem, or a plain sign error, or what not.

What is much more reliable (and, luckily, also much more efficient) are certain high-level checks: figure out the crux of an argument, see where a new idea enters the game, test a result against the common counterexamples which you know prevent related statements from being true, see what part of the argument overcomes, in the present situation, the phenomenon that occurs in all these counterexamples, and so on.

Personal experience

I tend to regularly find critical mathematical mistakes when refereeing papers (although I haven't counted the precise percentage where I found such a mistake) - but I don't remember a single instance where I found a mistake by checking every line in a proof, noticing at one step that "this line doesn't follow from the previous line". In most cases it goes much more like this:

"Ok, Theorem 1 claims this and that. Under the additional assumption A, which is not there, I know that it's true - let me see if I can remember how the argument goes... Ah yes, and for this particular part of the argument I would need assumption A. Now let's see how they do it without this assumption, because this feels like quite a strong claim.

[Start to read the proof, skip the first ten lines because they contain just the usual stuff which one always does to prove results of this type.]

Ah, here's where it get's interesting. They say that claim C is true. Hmm, well, I can see that claim C, once established, will probably imply the theorem. Now let's see how they can get C without A. Ah, ok, here's the crucial argument. Hmm, but I don't understand why this should be true. Let me think again - ok, I have an example which shows that the crucial argument does not work, in general."

This can happen in many variations - but the point is always that, with increasing experience, one often does not need to check every line since one knows how the standard arguments in the field go. So instead one looks for the crucial parts of arguments where things happen that seem to be surprising or non-obvious (to somebody familiar with the standard techniques in the field).

Related

Closely related, and certainly worth a read: Terry Tao's blog post about the three stages of rigour in mathematics.

Answer 2

Here is my perspective as a frequent reviewer in several areas of pure mathematics. I cannot speak for other people so I’ll answer the title question as if it’s asking about my specific practices:

Do peer-reviewers you ignore details in complicated mathematical computations and theorems?

Utopian short answer: no

Realistic short answer: sometimes, but only to the minimal extent necessary

Detailed answer: I consider it my responsibility as a reviewer to check correctness of the results. Correctness comes before anything else: if the paper is not correct it shouldn’t get published, period, and peer review is precisely the sort of adversarial process designed to minimize the risk that authors will get away with mistakes, sloppiness and outright dishonesty (all of which are common human behaviors). Not even trying to check for correctness simply defeats the purpose of peer review.

The only exception to the above is if a journal asks me to review a paper and explicitly says I don’t need to check correctness. And that has only ever happened to me in requests for a “quick opinion”, where I knew that if my opinion was favorable then a more formal and detailed referee report would get solicited later, and that report would include checking for correctness. If a journal asked me for a normal referee report but said I don’t need to check correctness, I would frankly worry whether such a journal should exist and what the point of it is. It would almost certainly devalue my opinion of the journal and of any paper published in it. But as I said, this is not something I’ve ever experienced.

That being said, there are utopian ideals and there is reality. In reality, it is not always practical for authors to include all the formal details of their arguments and calculations (doing so might inflate, say, a 60 page paper to twice the length, ensuring that it wouldn’t be read even by the very small number of people who would be willing to read through a very technical 60 page paper). And similarly it is not always practical for a reviewer to verify all the authors’ calculations - certainly the ones that were omitted, and even the ones that were included. So the only way for me to maintain the ideals I was espousing above would be to simply refuse to review papers where I knew I could not check all the details in a reasonable amount of time.

However, refusing to review a paper could carry its own negative consequences, since many review requests are in specialized areas where I am one of only a few people (or in some cases literally the only person) with the relevant expertise to be able to evaluate the paper. So if I refuse the assignment, it would fall to someone less qualified, who would still likely not check all the details, or worse, the paper couldn’t get reviewed at all. So the outcome of refusing to compromise my ideals could be worse than if I accepted to do the review.

The upshot of this analysis is that sometimes ideals must be compromised a bit. So what I end up doing in such situations (and what I would guess all serious mathematicians do when reviewing papers for respectable journals) is not check all of the details, but check some of the details — as many as can be checked in a reasonable amount of time — and beyond that, I try to look at the broader picture of the arguments the authors are making, see if they make sense by applying my intuition and experience with similar analyses, and running various “sanity checks” to see if the results seem plausible. It is a kind of probabilistic correctness checking, which can still work pretty well (I have caught many mistakes following this approach). With the level of experience I have doing this, I would like to believe that the probability of a serious error managing to get past me is fairly small. Although it is not zero obviously (in fact let’s be honest, it’s not zero even when I do check all the details).

Bottom line, my recommendation to any researchers reading this is, do not allow yourself to be sloppy in writing your papers out of a belief that you will get away with it because the referee doesn’t have time to check what you did. That would be doing a great disservice both to yourself and to your research area. Moreover, it is bound to catch up with you sooner or later.

Answer 3

Indeed, in the last few decades, there has been a clearer mandate from editors to reviewers, to not check "correctness", but, more about "suitability for the journal".

This does pointedly suggest that refereed-journal-publication is not so much a certification of correctness, but a certification of having passed some status-gate-keeping.

Nevertheless, if/when the result is unsurprising, expected, and (therefore?) probably correct, no one cares too much about checking the details. No controversy is being opened. And, in fact, maybe no one really cares...

If/when a paper makes a big claim, then people do start looking critically at the details.

Answer 4

It's a bit of both. First, peer review explicitly states that the validity of the statements is not being checked, and is the responsibility of the author. Peer review decides if the results are original, significant and interesting, that is all.

Having said that, many referees do check arguments. For complicated computations, maybe not every line is checked, but whether it passes the 'sniff test'. Does every line look like it follows from the previous line?

Some referees will check every detail, some will be more holistic about it.

Edit: This was mentioned in the comments, but it deserves to be in the answer, especially as for some reason it was accepted over the other, better, answers. I said that referees are not checking validity. By this I mean that a mathematical theorem is either true or false, and referees do not, and realistically could not, check every paper in sufficient detail to confirm validity. And even then that just means two people have checked it, not that it is true.

Referees are certainly asked to comment on whether they think that a paper is correct. This is a very different thing: I can make a guess as to whether the paper is broadly correct, but I might well be wrong. It is down to the individual where on the spectrum, between checking every detail and just looking over the proofs, they lie.

But very few papers over, say, 50 pages long, have had every detail checked. One of my more recent papers (among other things) corrects the main theorems of three other major papers. Two of these three papers were peer reviewed, but the errors were not caught. This is neither the authors' nor the referees' fault, as the errors were subtle and the result could not be verified computationally at the time. It was only by approaching the problem in a completely different way that the errors came to light.

Do I believe the main theorem of this paper? Of course, but then so did the authors of the other papers. Do I believe my paper is error-free? No. I doubt if any paper over 50 pages long is error-free, because humans are not error-free.