62

I am in a bit of a situation. Basically, the research on which I was basing my PhD was carried out by a former post-doc in our group. After one year and a half not being able to replicate his results, I exploited my admin privileges on our cluster to download his codes. I discovered that the algorithms he used in his publication are different from the ones used in the experiments, as the ones he claims he is using are not really applicable and the ones he is using do not have the claimed complexity. He even went so far to provide made up (theoretical) timings to cover this up.

My supervisor knows about this issue from me and he also knows that the post-doc has stopped responding to my mails trying to replicated his work. My supervisor is very hands-off and I don't believe that he has any fault in this but his name is on the publication. Moreover, I feel he is disappointed in the post-doc and feels guilty. So, it seems to me that pressing the issue does no good for me and I have stopped mentioning the topic to my supervisor. (I am still quite angry about this issue as this has cost me about a year and a half of my PhD)

I have been able to salvage the original idea of the publication and come up with an algorithm which has the correct complexity and we are now writing the paper. To the reader these two publications might appear extremely similar and I would like to include a sentence in the introduction that the previous publication is not a working algorithm. What is a good, diplomatic way of doing this?

  • 53
    There isn’t a diplomatic way to expose someone as having committed academic fraud. You either expose him undiplomatically and face the shitstorm that is likely to follow, or give up the idea of insinuating that his algorithm doesn’t work as claimed. – Dan Romik Apr 26 '21 at 15:51
  • 8
    @DanRomik While I fully agree that in case of fraud, there is no nice way to resolve the issue, I would give first give the benefit of doubt to the people involved. Being sloppy, making wrong assumptions, or making errors, even when they are serious is not the same as committing academic fraud. – Greg Apr 27 '21 at 03:18
  • 5
    @Greg did you read the question? Academic fraud is what the question is about, and I don’t see a point in questioning OP’s premise, which seems based on very clear evidence. – Dan Romik Apr 27 '21 at 03:39
  • 7
    @DanRomik It's quite possible that the OP has missed or misinterpreted some of the post-docs work, especially when they haven't even been able to talk to them about it. It seems safer to approach it as improving/making corrections to prior work than to approach it with big accusations. – setholopolus Apr 27 '21 at 19:32
  • @setholopolus sure, anything’s possible. I don’t have a strong opinion about it, I was just addressing the question as it is written. – Dan Romik Apr 27 '21 at 21:33
  • I was tempted to agree with Dan Romik's Comment yet how would it be wrong to say "Other/certain/previous publications have suggested (detail) but (refutation)"? – Robbie Goodwin Apr 28 '21 at 21:19
  • 3
    I exploited my admin privileges on our cluster to download his codes → I suffer by just reading this part. Besides your actual problem, you should have your admin rights revoked after such a thing (at a minimum). Depending on the jurisdiction this can be theft, exposition of private data, and other monstruosities. – WoJ Apr 29 '21 at 13:47
  • 1
    Crap like this is why I left academia. Even at a global top-20 university I found this kind of stuff to be way more common than you would think. Oddly, in the private sector, if it doesn't work, you don't get paid. Weird to think that capitalism keeps people more honest than the ivory towers of academia, but welcome to the future. – J... Apr 29 '21 at 15:07
  • 3
    @WoJ It could just be poor ESL phrasing - given that the research belongs to the group, it's quite likely that OP actually has free access to those documents and is simply representing what happened poorly. – J... Apr 29 '21 at 21:52

5 Answers5

71

There are lots of ways to go about this.

Since your advisor is a co-author on the previous paper, this may be difficult, since they are invested in the existing paper and have some credibility at stake. Theoretically, they should be happy to retract/rewrite the older paper, but we are human. You are probably correct that they feel guilty- everyone with their name on the paper is ultimately responsible for all of the content.

At the same time, this can make some things easier for you. Finding an error in previously published work opens some new avenues for publication. Now, instead of a simple sentence, you can issuing a comment or a correction on a previous paper - many journals will allow you to publish a correction or remark on their previous articles, particularly if you can demonstrate a major problem or flaw.

Critically, do not be a jerk about this process. Simply state that there has been an oversight or error in the work by XYZ. Do not speculate as to why it went wrong or how. Even with evidence that their results do not match their data, there could be numerous reasons for it. Journals don't want to be told they did a bad job, neither does your advisor- but everyone would prefer you correct an error softly and with tact rather than letting someone else do it loudly and disruptively.

Edit: I am not sure your evidence is legally obtained, nor am I fully confident from this post that the evidence is conclusive that they were fraudulent. Until both of those conditions are met, I wouldn't recommend making accusations of data fabrication - publicly or privately. You will have, in all likelihood, a legal battle - for which I am not a lawyer. But you can conclusively say, as another commenter mentioned: "We were not able to reproduce the results."

RegressForward
  • 1,551
  • 7
  • 12
  • 42
    That's right, as he couldn't reproduce the result of the previous paper, and he has a working, thus probably different algorithm now, he just has to write: "A similar algorithm was presented in [X], but we were unable to reproduce the result. Our algorithm is novel because of [reasons], and is able to produce the excepted result." Not being able to reproduce something is a pretty big deal already. (This may be complicated by the authorship of the supervisor again, though.) @rptr5 – Neinstein Apr 27 '21 at 06:47
  • 6
    If anything I'd consider a research more credible if in a later paper they openly re-analyze their previous approach and correct their findings. I can understand though that this can be hard for some people. –  Apr 27 '21 at 11:35
  • I totally disagree. Somebody reading not being able to reproduce a claimed better result might read it as you being incompetent, while in reality the result was fake. So I think this is a bad answer – Hakaishin Apr 27 '21 at 15:12
  • 8
    @Hakaishin It might vary based on the fields, but in my field (CS), the phrase "unable to reproduce prior work" doesn't have negative connotation. It simply means there could be some details that were omitted, or there could be a mistake in the original paper, or the author could not get the same data set/split, and many other reasons. So it's still reasonable to say. – justhalf Apr 27 '21 at 16:06
  • 2
    @justhalf - that's great for CS especially in the light of this paper: Producing wrong data without doing anything obviously wrong! (Mytkowicz, Diwan, Hauswirth, Sweeney, 2009) – davidbak Apr 27 '21 at 18:05
  • 1
    Also remember that this paper was published and no one during the review and approval process picked up on the errors either. – JeffC Apr 27 '21 at 18:46
  • 5
    It seems patently implausible (in my field anyway) that an introduction as bare of details as "We could not reproduce [results from our own group], so we [redid it without further analysis]" would ever get past peer-review. I would expect to be asked aggressively about this in the review, and I might even expect that some reviewers would sound the alarm about it. – nben Apr 28 '21 at 15:47
  • Especially when it comes to algorithms, bugs and programming errors happen quite frequently. That does not necessarily imply that the post doc knowingly commited fraud. – HerpDerpington Apr 29 '21 at 15:40
  • @Hakaishin, this student believes it is fake. It's possible that the previous student made an error, misunderstood the task, misunderstood the results, copied the results poorly in transfer, etc. If it is possible to believe incompetence by a PhD student, this seems like the safest route to go unless you have evidence in hand that it is fraudulent and you had the rights to that evidence, of which is unclear from this post. It seems like several alarming things are happening- to which I recommend doing honest, nonspeculative work so you don't get burned next! – RegressForward Apr 29 '21 at 18:09
  • 1
    I broadly agree with the thrust of this answer, but I think the wording of "we were not able to reproduce the results" is a bit better than "these results don't appear to be repeatable". The former is factual, whereas the latter is actually quite vague. If you want to stay factual and really make a point, you could perhaps say "even after obtaining the original code, we were unable to reproduce the results". That conveys the point that at best it's non-trivial or impossible to reproduce the results even with the original code in hand, when it should be easy, but it avoids direct blame. – Stuart Golodetz Apr 29 '21 at 22:52
  • Notably, that wording is completely factual, and doesn't say more than you want: it doesn't say how you obtained the original code, it doesn't claim that the results can't be repeated by anyone, which you would have to show, and it doesn't say in what manner you used the original code whilst failing to produce the results. It simply says you had the code, and you didn't manage it, for whatever reason. People may (rightly) assume that you're not so incompetent that you can't run code to reproduce trivially reproducible results, and that the old code is bad. But that's just their assumption. – Stuart Golodetz Apr 29 '21 at 22:57
28

If your advisor knows that the paper was published with fake claims/data and is not willing to take action, I would sincerely consider working under someone else's supervision. This should be something that keeps him up at night for one or two days, but after accepting the hit that his career will take for this mistake, it is absolutely fundamental that he takes action - even if the action goes against his own reputation.

If your advisor is also an author of the paper, this matter is also interesting for another reason: It exposes how little some of the authors know what the hell is going on behind a paper with their names...

QuantumBrick
  • 2,121
  • 1
  • 15
  • 24
  • 4
    "Gift authorship" is pretty common in a lot of stem fields. Often it is people's advisors who are receiving the authorship for merely providing funding to their group members. – Taw Apr 27 '21 at 05:16
  • 4
    @Taw yes, I know. What do you think of this? Personally, I find it a very bad model. – QuantumBrick Apr 27 '21 at 06:09
  • 13
    Convincing funding agencies that an idea is worth exploring and to get you money for it is non trivial. You should try it some times. – Marianne013 Apr 27 '21 at 13:22
  • 2
    @Marianne013 People are not always hired with project money. Many research institutes have a fixed number of available positions for new postdocs/PhD students, regardless of what the project is. In fact, funding for some institutes is so insanely high that it is sometimes hard to fill up all the spots. Besides, the role of a "sponsor" should not be taken as equivalent to the role of an author. The last name convention is also inadequate, because it can be either the group leader or just the guy who paid the bills. Donating money to the circus does not turn you into a clown. – QuantumBrick Apr 27 '21 at 13:48
  • 6
    Authorship is more complex than some people think. Many papers in stem fields require a variety of skills. I work a lot with biologists and doctors. I hope I'm not expected to check their cultures or observe them while they visit patients. And I'm sure nobody expects them to check my models and my code. – Luca Citi Apr 27 '21 at 20:20
  • 5
    @LucaCiti it doesn't matter. After malpractice is discovered, the person in charge is expected to take action – QuantumBrick Apr 27 '21 at 20:40
  • 1
    @QuantumBrick - This is only true of a vanishingly small proportion of institutes, maybe 2-3% of the scientific workforce (funding for a fixed number of positions), at least in biology. The position of PI on the paper is not because they are the sponser, but rather, beause they got the funding, they must have come up with the concept for the research project in the first place. – Ian Sudbery Apr 30 '21 at 12:01
  • 1
    No one author can possibly know the details of everything on a paper. If one person understands everything another person did, why would they both need to be on the paper. The last paper I published had 10,000 lines of code, and only two of the 8 authors knew any programming. That doesn't mean that the senior author shouldn't take action if malpractice is discovered, just that its not necessarily their fault if they didn't spot it before. – Ian Sudbery Apr 30 '21 at 12:03
  • @IanSudbery I agree with everything you said. I never said it was their fault, only that they need to take action... I'm just a little bit tired of seeing people having ideas, working them by themselves, writing the whole paper, and adding the name of someone who didn't do anything just because _________ (insert justification). Maybe this is precisely one situation where they can be useful by sharing the guilt, who knows. – QuantumBrick Apr 30 '21 at 12:33
  • 1
    At least in biology it is rare that the senior author didn't do anything. Mostly likely the whole idea behind the project was theirs. Indeed, this is often the case because Students have 4 year contracts, and postdocs have 3 year contracts, and most papers these days have at least 10 person-years of work in them. So its highly unlikely (although not impossible) that a student or postdoc will have an idea and take it all the way to publication, on their own, without input from their supervisor. – Ian Sudbery Apr 30 '21 at 15:33
  • 1
    I think what both I and @LucaCiti are objecting to is this sentence: "It exposes how little some of the authors know what the hell is going on behind a paper with their names..." – Ian Sudbery Apr 30 '21 at 15:33
  • @IanSudbery I believe this is then highly field-dependent. In physics, at least among me and all my acquaintances and friends, it is very common for post-docs and PhD students to include people that have done literally nothing as authors. I have done it many times myself. – QuantumBrick Apr 30 '21 at 15:45
11

Most universities should have a research integrity officer. You could contact them to ask what you should do next. We had to go through ethical research training, and the training said that you could even ask them a question "hypothetically" to explore your options without having to jump into accusations.

Anna Duricy
  • 111
  • 3
8

If the previous paper is just sloppy, you can write something of the sort "We present an improved analysis of..." and then explain in your paper the advantages of your technique.

If there is scientific misconduct in the previous paper, ideally your supervisor would have it retracted. Also depends to a certain extent if the former postdoc is still in the same field.

Mister Mak
  • 1,401
  • 3
  • 11
  • 3
    I would like to second the retraction suggestion, but in principle, an erratum might be sufficient, if, as the topicstarter says, "To the reader these two publications might appear extremely similar". – sleepy Apr 26 '21 at 16:52
  • 2
    This is not enough imho. I would add "We were not able to replicate the previous results." – PatrickT Apr 27 '21 at 01:58
5

Have a sit-down with your supervisor and speak openly and frankly about this. Prepare carefully how you are going to put this into words. Ask him how he would like to address the problem, but ultimately accept his decision, whether he bravely faces it or shrugs it off, because you have little to gain in dragging him down.

Write up a "Corrigendum" in which, with your supervisor's consent, you make references to the fact that you were unable to replicate the previous results. If your supervisor wishes, he can allege a "spreadsheet" error in the previous study, otherwise just leave it vague.

Hopefully no-one was seriously injured in the process: The time you spent attempting to replicate a fraudulent result is not entirely wasted, I'm sure. Whatever evidence you have uncovered about the fraud would not stand in a court of law. You have learned a valuable lesson: academia does have its fair share of incompetence and fraud.

PatrickT
  • 880
  • 7
  • 13