7

I co-authored a method, which yielded state-of-the-art results in several machine learning benchmarks. However, the paper got rejected from two conferences and then from a super slow journal. With the time for improvements, it all took 3 years. The method is no longer state-of-the-art, but I'd still like to publish the article.

The question is: what's the most elegant way to write (e.g. in the abstract or contributions section) that the method was state at the moment of creating it, but it's no longer anymore? If that matters, there's an old pre-print on the arXiv, so at least there's some proof that it was SOTA.

In other words: I want to make it clear to reviewers that it was SOTA, but I can't brag about it like it's SOTA anymore.

EDIT: The previous rejection reasons were mostly connected to not being novel enough (it's an incremental work rather than a revolution, but we'll aim at less prestigious place to publish it) and some missing things like additional ablations (which are intended to be included in the journal version). We won't aim at any A* conference anymore, I just want it to make it through to a decent place, since anything that I don't publish is invisible for academia in my country. It wasn't a desk reject, we just aimed too high at first.

Dominik Filipiak
  • 246
  • 3
  • 8
  • 3
    I cab see why you would like to publish something but I don't see what you could do to attract a publisher now if publishers weren't interested when (by your estimate) it was SOTA. – Ethan Bolker Feb 18 '24 at 23:23
  • 1
    The issue is that publishing this now does not advance the state of the art. Why would a (reputable) publisher take such a paper? – Wolfgang Bangerth Feb 19 '24 at 04:30
  • @EthanBolker because there's still some contributions - it was not enough for the initial venues, but we aimed too high. Please see the edited question. – Dominik Filipiak Feb 19 '24 at 13:53
  • @WolfgangBangerth Please see the updated question – Dominik Filipiak Feb 19 '24 at 13:53
  • What exactly does it mean that the method is no longer SOTA? Is it just that better results on benchmarks have been achieved by recent approaches, or have people done something that makes what you did itself redundant because for example people have solved a specific issue that made your method strong in a better way, so people would read your paper and immediately think that the issues should be dealt with in the way a now available other method does? In the first case your method can still be of methodological interest. – Christian Hennig Feb 20 '24 at 18:13
  • I think for publication chances in the present and also for the interest of the paper it is rather irrelevant that the method was SOTA at some point, and as a reviewer I wouldn't want to see such a statement in the paper. Authors need to convince me that it is worthwhile to have this in the literature now, and a performance that people improved upon in the meantime doesn't do that. Methodological interest on the other hand can do it; maybe potential to inspire further improvements etc. – Christian Hennig Feb 20 '24 at 18:16

3 Answers3

6

The answer to your question will critically depend upon why your original conference and journal submissions were rejected. You mention that your methods were "state of the art" but it is not clear whether that was only a self-assessed view, or whether there were other indicators that it was indeed state of the art.

If the material was considered to be uninteresting or significantly flawed even a few years ago, then that obviously weighs against the likelihood of now finding a journal that would want to publish it. On the other hand, if you got rejections for reasons that indicated that the work was good but the submission required revision before re-submission, that is an entirely different matter but it changes the kind of journal to which you might now consider sending your paper.

Response to comment

Don't get misled. There is a great deal of difference between:

  1. "advancing the state of the art" (which refers to whether a study or method contributes to the the whole body of knowledge about a domain), and
  2. being state-of-the-art, which is a kind of synonym for *current bleeding edge, and frequently represents a fusion of fashion together with the currently preferred trade-off between competing resources (e.g., availability and required size of the training corpus, memory, speed of execution and speed of training).

I recall a time in the 1990s when RISC architecture was the hot-new thing. If you wanted to look "state-of-the-art" at conferences on hardware architecture, you'd want to be presenting work on RISC. Sure, RISC is dominant in mobile embedded systems, but Intel and AMD are still making significant progress in CISC systems ... and I imagine researchers are still publishing about them.

It sounds as if your own study might at one stage have both advanced the state of the art, and been SoA. The fact that it is no longer SoA does not mean that it couldn't, if you were to publish it, advance the state of the art. Look, for example at this answer to a recent question about "failing ideas".

CrimsonDark
  • 11,150
  • 1
  • 28
  • 64
6

That's a crummy situation to be in, and I'm sorry you have to deal with it.

I'm assuming that your "state of the art" statement is based on model performance compared to the field at the time you made it. If that's the case, your best bet is probably to emphasize what advantage/contribution your method brings to the table that make it worth publishing now despite the fact that it is a step back from what current SOTA ML methods are now capable of.

For example - is your method easier to use/install than current SOTA? Does it require less data? Is it a smaller/lighterweight model? Did you do something cool in the model construction that other people would be interested in building off of? Basically you need to demonstrate in some way that the loss in performance compared to current SOTA is outweighed by other factors.

Also, as you're probably aware, you're going to have a much better chance of getting this published at all in a lower tier journal, as most of the journals with a high impact factor are going to have reviewers that will not be impressed if the method isn't CURRENTLY better than what's already out there, regardless of any non-performance-related reasons that people would be interested in the mode.

WhatTheDuck
  • 1,248
  • 1
  • 11
1

If you can afford the publication fee, just send it off to Scientific Reports, which reviews for correct methods and interpretation but not for novelty and impact, and be done with it. Scientific Reports has surprisingly good citation metrics despite that.

Significance
  • 6,054
  • 1
  • 21
  • 40
  • 1
    nice idea. Won't it be considered as an MDPI-equivalent though (that is, "pay-to-win")? Not sure what's the perception of SR in academia, never published there. – Dominik Filipiak Feb 22 '24 at 15:55
  • 1
    Scientific Reports is a Nature journal, so it gets a bit of glow for that. It also has a fast turn-around which helps make it popular. A lot of my colleagues, some of whom also publish in Nature's flagship journal and have very high h-indices, send their good but less-exciting work to Scientific Reports. – Significance Feb 29 '24 at 23:25