3

I am a researcher in the energy field. I am trying to search for academic publications without the keyword optim* (where * represents a wildcard) not only in the abstract and title, but also in full text. The goal is to filter out any study that uses optimization.

I have tried Google Scholar, Scopus and Web of Science, but it looks like none of them supports full text search. Is there any search engine that supports that?

CrimsonDark
  • 11,150
  • 1
  • 28
  • 64
cctau7283
  • 31
  • 1
  • 3
    Apparently PubMed Central does but it likely doesn't have the coverage you want. Seems worth asking your local research librarian for advice. Especially since this looks like a non-trivial thing to filter out. At the very least, you'd have to do additional filtering to remove papers using other terminology, including maximizing/minimizing as well as the names of various optimization methods. – Anyon Feb 05 '24 at 09:08
  • 3
    One problem with filtering out words related to optimization is that you'll filter out papers that mention optimization in the introductory literature summary or in a footnote (e.g. the author states that ideas studied in an earlier paper using optimization methods will be studied in the present paper without making use of optimization methods), and even papers that simply say that optimization methods will not be used in the paper. – Dave L Renfro Feb 05 '24 at 10:01
  • 1
    Related question: https://academia.stackexchange.com/questions/204945/searching-for-articles-containing-a-phrase-in-the-full-text-body-but-are-absent/204946#204946 – The Doctor Feb 05 '24 at 10:34
  • 2
    "We think these methods are optimal for this problem." "We are optimistic that this method will be useful in future." – David A. Craven Feb 05 '24 at 11:06
  • 1
    "We do not use any form of optimization in this work." – David Ketcheson Feb 05 '24 at 14:41

1 Answers1

2

This is probably not what you want to do. We are in an age where AI is becoming much more intelligent in helping us find what we want when we search, but classic keyword searches do not use AI and they do not work the way you seem to think. My comments that follow apply to classic keyword searching, not to rapidly evolving AI-based searching (for which I have no general comments, other than that its potential is very exciting).

It is a common misunderstanding that with a keyword search, we tell the computer what we want it to find for us. No, a keyword search tells the computer what we do NOT want to see. This makes more sense if you think of something like this: suppose we have 1000 articles in our database. A keyword search of "apple* OR banana*" does not tell the computer to give us all articles about apples or bananas. Rather, it tells the computer to show us everything it has except for articles that do not mention apples or bananas. Suppose the computer comes up with 500 articles that mention apples or bananas. But 400 of these talk about pears. I'm not interested in pears, only apples or bananas. If I then search for "apple* OR banana* NOT pear*", I probably would not get what I want. I might end up with 25 articles that indeed talk about apples and bananas with no mention of pears, but I would exclude 375 articles that mention all three. Most likely some of those excluded articles would be very interesting to me, especially those that talk about "why apples and bananas are great but pears are not".

Please excuse my simplistic example, but I hope it makes it clear that by searching for "NOT optim*" (or something like that), you would exclude articles that cover "the energy topic you really want, but not focusing on optimization".

Keyword searches are best used as an initial broad net that should catch many articles. The NOT keyword should be used very sparingly. The appropriate approach would be to search for the keywords that you want and then read the titles and abstracts of all the articles you identify and then manually exclude the ones that are overly focused on optimization. I hope that makes sense.

Tripartio
  • 9,347
  • 2
  • 24
  • 49
  • Ok - your main point seems to be just that you shouldn't search for NOT optim* because then you might exclude lots of papers that actually are relevant - in other words, the same point made by two people in the comments under the original question. – toby544 Feb 08 '24 at 13:53
  • 1
    "It is a common misunderstanding that with a keyword search, we tell the computer what we want it to find for us. No, a keyword search tells the computer what we do NOT want to see." I don't think there is any common misunderstanding. Everyone knows that if you search for apple OR banana you will find all articles that contain one of those words. – toby544 Feb 08 '24 at 13:54