69

Is there an easy way or existing tool to generate a historical chart with the occurrence of one or more user-specified keyword(s) (combinations) in academic publications, based on the words in the title and/or abstract of a paper?

There is a almost-what-I-was-looking-for tool called Google Scholar Trend Miner, but it seems to be not working anymore, as it reports after hitting Go: "It seems that Google found out that we are a bot and started offering its CAPTCHA. Please, wait some hours and try again"

Noble P. Abraham
  • 2,452
  • 1
  • 21
  • 34
Rabarberski
  • 795
  • 1
  • 6
  • 7

6 Answers6

37

If you have access to it, you can very easily do that with Thomson Reuters’ Web of Science portal. Run any query you want, probably starting with the simplest one: Topic=XXXX. Then, select “Analyze results” at the top-right bottom-left of the results list, and sort them by year of publication:

enter image description here

Rabarberski
  • 795
  • 1
  • 6
  • 7
F'x
  • 73,069
  • 23
  • 262
  • 392
  • Great. I couldn't find the "Analyze results" link immediately. Seems it only appears with the "Web of Science" tab selected and not for "All databases" – Rabarberski Sep 26 '12 at 12:53
  • @Rabarberski some of the functionality is avaible only from the "Web of Science" tab, indeed… I've never really understood why. – F'x Sep 26 '12 at 13:01
16

I've written a small Python tool for this purpose. It scrapes Google scholar for each year in a given time span, extracts the occurrences and saves them to a CSV file. The tool is located at: https://github.com/Pold87/academic-keyword-occurrence

Pold
  • 261
  • 2
  • 2
  • the scraper gives an error for too many requests. This error handling is missing. Also, it would be nice to have an option to search exact results by adding quotation marks string using, " /" ". – Manish Jun 07 '22 at 02:26
  • 1
    The error handling is implemented but Google simply only allows a certain number of requests. The option to search for exact results is also implemented (see the closed issues on GitHub for an example). – Pold Jun 08 '22 at 08:17
13

Edit: I though I would update this answer with a tool I recently discovered--trends.google.com

It's not specific to peer-reviewed publications, but allows one to search keywords by country, category (e.g., science), and web/image/news/.

enter image description here

Google's Ngram viewer is also useful. It's quick and easy for seeing trends as far back as the 1800's.

enter image description here

derelict
  • 2,479
  • 18
  • 24
6

You can have a look at the arXiv cultoromics website at http://arxiv.culturomics.org, which however searches in articles on the arXiv. Depending on your field of interest, this may or may not be good enough.

Pieter Naaijkens
  • 8,011
  • 1
  • 40
  • 41
6

Scholar Plotr doesn't work anymore due to rate limiting by Google. An alternative easy to use and free tool for anyone without academic licences is Dimensions AI:

app.dimensions.ai

It works with it's own publications database and shows a lot of statistics e.g. publications per year, citations per year.

very
  • 165
  • 2
  • 7
  • I wouldn't say GooSchol only shows academic research. – Kimball Aug 08 '17 at 13:52
  • according to themselves they do: "Google Scholar includes journal and conference papers, theses and dissertations, academic books, pre-prints, abstracts, technical reports and other scholarly literature from all broad areas of research. You'll find works from a wide variety of academic publishers, professional societies and university repositories, as well as scholarly articles available anywhere across the web. Google Scholar also includes court opinions and patents." from google scholar – very Aug 09 '17 at 07:58
  • but of course something might slip in there, that isn't of a certain standard. – very Aug 09 '17 at 08:05
  • 2
    I just meant that not all academic books and scholarly literature counts under research. For instance, some course notes I've written are indexed by Google Scholar. While they're certainly academic, most of them are not research level. – Kimball Aug 09 '17 at 08:59
  • 1
    Now they recommend https://app.dimensions.ai/discover/publication on the top of the their webpage, since their site doesn't work reliably any more. – Armut Oct 11 '21 at 22:37
3

Web of Science is a great resource for a historical review or a trend analysis of a keyword or subject. Another resource is Scopus. To use Scopus to search for a review of a phrase, word, or keyword from a controlled vocabulary keyword such as MeSH or EMTREE, enter the phrase, word or keyword in the search box and select the appropriate search filter to the right of the search box. You can search by title, abstract, keyword, or any combination of the three. Results can be filtered by a number of options and also can be exported for further analysis. The caveat to databases such as Web of Science or Scopus is to check the date range of the materials indexed.

user2768
  • 31
  • 1