Introducing a novelty indicator for scientific research: validating the knowledge-based combinatorial approach

Kuniko Matsumoto, Sotaro Shibayama, Byeongwoo Kang, Masatsura Igami

Research output: Contribution to journalArticlepeer-review


Citation counts have long been considered as the primary bibliographic indicator for evaluating the quality of research—a practice premised on the assumption that citation count is reflective of the impact of a scientific publication. However, identifying several limitations in the use of citation counts alone, scholars have advanced the need for multifaceted quality evaluation methods. In this study, we apply a novelty indicator to quantify the degree of citation similarity between a focal paper and a pre-existing same-domain paper from various fields in the natural sciences by proposing a new way of identifying papers that fall into the same domain of focal papers using bibliometric data only. We also conduct a validation analysis, using Japanese survey data, to confirm its usefulness. Employing ordered logit and ordinary least squares regression models, this study tests the consistency between the novelty scores of 1871 Japanese papers published in the natural sciences between 2001 and 2006 and researchers’ subjective judgments of their novelty. The results show statistically positive correlations between novelty scores and researchers’ assessment of research types reflecting aspects of novelty in various natural science fields. As such, this study demonstrates that the proposed novelty indicator is a suitable means of identifying the novelty of various types of natural scientific research.
Original languageEnglish
Pages (from-to)6891-6915
Publication statusPublished - 2021

Subject classification (UKÄ)

  • Information Studies

Free keywords

  • bibliometrics
  • novelty
  • reference combination
  • validation


Dive into the research topics of 'Introducing a novelty indicator for scientific research: validating the knowledge-based combinatorial approach'. Together they form a unique fingerprint.

Cite this