Abstract: A paradigm shift in peer review quality research——driven by the progressive popularity of open peer review and the rapid advancement of natural language processing jointly——is happening, while peer review comment mining (PRCM) is becoming universal unobtrusively. This paper comprehensively summarizes the relevant literature on PRCM to clarify the general characteristics of this emerging research direction, plus the opportunities and challenges it may bring, with the purpose of providing a reference guide for subsequent studies. The literature review shows, nowadays, the vast majority of PRCM-related research focuses on the investigation of prediction capability, emotion expression, and argumentation logic with respect to peer review comments, with special attention paid to the feasibility and validity of various text mining methods when applied to interpreting and analyzing peer review comments. In addition, the data samples used in PRCM-related research are limited to conference papers on computer science (in English).

Key words: Peer review, Peer review quality, Peer review comment mining, Text mining, Science and technology evaluation