信息资源管理学报 ›› 2022, Vol. 12 ›› Issue (4): 56-69.doi: 10.13365/j.jirm.2022.04.056

所属专题: 学术评价改革研究

• 学术专栏-学术评价 • 上一篇    下一篇

一种面向内容差异的学术论文评价方法

陈玥彤 王昊 李跃艳 张卫 邓三鸿   

  1. 1.南京大学信息管理学院,南京,210023;
    2.南京大学江苏省数据工程与知识服务重点实验室,南京,210023
  • 出版日期:2022-07-26 发布日期:2022-09-18
  • 作者简介:陈玥彤,硕士生,研究方向为科学计量与知识挖掘;王昊(通讯作者),教授,博士生导师,研究方向为知识本体构建及应用、数据挖掘技术应用、科学评价和引文分析,Email:ywhaowang@nju.edu.cn;李跃艳,博士生,研究方向为知识组织和本体构建;张卫,博士生,研究方向为领域知识组织与自然语言处理:邓三鸿,教授,博士生导师,研究方向为信息检索与知识管理。
  • 基金资助:
    本文系国家社科基金项目“大数据环境下学术成果真实价值与影响的实时预测及长期评价研究”(19BTQ062)、江苏省“六大人才高峰”高层次人才项目“多粒度学术对象区分性测度和分析研究”(JY001)、江苏省研究生实践创新计划《基于文档区分能力的学术论文评价方法》(SJCX21_0020)的研究成果。

An Academic Articles Evaluation Method Oriented to Content Differentiation

Chen Yuetong Wang Hao Li Yueyan Zhang Wei Deng Sanhong   

  1. 1.School of Information Management, Nanjing University, Nanjing,210023;
    2.Jiangsu Key Laboratory of Data Engineering and Knowledge Service, Nanjing University, Nanjing,210023
  • Online:2022-07-26 Published:2022-09-18

摘要: 当前学者普遍采取被引频次、h指数等外部指标评价学术论文,往往难以反映论文的内容质量,甚至可能引发重“量”轻“质”的趋向,从而抑制学术创新和学科发展。本研究从差异性的视角提出一个新的论文评价指标模型,以图书馆、情报与文献学学科的论文作为研究对象,该模型通过结合具有语义表征的BERT模型和ADC方法测度研究内容层面的论文差异程度,通过空间分布特征分析论文对象的区分能力特征,并从期刊、学者和主题的角度分别探讨论文ADC总体水平差异。实验结果表明,本研究提出的ADC指标模型用于衡量学术论文的内容差异性具有一定的合理性,论文区分能力越强表明其研究内容越具独特性,越弱则说明研究同质性越明显;档案学期刊由于学科术语和研究对象的独特性而明显区别于图书馆学和情报学期刊;高产学者中偏重情报学学科的人数较多,研究热点和前沿应用的论文由于主题更为深入、丰富而表现出更高的区分性,而偏向于介绍传统理论和综述的文章由于内容更加普遍和成熟而表现出较低的区分性。

关键词: 论文区分能力, 学术论文, 内容差异性, LDA主题分析, 学术评价

Abstract: At present, scholars generally adopt external indicators such as citation frequency and hindex to evaluate academic articles, but it is often difficult to reflect the quality of the content of the papers, and may even lead to the tendency of emphasizing "quantity" rather than "quality", thus inhibiting academic innovation and discipline development. In this paper, we propose a method to evaluate the academic articles from the perspective of content differentiation. The research object of this paper is the articles from the discipline of library, information and archival Science. By combing BERT with semantic representation and the ADC (article discriminative capacity), this paper measures the difference degree of articles at the level of research content, and analyzes the article discriminative capacity by the numerical distribution characteristics. Differences in the overall level of ADC are explored separately from the perspective of journals, scholars, and topics. The experiments show that the ADC proposed in this study is reasonable for measuring the content differences of articles. The articles with higher discriminative capacity have more unique research content, otherwise there will be more homogenized content. Archival journals are clearly distinguished from library and information journals due to the uniqueness of subject terminology and research objects. Authors studying information science make up a large percentage of productive authors. Articles on hot topics and frontier applications show higher discriminative capacity due to more indepth and popular topics, while articles introducing traditional theories and literature review show lower discriminative capacity due to more general and mature contents.

Key words: Article discriminative Capacity, Academic articles, Content differentiation, LDA, Academic evaluation

中图分类号: