关键词: 少样本, 大模型, 模型蒸馏, 科技实体, 实体抽取

Abstract: Faced with issues such as high resource consumption, long processing times, and poor scalability in scientific entity extraction tasks, this paper proposes a collaborative training framework that balances the advantages of both small and large language models. The paper tests the effectiveness of this learning framework's model training under different data scales. Using four datasets from different fields—NCBI, BC4CHEMD, S800, and SCIERC—this method is shown to achieve results consistent with full-data fine-tuning in few-shot environments. The paper provides an in-depth analysis of the limitations of large language model prediction strategies in scientific entity extraction tasks and systematically tests the model performance exhibited by small and large language models under different data scales through multiple rounds of collaborative training. Additionally, from the dual perspectives of small model recognition strategies and training data similarity, this paper thoroughly examines the reasons for the improved performance of the proposed learning framework. The collaborative training framework built in this paper enables the simultaneous exploitation of large language model cognitive advantages and small model low-cost, high-efficiency operations, thus better supporting efficient extraction of bibliometric information in low-resource, few-shot environments.

Key words: Few-shot, Large language model, Model distillation, Scientific entities, Entity extraction