关键词: 网民情感, 多模态融合, 情感识别, 双向长短期记忆模型, 微调卷积神经网络, 网络舆情, 舆情监测

Abstract: The research of sentiment recognition of online users mostly are based texts, lacking the research which consider the texts and attached images to recognize the sentiment. The paper proposed a DNNs-SVM multimodal fusion model. In the extraction of textual features, we used word2vec model to represent texts. And a BiLSTMs model was built to extract the features of texts. In the extraction of visual features, we built a fine-tuned CNNs, using VGG16 as base model, to extract the features of images. We concatenated textual features and visual features in feature-level. Then the fused features were fed into SVM classifier to complete multimodal sentiment recognition. Additionally, the proposed model was compared to designed baseline models. The baseline models were word2vec+BiLSTMs, BERT+BiLSTMs, CNNs, fine-tuned CNNs and DNNs. The results showed that the results of fused features outperformed that of unimodal features and the proposed model outperformed all baseline models.

Key words: Sentiment of online users; Multimodal fusion, Sentiment recognition, BiLSTMs, Fine-tuned CNNs, Online public opinion, Public opinion monitoring