Computer Science and Application
Vol. 10  No. 11 ( 2020 ), Article ID: 38831 , 14 pages
10.12677/CSA.2020.1011218

[目的/意义]近年来，将注意力机制与LSTM模型结合的方法常被用于要素类情感分类任务，但是该方法存在参数多、训练时间长的弊端。门控卷积神经网络模型不仅结构简单、参数少、运行时间短，还能分别提取要素特征和情感特征、具有较高的分类精度，但是该模型采用的要素类嵌入是预定义的、与上下文无关。对要素类情感分类任务来说，要素类的质量对预测文本在要素类上情感极性的准确率的重要性不言而喻。[方法/过程]本文关联要素类提取和要素类情感分类任务，提出融合自注意力机制下的要素类特征的门控卷积神经网络模型，通过结合自注意力机制的神经网络提取出基于上下文优化的要素类嵌入，然后将优化后的要素类向量和文本词向量通过门控卷积神经网络进行训练。[结果/结论]在2014年至2016年的SemEval数据集上的实验结果表明，本文提出的模型能有效改善要素类提取的效果和提高要素类情感分类的分类准确率。

Research on Aspect Category Sentiment Classification Based on Gated Convolution Neural Network Combined with Self-Attention Mechanism

Ying Zhang, Jianguo Zheng

School of Management, Donghua University, Shanghai

Received: Nov. 4th, 2020; accepted: Nov. 19th, 2020; published: Nov. 26th, 2020

ABSTRACT

[Purpose/Significance] In recent years, a common method for aspect category sentiment classification is to combine LSTM model with attention mechanism. Compared to that, the gated convolutional neural network model not only has a simple structure, fewer parameters and shorter training time, but also achieves higher classification accuracy being able to extract aspect features and emotion features. [Method/Process] Considering that the quality of aspect category is crucial for aspect category sentiment classification, this paper coupled aspect category extraction and aspect category sentiment classification, and put forward Gated Convolutional Neural Network with Self Attention-based Aspect Embedding (GCAE_SelfAtt) model to relate the aspect category embeddings to corresponding context, and to achieve a higher accuracy. [Result/Conclusion] The experiment on SemEval dataset shows that GCAE_SelfAtt model does help to extract more coherent aspect categories and achieve higher accuracy for sentiment classification.

Keywords:Aspect Category Extraction, Aspect Category Sentiment Classification, Self-Attention Mechanism, Gated Convolutional Nueral Network

1. 引言

2. 相关工作

Table 1. Summary on aspect-level sentiment classification research based on deep learning

3. 融合自注意力机制下的要素类特征的门控卷积神经网络模型

Figure 1. Schematic diagram of GCAE_SelfAtt model

3.1. 任务定义

3.2. 构建词向量输入层

3.3. 句子表示层

${y}_{s}=\frac{1}{n}{\sum }_{i=1}^{n}{e}_{{w}_{i}}$ (1)

${d}_{i}={e}_{{w}_{i}}^{T}\cdot M\cdot {y}_{s}$ (2)

${a}_{i}=\frac{\mathrm{exp}\left({d}_{i}\right)}{{\sum }_{j=1}^{n}\mathrm{exp}\left({d}_{j}\right)}$ (3)

${z}_{s}={\sum }_{i=1}^{n}{a}_{i}{e}_{{w}_{i}}$ (4)

$s\left({Q}_{i},{K}_{j}\right)=\frac{{Q}_{i}{K}_{j}^{\text{T}}}{\sqrt{{d}_{z}}}=\frac{\left({e}_{{w}_{i}}{W}^{Q}\right){\left({e}_{{w}_{j}}{W}^{K}\right)}^{\text{T}}}{\sqrt{{d}_{z}}}$ (5)

${\alpha }_{ij}=\frac{\mathrm{exp}\left(s\left({Q}_{i},{K}_{j}\right)\right)}{{\sum }_{j=1}^{n}\mathrm{exp}\left(s\left({Q}_{i},{K}_{j}\right)\right)}$ (6)

${z}_{i}={\sum }_{j=1}^{n}{\alpha }_{ij}{V}^{j}={\sum }_{j=1}^{n}{\alpha }_{ij}\left({e}_{w}{}_{j}{W}^{V}\right)$ (7)

Figure 2. Schematic diagram of vector representation of sentences based on self-attention mechanism

$Q=X{W}^{Q}$ (8)

$K=X{W}^{K}$ (9)

$V=X{W}^{V}$ (10)

$Z=\text{Attention}\left(Q,K,V\right)=\text{softmax}\left(s\left(Q,K\right)\right)V$ (11)

3.4. 句子重构层

${p}_{t}=\text{softmax}\left(W\cdot {z}_{s}+b\right)$ (12)

${r}_{s}={T}^{\text{T}}\cdot {p}_{t}$ (13)

3.5. 卷积层与GTRU

${a}_{i}=\text{relu}\left({X}_{i:i+k}\ast {W}_{a}+{V}_{a}{v}_{a}+{b}_{a}\right)$ (14)

${s}_{i}=\mathrm{tanh}\left({X}_{i:i+k}\ast {W}_{s}+{b}_{s}\right)$ (15)

${c}_{i}={s}_{i}×{a}_{i}$ (16)

3.6. 池化层

$c=\left[{c}_{1},{c}_{2},\cdots ,{c}_{n-k+1}\right]$ (17)

$\stackrel{^}{c}=\mathrm{max}\left(c\right)$ (18)

3.7. 情感分类层

$\stackrel{^}{C}=\left[{\stackrel{^}{c}}_{1},{\stackrel{^}{c}}_{2},\cdots ,{\stackrel{^}{c}}_{N}\right]$ (19)

$\stackrel{^}{y}=\text{softmax}\left(W\cdot \stackrel{^}{C}+B\right)$ (20)

3.8. 模型训练

$J\left(\theta \right)={\sum }_{s\in D}{\sum }_{i=1}^{m}\mathrm{max}\left(0,1-{r}_{s}{z}_{s}+{r}_{s}{n}_{i}\right)$ (21)

$U\left(\theta \right)=‖{T}_{n}\cdot {T}_{n}^{\text{T}}-I‖$ (22)

$L\left(\theta \right)=J\left(\theta \right)+\lambda U\left(\theta \right)$ (23)

$\mathcal{L}=-{\sum }_{i}{\sum }_{j}{y}_{i}^{j}\mathrm{log}{\stackrel{^}{y}}_{i}^{j}$ (24)

4. 实验

4.1. 实验数据

Table 2. Data distribution of the datasets

4.2. 实验参数设置

4.3. 要素类提取实验结果与实验分析

Figure 3. Visualization diagram of the representative words of the aspect categories of ABAE model

Figure 4. Visualization diagram of the representative words of the aspect categories of SABAE model

Figure 5. Average accuracy of representative words based on user evaluation

4.4. 要素类情感分类实验结果

Table 3. Accuracy and training time of aspect category sentiment classification under the ablation experiment

5. 总结

Research on Aspect Category Sentiment Classification Based on Gated Convolution Neural Network Combined with Self-Attention Mechanism[J]. 计算机科学与应用, 2020, 10(11): 2064-2077. https://doi.org/10.12677/CSA.2020.1011218

1. 1. Xue, W. and Li, T. (2018) Aspect Based Sentiment Analysis with Gated Convolutional Networks. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 1, 2514-2523. https://doi.org/10.18653/v1/P18-1234

2. 2. 何炎祥, 孙松涛, 牛菲菲, 等. 用于微博情感分析的一种情感语义增强的深度学习模型[J]. 计算机学报, 2017, 40(4): 773-790.

3. 3. Zhang, L., Wang, S. and Liu, B. (2018) Deep Learning for Sentiment Analysis: A Survey. WIREs Data Mining and Knowledge Discovery, 8, e1253. https://doi.org/10.1002/widm.1253

4. 4. LeCun, Y., Bengio, Y. and Hinton, G. (2015) Deep Learning. Nature, 521, 436-444. https://doi.org/10.1038/nature14539

5. 5. Socher, R., Perelygin, A., Wu, J., et al. (2013) Recursive Deep Models for Semantic Compositionality over a Sentiment Treebank. Empirical Methods in Natural Language Processing, 1631-1642.

6. 6. Kim, Y. (2014) Convolutional Neural Networks for Sentence Classification. Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP), Doha, October 2014, 1746-1751. https://doi.org/10.3115/v1/D14-1181

7. 7. Wang, X., Liu, Y., Sun, C., et al. (2015) Predicting Polarities of Tweets by Composing Word Embeddings with Long Short-Term Memory. Proceedings of the 53rd Annual Meeting of the Asso-ciation for Computational Linguistics and the 7th International Joint Conference on Natural Language Processing, 1, 1343-1353. https://doi.org/10.3115/v1/P15-1130

8. 8. Mnih, V., Heess, N., Graves, A., et al. (2014) Recurrent Models of Vis-ual Attention. arXiv: Learning.

9. 9. Zhang, L. and Liu, B. (2014) Aspect and Entity Extraction for Opinion Mining. In: Chu, W., Ed., Data Mining and Knowledge Discovery for Big Data. Studies in Big Data, Vol. 1. Springer, Berlin, Hei-delberg. https://doi.org/10.1007/978-3-642-40837-3_1

10. 10. Mimno, D., Wallach, H., Talley, E.M., et al. (2011) Optimizing Semantic Coherence in Topic Models. Proceedings of the 2011 Conference on Empirical Methods in Natural Language Proceedings, Edinburgh, 27-31 July, 262-272.

11. 11. Yan, X., Guo, J., Lan, Y., et al. (2013) A Biterm Topic Model for Short Texts. Proceedings of the 22nd International Conference on World Wide Web, May 2013, 1445-1456. https://doi.org/10.1145/2488388.2488514

12. 12. Wang, L., Liu, K., Cao, Z., et al. (2015) Sentiment-Aspect Extrac-tion Based on Restricted Boltzmann Machines. Proceedings of the 53rd Annual Meeting of the Association for Computa-tional Linguistics and the 7th International Joint Conference on Natural Language Processing, Beijing, 26-31 July, 616-625.

13. 13. He, R., Lee, W.S., Ng, H.T., et al. (2017) An Unsupervised Neural Attention Model for Aspect Extrac-tion. Proceedings of the 55th Annual Meeting of the Association for Computational Linguistics, 1, 388-397. https://doi.org/10.18653/v1/P17-1036

14. 14. Wilson, T., Wiebe, J. and Hoﬀmann, P. (2015) Recognizing Contextual Polarity in Phrase-Level Sentiment Analysis. International Journal of Computer Applications, 7, 347-354.

15. 15. Wu, Y., Wang, M. and Jin, P. (2008) Disambiguating Sentiment Ambiguous Adjectives. International Conference on Natural Language Processing and Knowledge Engineering, Beijing, 19-22 October 2008, 1-8. https://doi.org/10.1109/NLPKE.2008.4906816

16. 16. Tang, D., Qin, B., Feng, X., et al. (2015) Effective LSTMs for Target-Dependent Sentiment Classification. Proceedings of COLING 2016, the 26th International Conference on Com-putational Linguistics: Technical Paper, Osaka, 11-17 December 2017, 3298-3307.

17. 17. Wang, Y., Huang, M., Zhu, X., et al. (2016) Attention-Based LSTM for Aspect-Level Sentiment Classification. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, November 2016, 606-615. https://doi.org/10.18653/v1/D16-1058

18. 18. Liu, J. and Zhang, Y. (2017) Attention Modeling for Targeted Senti-ment. Proceedings of the 15th Conference of the European Chapter of the Association for Computational Linguistics, Valencia, April 2017, 572-577. https://doi.org/10.18653/v1/E17-2091

19. 19. Tang, D., Qin, B. and Liu, T. (2016) Aspect Level Sentiment Classifica-tion with Deep Memory Network. Proceedings of the 2016 Conference on Empirical Methods in Natural Language Processing, Austin, November 2016, 214-224. https://doi.org/10.18653/v1/D16-1021

20. 20. Ma, D., Li, S., Zhang, X., et al. (2017) Interactive Attention Networks for Aspect-Level Sentiment Classification. Proceedings of the 26th International Joint Conference on Artificial Intelli-gence, 4068-4074. https://doi.org/10.24963/ijcai.2017/568

21. 21. Chen, P., Sun, Z., Bing, L., et al. (2017) Recurrent Attention Network on Memory for Aspect Sentiment Analysis. Proceedings of the 2017 Conference on Empirical Methods in Natural Lan-guage Processing, Copenhagen, September 2017, 452-461. https://doi.org/10.18653/v1/D17-1047

22. 22. Li, X., Bing, L., Lam, W., et al. (2018) Transformation Networks for Target-Oriented Sentiment Classification. Proceedings of the 56th Annual Meeting of the Association for Computational Linguistics, 1, 946-956. https://doi.org/10.18653/v1/P18-1087

23. 23. Tang, J., Lu, Z., Su, J., et al. (2019) Progressive Self-Supervised Atten-tion Learning for Aspect-Level Sentiment Analysis. Proceedings of the 57th Annual Meeting of the Association for Computational Linguistics, Florence, July 2019, 557-566. https://doi.org/10.18653/v1/P19-1053

24. 24. Chen, Z., Mukherjee, A., Liu, B., et al. (2014) Aspect Extraction with Automated Prior Knowledge Learning. Proceedings of the 52nd Annual Meeting of the Association for Computational Linguistics, 1, 347-358. https://doi.org/10.3115/v1/P14-1033

25. 25. Ren, S., He, K., Girshick, R., et al. (2015) Faster R-CNN: Towards Re-al-Time Object Detection with Region Proposal Networks. Neural Information Processing Systems, 91-99.