Diagnostic Classifiers for Explaining a Neural Model with Hierarchical Attention for Aspect-based Sentiment Classification
DOI:
https://doi.org/10.13052/jwe1540-9589.2218Keywords:
Aspect-based sentiment classification, neural rotatory attention model, diagnostic classificationAbstract
The current models proposed for aspect-based sentiment classification (ABSC) are mainly developed with the purpose of providing high rates of accuracy, regardless of the inner working which is usually difficult to understand. Considering the state-of-art model LCR-Rot-hop++ for ABSC, we use diagnostic classifiers to gain insights into the encoded information of each layer. Starting from a set of various hypotheses, we test how sentiment-related information is captured by different layers of the model. Given the model architecture, information about the related words to the target is easily extracted. Also, the model is able to detect to some extent information about the sentiments of the words and, in particular, sentiments of the words related to the target. However, the model is less effective in extracting the aspect mentions associated with a word and the general structure of the sentence.
Downloads
References
Y. Adi, E. Kermany, Y. Belinkov, O. Lavi, and Y. Goldberg. Fine-grained analysis of sentence embeddings using auxiliary prediction tasks. In 2017 International Conference on Learning Representations (ICLR 2017), 2016.
A. Barbalau, A. Cosma, R. T. Ionescu, and M. Popescu. A generic and model-agnostic exemplar synthetization framework for explainable AI. In 31st European Conference on Machine Learning and Principles and Practice of Knowledge Discovery in Databases (ECML-PKDD 2020), volume 12458 of LNCS, pages 190–205. Springer, 2020.
Y. Belinkov, N. Durrani, F. Dalvi, H. Sajjad, and J. R. Glass. What do neural machine translation models learn about morphology? In 55th Annual Meeting of the Association for Computational Linguistics (ACL 2017), pages 861–872. ACL, 2017.
S. Bird, E. Klein, and E. Loper. Natural Language Processing with Python, Analyzing Text with the Natural Language Toolkit, volume 44. O’Reilly, 2010.
G. Brauwers and F. Frasincar. A survey on aspect-based sentiment classification. ACM Computing Surveys, 2021.
N. Chawla, K. Bowyer, L. Hall, and W. Kegelmeyer. SMOTE: Synthetic minority over-sampling technique. Journal of Artificial Intelligence Research, 16:321–357, 2002.
D. Chen and C. D. Manning. A fast and accurate dependency parser using neural networks. In 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP 2014), pages 740–750. ACL, 2014.
G. Chrupała and A. Alishahi. Correlating neural and symbolic representations of language. In 57th Annual Meeting of the Association for Computational Linguistics (ACL 2019), pages 2952–2962. ACL, 2019.
J. Devlin, M.-W. Chang, K. Lee, and K. Toutanova. BERT: Pre-training of deep bidirectional transformers for language understanding. In 2009 Conference of the North American Chapter of the Association of Computational Linguistics: Human Language Techniques (NAACL-HLT 2019), pages 4171–4186, 2019.
K. Geed, F. Frasincar, and M. M. Truşcǎ. Explaining a deep neural model with hierarchical attention for aspect-based sentiment classification using diagnostic classifiers. In 22nd International Conference on Web Engineering (ICWE 2022), volume 13362 of LNCS, pages 268–282. Springer, 2022.
S. Grimm, A. Abecker, J. Völker, and R. Studer. Ontologies and the Semantic Web, pages 507–579. 2011.
S. Hochreiter and J. Schmidhuber. Long short-term memory. Neural Computation, 9(8):1735–1780, 1997.
D. Hupkes and W. Zuidema. Visualisation and ‘diagnostic classifiers’ reveal how recurrent and recursive neural networks process hierarchical structure (extended abstract). In 27th International Joint Conference on Artificial Intelligence (IJCAI 2018), pages 5617–5621. International Joint Conferences on Artificial Intelligence Organization, 2018.
J. Jumelet and D. Hupkes. Do language models understand anything? On the ability of LSTMs to understand negative polarity items. In 2018 EMNLP Workshop: Analyzing and Interpreting Neural Networks for NLP (BlackBox NLP 2019), pages 222–231. ACL, 2018.
T. Kenter and M. de Rijke. Short text similarity with word embeddings. In 24th ACM International on Conference on Information and Knowledge Management (CIKM 2015), pages 1411–1420. ACM, 2015.
S. Kiritchenko, X. Zhu, C. Cherry, and S. Mohammad. NRC-Canada-2014: Detecting aspects and sentiment in customer reviews. In 8th International Workshop on Semantic Evaluation (SemEval 2014), pages 437–442. ACL, 2014.
B. Liu. Sentiment Analysis: Mining Opinions, Sentiments, and Emotions. Cambridge University Press, 2 edition, 2020.
C. D. Manning, M. Surdeanu, J. Bauer, J. R. Finkel, S. Bethard, and D. McClosky. The stanford CoreNLP natural language processing toolkit. In 52nd Annual Meeting of the Association for Computational Linguistics (ACL 2014), pages 55–60. ACL, 2014.
L. Meijer, F. Frasincar, and M. M. Truşcă. Explaining a neural attention model for aspect-based sentiment classification using diagnostic classification. In 36th Annual ACM Symposium on Applied Computing, SAC 2021, pages 821–827. ACM, 2021.
A. Menditto, M. Patriarca, and B. Magnusson. Understanding the meaning of accuracy, trueness and precision. Accreditation and Quality Assurance, 12:45–47, 2007.
A. More. Survey of resampling techniques for improving classification performance in unbalanced datasets. arXiv preprint arXiv:1608.06048.
M. S. Mubarok, K. Adiwijaya, and M. D. Aldhi. Aspect-based sentiment analysis to review products using naïve bayes. In AIP Conference Proceedings, volume 1867, page 020060. AIP Publishing, 2017.
F. Pedregosa, G. Varoquaux, A. Gramfort, V. Michel, B. Thirion, O. Grisel, M. Blondel, P. Prettenhofer, R. Weiss, V. Dubourg, J. Vanderplas, A. Passos, D. Cournapeau, M. Brucher, M. Perrot, and E. Duchesnay. Scikit-learn: Machine learning in Python. Journal of Machine Learning Research, 12:2825–2830, 2011.
M. Pontiki, D. Galanis, H. Papageorgiou, I. Androutsopoulos, S. Manandhar, M. AL-Smadi, M. Al-Ayyoub, Y. Zhao, B. Qin, O. De Clercq, V. Hoste, M. Apidianaki, X. Tannier, N. Loukachevitch, E. Kotelnikov, N. Bel, S. M. Jiménez-Zafra, and G. Eryiğit. SemEval-2016 task 5: Aspect based sentiment analysis. In 10th International Workshop on Semantic Evaluation (SemEval 2016), pages 19–30. ACL, 2016.
K. Schouten and F. Frasincar. Survey on aspect-level sentiment analysis. IEEE Transactions on Knowledge and Data Engineering, 28(3):813–830, 2016.
K. Schouten and F. Frasincar. Ontology-driven sentiment analysis of product and service aspects. In 15th Extended Semantic Web Conference (ESWC 2018), volume 10843 of LNCS, pages 608–623. Springer, 2018.
Z. Shiliang and R. Xia. Left-center-right separated neural network for aspect-based sentiment analysis with rotatory attention. arXiv preprint arXiv:1802.00892, 2018.
M. M. Truşcǎ, D. Wassenberg, F. Frasincar, and R. Dekker. A hybrid approach for aspect-based sentiment analysis using deep contextual word embeddings and hierarchical attention. In 20th International Conference on Web Engineering (ICWE 2020), volume 12128 of LNCS, pages 365–380. Springer, 2020.
O. Wallaart and F. Frasincar. A hybrid approach for aspect-based sentiment analysis using a lexicalized domain ontology and attentional neural models. In 16th Extended Semantic Web Conference (ESWC 2019), volume 11503 of LNCS, pages 363–378. Springer, 2019.
S. Yanmin, A. Wong, and M. S. Kamel. Classification of imbalanced data: A review. International Journal of Pattern Recognition and Artificial Intelligence, 23, 2011.
Z. Zhang, Y. Wu, H. Zhao, Z. Li, S. Zhang, X. Zhou, and X. Zhou. Semantics-aware BERT for language understanding. In 34th AAAI Conference on Artificial Intelligence (AAAI 2021), pages 687–719. AAAI Press, 2020.
