Enhancing Suggestion Detection in Online User Reviews through Integrated Information Retrieval and Deep Learning Approaches
DOI:
https://doi.org/10.13052/jwe1540-9589.2335Keywords:
User generated content analysis, suggestion mining, information retrieval, information extraction, text mining, sentiment analysis, deep learningAbstract
In the aftermath of the COVID-19 pandemic, using web platforms as a communication medium and decision-making tool in online commerce has become widely acknowledged. User-generated comments, reflecting positive and negative sentiments towards specific items, serve as invaluable indicators, offering recommendations for product and organizational improvements. Consequently, the extraction of suggestions from mined opinions can enhance the efficacy of companies and organizations in this domain. Prevailing research in suggestion mining predominantly employs rule-based methodologies and statistical classifiers, relying on manually identified features. However, a recent trend has emerged wherein researchers explore solutions grounded in deep learning tools and techniques. This study aims to employ information retrieval techniques for the automated identification of suggestions. To this end, various methodologies, including distance measurement approaches, multilayer perceptron neural networks, support vector machines, regression logistics, convolutional neural networks utilizing TF-IDF, Bag of Words (BOW), and Word2Vec vectors, along with keyword extraction, have been integrated. The proposed approach is assessed using the SemEval2019 dataset to extract suggestions from the textual content of online user reviews. The obtained results demonstrate a notable enhancement in the F1 score, reaching 0.76 compared to prior research. The experiments further suggest that information retrieval-based approaches exhibit promising potential for this specific task.
Downloads
References
Christopher M Bishop. Pattern Recognition and Machine Learning. Springer, 2006.
Caroline Brun and Caroline Hagege. Suggestion mining: Detecting suggestions for improvement in users’ comments. Res. Comput. Sci., 70(79):171–181, 2013.
Jacob Devlin, Ming-Wei Chang, Kenton Lee, and Kristina Toutanova. Bert: Pre-training of deep bidirectional transformers for language understanding. arXiv preprint arXiv:1810.04805, 2018.
Yunxia Ding, Xiaobing Zhou, and Xuejie Zhang. Ynu_dyx at semeval-2019 task 9: A stacked bilstm for suggestion mining classification. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 1272–1276, 2019.
Aysu Ezen-Can and Ethem F Can. Hybrid rnn at semeval-2019 task 9: blending information sources for domain-independent suggestion mining. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 1199–1203, 2019.
Tirana Fatyanosa, Al Hafiz Akbar Maulana Siagian, and Masayoshi Aritsugi. DBMS-KU at SemEval-2019 task 9: Exploring machine learning approaches in classifying text as suggestion or non-suggestion. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 1185–1191, Minneapolis, Minnesota, USA, June 2019. Association for Computational Linguistics.
Jerome H Friedman. Greedy function approximation: A gradient boosting machine. Annals of statistics, pages 1189–1232, 2001.
Hitesh Golchha, Deepak Gupta, Asif Ekbal, and Pushpak Bhattacharyya. Helping each other: a framework for customer-to-customer suggestion mining using a semi-supervised deep neural network. arXiv preprint arXiv:1811.00379, 2018.
Andrew B Goldberg, Nathanael Fillmore, David Andrzejewski, Zhiting Xu, Bryan Gibson, and Xiaojin Zhu. May all your wishes come true: A study of wishes and how to recognize them. In Proceedings of Human Language Technologies: The 2009 Annual Conference of the North American Chapter of the Association for Computational Linguistics, pages 263–271, 2009.
Ian Goodfellow, Yoshua Bengio, and Aaron Courville. Deep Learning. MIT Press, 2016.
Claudia Iacob and Rachel Harrison. Retrieving and analyzing mobile apps feature requests from online reviews. In 2013 10th working conference on mining software repositories (MSR), pages 41–44. IEEE, 2013.
Harsh Jhamtani, Niyati Chhaya, Shweta Karwa, Devesh Varshney, Deepam Kedia, and Vineet Gupta. Identifying suggestions for improvement of product features from online product reviews. In International Conference on Social Informatics, pages 112–119. Springer, 2015.
Rosie Jones and Fuchun Peng. Web Search Query Rewriting, pages 3491–3493. Springer US, Boston, MA, 2009.
Armand Joulin, Edouard Grave, Piotr Bojanowski, and Tomas Mikolov. Bag of Tricks for Efficient Text Classification. 2016. Accessed: April 2024.
Navid Khaledian, Amin Nazari, Keyhan Khamforoosh, Laith Abualigah, and Danial Javaheri. Trustdl: Use of trust-based dictionary learning to facilitate recommendation in social networks. Expert Systems with Applications, 228:120487, 2023.
Mojtaba Kordabadi, Amin Nazari, and Muharram Mansoorizadeh. A movie recommender system based on topic modeling using machine learning methods. International Journal of Web Research, 5(2):19–28, 2022.
Naveen Kumar Laskari and Suresh Kumar Sanampudi. Explainable system for suggestion mining using attention. In 2022 8th International Conference on Advanced Computing and Communication Systems (ICACCS), volume 1, pages 679–684. IEEE, 2022.
Naveen Kumar Laskari and Suresh Kumar Sanampudi. Aspect-oriented suggestion mining from opinion reviews. Journal of Theoretical and Applied Information Technology, 101(12), 2023.
Tomas Mikolov, Ilya Sutskever, Kai Chen, Greg Corrado, and Jeff Dean. Distributed Representations of Words and Phrases and their Compositionality. 2013. Accessed: April 2024.
Sapna Negi, Kartik Asooja, Shubham Mehrotra, and Paul Buitelaar. A study of suggestions in opinionated texts and their automatic detection. In Proceedings of the Fifth Joint Conference on Lexical and Computational Semantics, pages 170–178, 2016.
Sapna Negi and P Buitelaar. Suggestion mining from opinionated text. Sentiment Analysis in Social Networks, pages 129–139, 2017.
Sapna Negi and Paul Buitelaar. Towards the extraction of customer-to-customer suggestions from reviews. In Proceedings of the 2015 Conference on Empirical Methods in Natural Language Processing, pages 2159–2167, 2015.
Sapna Negi, Tobias Daudert, and Paul Buitelaar. Semeval-2019 task 9: Suggestion mining from online reviews and forums. In Proceedings of the 13th international workshop on semantic evaluation, pages 877–887, 2019.
Thi-Lan Ngo, Khac Linh Pham, Hideaki Takeda, Son Bao Pham, and Xuan Hieu Phan. On the identification of suggestion intents from vietnamese conversational texts. In Proceedings of the 8th International Symposium on Information and Communication Technology, pages 417–424, 2017.
Samuel Pecar, Marian Simko, and Maria Bielikova. Nl-fiit at semeval-2019 task 9: Neural model ensemble for suggestion mining. arXiv preprint arXiv:1904.02981, 2019.
Sai Prasanna and Sri Ananda Seelan. Zoho at semeval-2019 task 9: Semi-supervised domain adaptation using tri-training for suggestion mining. arXiv preprint arXiv:1902.10623, 2019.
A Ramesh, K Pradeep Reddy, M Sreenivas, and Para Upendar. Feature selection technique-based approach for suggestion mining. In Evolution in Computational Intelligence: Proceedings of the 9th International Conference on Frontiers in Intelligent Computing: Theory and Applications (FICTA 2021), pages 541–549. Springer, 2022.
T Raghunadha Reddy, P Vijayapal Reddy, T Murali Mohan, and Raju Dara. An approach for suggestion mining based on deep learning techniques. In IOP Conference Series: Materials Science and Engineering, volume 1074, pages 1–10. IOP Publishing, 2021.
Radim Rehurek and Petr Sojka. Gensim: Topic modelling for humans. https://radimrehurek.com/gensim/. Accessed: April 2024.
Hinrich Schütze, Christopher D Manning, and Prabhakar Raghavan. Introduction to information retrieval, volume 39. Cambridge University Press Cambridge, 2008.
Hassan Shahmohammadi, MirHossein Dezfoulian, and Muharram Mansoorizadeh. Paraphrase detection using lstm networks and handcrafted features. Multimedia Tools and Applications, 80(4):6479–6492, 2021.
Jie Wu, Tong Yang, Zhiwei Zhou, and Narisa Zhao. Consumers’ affective needs matter: Open innovation through mining luxury hotels’ online reviews. International Journal of Hospitality Management, 114:103556, 2023.
Ping Yue, Jin Wang, and Xuejie Zhang. Ynu-hpcc at semeval-2019 task 9: using a bert and cnn-bilstm-gru model for suggestion mining. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 1277–1281, 2019.
Min Zhang, Lin Sun, Yuzhuo Li, G Alan Wang, and Zhen He. Using supplementary reviews to improve customer requirement identification and product design development. Journal of Management Science and Engineering, 2023.
Hua Zhao, PeiXin Zhang, Yue Liang, and Sitenda Amos. A comparative study on chinese open domain suggestion mining. Journal of Intelligent & Fuzzy Systems, 42(6):5385–5398, 2022.
Qimin Zhou, Zhengxin Zhang, Hao Wu, and Linmao Wang. Zqm at semeval-2019 task9: a single layer cnn based on pre-trained model for suggestion mining. In Proceedings of the 13th International Workshop on Semantic Evaluation, pages 1287–1291, 2019.
