Application of an Improved Convolutional Neural Network Algorithm in Text Classification
DOI:
https://doi.org/10.13052/jwe1540-9589.2331Keywords:
Text classification, convolutional neural network, support vector machine, attention mechanismAbstract
This paper proposes a text classification model based on a combination of a convolutional neural network (CNN) and a support vector machine (SVM) using Amazon review polarity, TREC, and Kaggle as experimental data. By adding an attention mechanism to simplify the parameters and using the classifier based on SVM to replace the Softmax layer, the extraction effect of feature words is improved and the problem of weak generalization ability of the CNN model is solved. Simulation experiments show that the proposed algorithm performs better in precision rate, recall rate, F1 value, and training time compared with CNN, RNN, BERT and term frequency-inverse document frequency (TF-IDF).
Downloads
References
D. W. Otter, J. R. Medina, J. K. Kalita, “A survey of the usages of deep learning for natural language processing” IEEE Transactions on Neural Networks and Learning Systems, vol. 32, no. 2, pp. 604–624, 2021.
H. Azarbonyad, M. Dehghani, M. Marx, et al., “Learning to rank for multi-label text classification: Combining different sources of information”, Natural Language Engineering, vol. 27, no. 1, pp. 89–111, 2021.
W. F. Liu, J. M. Pang, N. Li, et. al., “Research on multi-label text classification method based on tALBERT-CNN”, International Journal of Computational Intelligence Systems, vol. 14, no. 1, 2022.
K. M. Schneider, “A new feature selection score for multinomial naive bayes text classification based on kldivergence,” Proceedings of the ACL Interactive Poster and Demonstration Sessions, 2004, pp. 186-189.
A. Z. Guo, Y. Tao, “Based on rough sets and the associated analysis of KNN text classification research”, International Symposium Distributed Computing and Applications Business Engineering and Science, 2015, pp. 485–488.
Y. Chali, S. A. Hasan, “Query-focused multi-document summarization: automatic data annotations and supervised learning approaches”, Natural Language Engineering, vol. 18, pp. 109–145, 2012.
G. G. Chowdhury, “Natural language processing. Annual review of information science and technology”, vol. 37, no. 1, pp. 51–89, 2003.
Q. Li, H. Peng, J. Li, et al., “A text classification survey: from shallow to deep learning”, arXiv preprint arXiv:2008.00364, 2020.
C. Cortes, V. Vapnik, “Supportvector networks”, Machine learning, vol. 20, no. 3, pp. 273-297, 1995.
T. Joachims, “Text categorization with support vector machines: Learning with many relevant features”, European Conference on Machine Learning, 1998, pp. 137-142.
J. H. Friedman, “Stochastic gradient boosting”, Computational Statistics and Data Analysis, vol. 38, no. 4, pp. 367-378, 2002.
H. Wu, Y. Liu, J. Wang, “Review of text classification methods on deep learning. Computers Materials and Continua, vol. 63, pp. 1309-1321, 2020.
O. Zennaki, N. Semmar, L. Besacier, “A neural approach for inducing multilingual resources and natural language processing tools for low-resource languages”, Natural Language Engineering, vol. 25, no. 1, pp. 43–67, 2019.
G. Arevian, “Recurrent neural networks for robust realworld text classification”, IEEE/WIC/ACM International Conference on Web Intelligence (WI’07). IEEE, 2007, pp. 326-329.
S. Hochreiter, J. Schmidhuber, “Long shortterm memory”, Neural Computation, vol. 9, no. 8, pp. 1735-1780, 1997.
X. S. Yu, S. Shen, P. Chen, et al., “Electronic medical record classification method based on LSTM of text word features dimensionality reduction”, IEEE International Conference on Software Quality, Reliability and Security Companion, 2021, pp. 365–371.
A. Vaswani, N. Shazeer, N. Parmar, et al., “Attention is all you need”, Advances in Neural Information Processing Systems, 2017, 30.
W. Q. Gao, H. Huang, “A gating context-aware text classification model with BERT and graph convolutional networks”, Journal of Intelligent and Fuzzy Systems, 40(3), 4331–4343, 2021.
N. Kalchbrenner, E. Grefenstette, P. Blunsom, “A convolutional neural network for modelling sentences”, Annual meeting of the Association for Computational Linguistics, 2014.
Y. Chen, “Convolutional neural network for sentence classification”, University of Waterloo, 2015.
W. J. Tao, D. Chang, “News text classification based on an improved convolutional neural network”, Tehnicki Vjesnik-Technical Gazette, vol. 26, no. 5, pp. 1400–1409, 2019.
J. Peng, S. Q. Huo, “Few-shot text classification method based on feature optimization”, Journal of Web Engineering, vol. 22, no. 3, pp. 497–514, 2023.
K.V. Raju, M. Sridhar, “Review-based sentiment prediction of rating using natural language processing sentence-level sentiment analysis with bag-of-words approach”, First International Conference on Sustainable Technologies for Computation Intelligence, 2020, pp. 807–821, 2020.
N. I. Widiastuti, “Convolution neural network for text mining and natural language processing”, 2nd International Conference on Informatics, Engineering, Science, and Technology, 2019.
I. Ali, N. Mughal, Z. H. Khand, et. al., “Resume classification system using natural language processing and machine learning techniques”, Mehran University Research Journal of Engineering and Technology, vol. 41, no. 1, pp. 65–79, 2022.
Y. L. Gong, N. N. Lu, J. J. Zhang, “Application of deep learning fusion algorithm in natural language processing in emotional semantic analysis”, Concurrency and Computation-Practice and Experience, vol. 31, no. 10, 2019.
Y. Li, X. J. Yang, M. Zuo, et. al., “Deep structured learning for natural language processing”, ACM Transactions on Asian and Low-resource Language Information Processing, vol. 20, no. 3, 2021.
M. Madeja, J. Poruban, “Unit under test identification using natural language processing techniques”, Open Computer Science, vol. 11, no. 1, pp. 22–32, 2021.
