A cross-language short text classification model based on BERT and multilayer collaborative convolutional neural network (MCNN)
Abstract
This study focuses on cross-lingual short text classification tasks and aims to combine the advantages of BERT and Multi-layer Collaborative Convolutional Neural Network (MCNN) to build an efficient classification model. BERT model provides rich semantic information for text classification with its powerful language understanding and bidirectional context modeling ability, while MCNN effectively extracts local and global features in text through multi-layer convolution structure and collaborative working mechanism. In this study, the output of BERT is used as the input of MCNN, and MCNN is used to further mine the deep features in the text, so as to realize the high-precision classification of cross-lingual short text. The experimental results show that the model has achieved significant performance improvement on the dataset, which provides a new effective solution for cross-lingual short text classification tasks.
References
1. Luhn H P. The Automatic Creation of Literature Abstracts. Ibm Journal of Research and Development, 1958, 2(2): 159-165.
2. Maron M E，Kuhns J L. On Relevance, Probabilistic Indexing and Information Retrieval. Journal of the ACM, 1960, 7(3): 216-244.
3. Akhter M P, Zheng J, Naqvi I R, et al. Document-Level Text Classification Using Single-Layer Multisize Filters Convolutional Neural Network. IEEE Access, 2020, 8: 42689-42707.
4. Deng J, Cheng L, Wang Z. Attention-based BiLSTM fused CNN with gating mechanism model for Chinese long text classification. Computer Speech And Language, 2021, 68: 101182.
5. Jin Y, Zhu Q, Deng X. Weighted hierarchy mechanism over BERT for long text classifification//International Conference on Artificial Intelli‐ gence and Security, 2021: 566-574.
6. Szegedy C, Liu W, Jia YQ, et al. Going deeper with convolutions //Proceedings of the IEEE Conference on Computer Vision and Pat‐tern Recognition, 2015: 1-9.
7. Xu P, Luo Z X, Huang XK. Research on sentiment analysis of product reviews based on Bert BiLSTM. Intelligent Computer and Applications, 2022, 12(11): 186-191.
8. Hao W, Jian W, Dongliang N, et al.Transmission line fault cause identification method based on transient waveform image and MCNN-LSTM. Measurement, 2023, 220
9. Nan Z, Lin-Shuang Z. Method for real-time prediction of cutter wear during shield tunnelling: A new wear rate index and MCNN-GRU. MethodsX, 2023, 10102017-102017.
10. Wang A, Qi Y, Baiyila D .C-BERT: A Mongolian reverse dictionary based on fused lexical semantic clustering and BERT.Alexandria Engineering Journal, 2025, 111385-395.
11. Nahali S, Safari L, Khanteymoori A, et al.StructmRNA a BERT based model with dual level and conditional masking for mRNA representation. Scientific Reports, 2024, 14(1): 26043-26043.
12. Darraz N, Karabila I, Ansari E A, et al.Integrated sentiment analysis with BERT for enhanced hybrid recommendation systems. Expert Systems With Applications, 2025, 261125533-125533.
13. Murthy D, Keshari S, Arora S, et al.Categorizing E-cigarette-related tweets using BERT topic modeling.Emerging Trends in Drugs, Addictions, and Health, 2024, 4100160-100160.
14. Nouri A, Hossain S M. CoRBS: a dynamic storytelling algorithm using a novel contextualization approach for documents utilizing BERT features.Knowledge and Information Systems, 2024, (prepublish): 1-36.
15. Ullah F, Gelbukh A, Zamir T M, et al.Enhancement of Named Entity Recognition in Low-Resource Languages with Data Augmentation and BERT Models: A Case Study on Urdu.Computers, 2024, 13(10): 258-258.
16. Powroznik P, Paszkowska S M, Rejdak R, et al.Automatic Method of Macular Diseases Detection Using Deep CNN-GRU Network in OCT Images. Acta Mechanica et Automatica, 2024, 18(4): 197-206.
17. Chuquimarca E L, Vintimilla X B, Velastin A S. A review of external quality inspection for fruit grading using CNN models. Artificial Intelligence in Agriculture, 2024, 141-20.
18. Xin Y, Z, Zhong Z, et al.Lateral spread prediction based on hybrid CNN-LSTM model for hot strip finishing mill. Materials Letters, 2025, 378137594-137594.
19. Li M, Zhou Q, Han X, et al. Prediction of reference crop evapotranspiration based on improved convolutional neural network (CNN) and long short-term memory network (LSTM) models in Northeast China. Journal of Hydrology, 2024, 645(PA): 132223-132223.
20. Cui X, Zhu J, Jia L, et al. A novel heat load prediction model of district heating system based on hybrid whale optimization algorithm (WOA) and CNN-LSTM with attention mechanism.Energy, 2024, 312133536-133536.
21. Yang F, Wang B. Dual Channel‐Spatial Self‐Attention Transformer and CNN synergy network for 3D medical image segmentation. Applied Soft Computing, 2024, 167(PB): 112255-112255.
22. Nazir I M, Akter A, Wadud H A M, et al. Utilizing customized CNN for brain tumor prediction with explainable AI. Heliyon, 2024, 10(20): e38997-e38997.
23. Bai X, Wan Y, Wang W. CEPDNet: a fast CNN-based image denoising network using edge computing platform. The Journal of Supercomputing, 2024, 81(1): 100-100.
24. Rajasekaran V, Tamilselvan L. A hybrid model for detecting e-commerce product returns using CNN-LSTM. Multimedia Tools and Applications, 2024, (prepublish): 1-13.
25. Çağatay BerkeErdaş, EmreSümer. CNN‐Based Neurodegenerative Disease Classification Using QR‐Represented Gait Data. Brain and Behavior, 2024, 14(10): e70100-e70100.
Copyright (c) 2024 Qiong Hu
This work is licensed under a Creative Commons Attribution 4.0 International License.
Copyright on all articles published in this journal is retained by the author(s), while the author(s) grant the publisher as the original publisher to publish the article.
Articles published in this journal are licensed under a Creative Commons Attribution 4.0 International, which means they can be shared, adapted and distributed provided that the original published version is cited.
Submit a Paper
