Multi-Rhythm Capsule Network Recognition Structure for Motor Imagery Classification


  • Meiyan Xu Xiamen University, China and Minnan Normal University, China
  • Junfeng Yao Xiamen University, China
  • Yifeng Zheng Minnan Normal University, China
  • Yaojin Lin Minnan Normal University, China



Capsule network, deep learning, brain machine interface, motor imagery, classification


Existing machine learning methods for classification and recognition of EEG motor imagery usually suffer from reduced accuracy for limited training data. To address this problem, this paper proposes a multi-rhythm capsule network (FBCapsNet) that uses as little EEG information as possible with key features to classify motor imagery and further improves the classification efficiency. The network conforms to a small recognition model with only 3 acquisition channels but it can effectively use the limited data for feature learning. Based on the BCI Competition IV 2b data set, experimental results show that the proposed network can achieve 2.41% better performance than existing cutting-edge methods.


Download data is not yet available.

Author Biographies

Meiyan Xu, Xiamen University, China and Minnan Normal University, China

Meiyan Xu received her B. A. and M. A. degrees in Mathematics and Applied Mathematics and Software Engineering from the Xiamen University, China in 2006 and 2010. Currently, she is a PhD at the software school of Xiamen University between 2016 and 2020, China. In 2021, he joined the faculty of School of Computer Science, Minnan Normal University, Zhangzhou, China. Her research interests include data analytics, data mining, machine learning, and brain computer interfaces.

Junfeng Yao, Xiamen University, China

Junfeng Yao received his Ph.D. degree in thermal engineering from the Central South University, China in 2001. He conducted his post-doctoral research work in Tsinghua University in the area of Electrical Simulation and Controlling from 2001 to 2003. He was a visiting scholar in Southern Polytechnic State University, USA from 2009 to 2010 and in University of Washington, USA from 2016 to 2017. He is now a professor at the software school of Xiamen University, China. His research interests are wide-reaching but mainly involve the areas of machine learning, artificial intelligence and computer graphics.

Yifeng Zheng, Minnan Normal University, China

Yifeng Zheng received the B.E. degree in computer science and technology from Minnan Normal University, Zhangzhou, China, in 2004, and received the M.E. degree and PhD degree in computer technology from China University of Petroleum-Beijing, Bejing, China, in 2016 and in 2020. In 2004, he joined the faculty of School of Computer Science, Minnan Normal University, Zhangzhou, China. His research interests include artificial intelligence, machine learning, deep learning and network communications.

Yaojin Lin, Minnan Normal University, China

Yaojin Lin received the Ph.D. degree in School of Computer and Information from Hefei University of Technology. He currently is a professor with Minnan Normal University. His research interests include data mining, and granular computing. He has published more than 80 papers in many conferences and journals, such as IJCAI, CVPR, TKDD, PR and so on.


H. Cecotti, M. P. Eckstein, and B. Giesbrecht, “Single-trial classification of event-related potentials in rapid serial visual presentation tasks using supervised spatial fltering,” IEEE Transactions on Neural Networks and Learning Systems, vol. 25, no. 11, pp. 2030–2042, 2014.

X. Zhang and D. Wu, “On the vulnerability of cnn classifiers in eegbased bcis,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 27, no. 5, pp. 814–825, 2019.

R. T. Schirrmeister, J. T. Springenberg, L. D. J. Fiederer, M. Glasstetter, K. Eggensperger, M. Tangermann, F. Hutter, W. Burgard, and T. Ball, “Deep learning with convolutional neural networks for eeg decoding and visualization,” Human Brain Mapping, vol. 38, no. 11, pp. 5391–5420, 2017.

Hongzhi, Y. Xue, L. Xu, Y. Cao, and X. Jiao, “A speedy calibration method using riemannian geometry measurement and other-subject samples on a p300 speller,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 26, no. 3, pp. 602–608, 2018.

N.-S. Kwak, K.-R. Müller, and S.-W. Lee, “A convolutional neural network for steady state visual evoked potential classification under ambulatory environment,” PLoS one, vol. 12, no. 2, p. e0172578, 2017.

Y. R. Tabar and Ugur Halici, “A novel deep learning approach for classification of eeg motor imagery signals,” Journal of Neural Engineering, vol. 14, no. 1, p. 016003, 2016.

Y. Ren and Y. Wu, “Convolutional deep belief networks for feature extraction of eeg signal,” International joint conference on neural Networks (IJCNN), Beijing, China, pp. 2850–2853, 2014.

J. Li and A. Cichocki, “Deep learning of multifractal attributes from motor imagery induced eeg,” International Conference on Neural Information Processing, Springer, Cham, pp. 503–510, 2014.

N. Lu, T. Li, X. Ren, and H. Miao, “A deep learning scheme for motor imagery classification based on restricted boltzmann machines,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 25, no. 6, pp. 566–576, 2016.

P. Wang, A. Jiang, X. Liu, J. Shang, and L. Zhang, “Lstm-based eeg classification in motor imagery tasks,” IEEE Transactions on Neural Systems and Rehabilitation Engineering, vol. 26, no. 11, pp. 2086–2095, 2018.

B. Hu, X. Li, S. Sun, and M. Ratcliffe, “Attention recognition in eegbased affective learning research using cfs +

knn algorithm,” IEEE/ACM Transactions on Compu- tational Biology and Bioinformatics, vol. 15, no. 1, pp. 38–45, 2016.

V. J. Lawhern, A. J. Solon, N. R. Waytowich, S. M. Gordon, C. P. Hung, and B. J. Lance, “Eegnet: a compact convolut- ional neural network for eeg-based brain–computer interfaces,” Journal of Neural Engineering, vol. 15, no. 5, p. 056013, 2018.

R. Mane, N. Robinson, A. P. Vinod, S.-W. Lee, and C. Guan, “A multi-view cnn with novel variance layer for motor imagery brain computer interface,” The 42nd Annual International Conference of the IEEE Engineering in Medicine Biology Society (EMBC), pp. 2950–2953, 2020.

G. Hinton, S. Sabour, and N. Frosst, “Matrix capsules with em routing,” International Conference on Learning Representations(ICLR), Vancouver, Canada, 2018.

Z. Zhang, W. Liao, X.-N. Zuo, Z. Wang, C. Yuan, Q. Jiao, H. Chen, B. B. Biswal, G. Lu, and Y. Liu, “Resting-state brain organization revealed by functional covariance networks,” Plos One, vol. 6, no. 12, p. e28817, 2011

K. Ang, Y. Zheng, H. Zhang, and C. Guan, “Filter bank common spatial pattern (fbcsp) in brain-computer interface,” Proc. IEEE Int. Joint Conf. Neural Netw., pp. 2390–97, 2008

K. Ang, Z. Chin, C. W. C, C. Guan, and H. Zhang, “Filter bank common spatial pattern algorithm on bci competition iv datasets 2a and 2b,” Frontiers in Neuroscience, vol. 6, no. 39, pp. 1–9, 2012.

K. P. Thomas, C. Guan, L. C. Tong, and V. A. Prasad, “An adaptive filter bank for motor imagery based brain computer interface,” The 30th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC’08), British Columbia, Canada, pp. 1104– 1107, 2008.

E. Gentile, A. Brunetti, K. Ricci, M. Delussi, and M. de Tommaso, “Mutual interaction between motor cortex activation and pain in fibromyalgia: Eeg-fnirs study,” PloS One, vol. 15, no. 1, p. e0228158, 2020.

L. R, B. C, and M.-P. G, “Bci competition 2008-graz data set b,” Graz University of Technology, pp. 1–6, 2018

B. Yang, H. Li, Q. Wang, and Y. Zhang, “Subject-based feature extraction by using fisher wpd-csp in brain–computer interfaces,” Computer Methods and Programs in Biomedicine, vol. 129, pp. 21–28, 2016.

S. Sabour and N. Frosst, “Dynamic routing between capsules,” 31st Conference on Neural Information Processing Systems, Long Beach, CA, USA, 2017.

Q. Zhang, Y. N. Wu, and S. Zhu, “Interpretable convolutional neural networks,” IEEE/CVF Conference on Computer Vision and Pattern Recognition, Seattle, WA, USA, 2018.

Y. Liu, Y. Ding, C. Li, J. Cheng, R. Song, F. Wan, and X. Chen, “Multichannel eeg-based emotion recognition via a multi-level features guided capsule network,” Computers in Biology and Medicine, vol. 123, 2020.

K.-W. Ha and J.-W. Jeong, “Motor imagery eeg classification using capsule networks,” Sensors, vol. 19, no. 13, p. 2854, 2019.