A Deep Convolutional Neural Network to Limit Virus Spread Using Facial Mask Segmentation
DOI:
https://doi.org/10.13052/jwe1540-9589.20414Keywords:
Mask segmentation, Face detection, Convolutional Neural Networks, Deep learningAbstract
Due to the recent COVID-19 outbreak the world has experienced many challenges. Limit and control the virus spread rate is one of them. This letter focuses on limiting the speed of virus spreading by monitoring the use of facial mask in crowded public environments such as tourism places, commercial centres, etc. The proposed method first accurately localizes faces using a state-of-the-art approach and, segments facial mask in a second step. The facial mask segmentation allows to distinguish whether the current subject is wearing a facial mask or not but also if it is properly covering the human face. Indeed, most recent face detection algorithms provide as output a set of facial features such as nose tip and mouth corners. By combining these facial features with facial mask segmentation, the proposed method detects real-time subjects that indirectly encourage virus spread in crowded environments. The proposed facial mask segmentation model is trained with pairs of RGB images and its corresponding alpha image created by extending the publicly available real-world masked face dataset. Further, the proposed model is pruned and optimized using the TensorRt library to be usable for real-world applications.
Downloads
References
P. Viola, M. J. Jones, ‘Robust real-time face detection’, International Journal of Computer Vision, pp. 137–154, 2004.
D. Chen, G. Hua, F. Wen, J. Sun, ‘Supervised transformer network for efficient face detection’, European Conference on Computer Vision, pp. 122–138, 2016.
K. Zhang, Z. Zhang, Z. Li, Y. Qiao, ‘Joint face detection and alignment using multitask cascaded convolutional networks’, IEEE Signal Processing Letters, pp. 1499–1503, 2016.
F. Liu, D. Zeng, Q. Zhao, X. Liu, ‘Joint face alignment and 3d face reconstruction’, European Conference on Computer Vision, pp. 545–560, 2016.
J. Deng, J. Guo, N. Xue, S. Zafeiriou, ‘Arcface: Additive angular margin loss for deep face recognition’, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4690–4699, 2019.
Z. Wang, G. Wang, B. Huang, Z. Xiong, Q. Hong, H. Wu, P. Yi, K. Jiang, N. Wang, Y. Pei, H. Chen, Y. Miao, Z. Huang, J. Liang, ‘Masked face recognition dataset and application’, arXiv preprint arXiv:2003.09093, 2020.
J. S. Park, Y. H. Oh, S. C. Ahn, S. W. Lee, ‘Glasses removal from facial image using recursive error compensation’, IEEE Transactions on Pattern Analysis and Machine Intelligence, pp. 805–811, 2005.
S. Saito, T. Li, H. Li, ‘Real-time facial segmentation and performance capture from rgb input’, European Conference on Computer Vision, pp. 224–261, 2016.
K. Lin, H. Zhao, J. Ly, C. Li, X. Liu, R. Chen, R. Zhao, ‘Face Detection and Segmentation Based on Improved Mask R-CNN’, Discrete Dynamics in Nature and Society, pp. 1–11, 2020.
K. Simonyan, A. Zisserman, ‘Very deep convolutional networks for large-scale image recognition’, International Conference on Learning Representations, 2015.
H. Noh, S. Hong, B. Han, ‘Learning deconvolution network for semantic segmentation’, Proceedings of the IEEE International Conference on Computer Vision, pp. 1520–1528, 2015.
M. Loey, G. Manogaran, M. H. N. Taha, N. E. M. Khalifa, ‘A hybrid deep transfer learning model with machine learning methods for face mask detection in the era of the COVID-19 pandemic’, Measurement, vol. 167, 2020.
A. Lodh, U. Saxena, A. Motwani, L. Shakkeera, V. Y. Sharmasth, ‘Prototype for Integration of Face Mask Detection and Person Identification Model–COVID-19’, 4th International Conference on Electronics, Communication and Aerospace Technology (ICECA), pp. 1361–1367, 2020.
M. M. Rahman, M. M. H. Manik, M. M. Islam, S. Mahmud, J.H. Kim, ‘An Automated System to Limit COVID-19 Using Facial Mask Detection in Smart City Network’, IEEE International IOT, Electronics and Mechatronics Conference (IEMTRONICS), pp. 1–5, 2020.
S. Yang, P. Luo, C. C. Loy, X. Tang, ‘Wider face: A face detection benchmark’, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5525–5533, 2016.
Z. Liu, P. Luo, X. Wang, X. Tang, ‘Deep learning face attributes in the wild’, Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 3730–3738, 2015.
O. Ronneberger, P. Fischer, T. Brox, ‘U-net: Convolutional networks for biomedical image segmentation’, International Conference on Medical Image Computing and Computer-Assisted Intervention, pp. 234–241, 2015.
CDC, ‘How to wear masks’ https://www.cdc.gov/coronavirus/2019-ncov/prevent-getting-sick/how-to-wear-cloth-face-coverings.html, updated September 2020.
