Lightweight Test-time Adaptation for Robust Out-of-distribution Face Recognition in Web Services
DOI:
https://doi.org/10.13052/jwe1540-9589.2462Keywords:
Web 3.0, face recognition, face verification, out-of-distribution, test-time adaptationAbstract
Face recognition systems have the potential to support diverse services in Web 3.0 applications, yet two critical challenges remain underexplored. First, existing benchmark datasets are demographically biased and underrepresent elderly East Asian users, limiting fair and inclusive deployment. Second, sensor noise, lighting shifts, and motion blur introduce out-of-distribution (OOD) corruptions that cause severe accuracy degradation and undermine reliability in decentralized environments. To address these issues, we introduce the Korean Senior Face Benchmark, consisting of 700 images of 70 Korean senior celebrities, enabling realistic assessment for an underrepresented demographic. We quantitatively demonstrate that recent state-of-the-art models suffer significant performance drops under realistic corruption conditions, highlighting the need for enhanced robustness. Finally, we show that a lightweight test-time adaptation (TTA) strategy can recover OOD performance without retraining, making it well-suited for edge devices and distributed infrastructures while preserving user privacy. Experiments show accuracy gains of up to 41.5% under the most severe corruptions, along with improvements in intra-class compactness and inter-class separability in the embedding space. The proposed benchmark and adaptation pipeline lay a practical foundation for building distributed, fair, and privacy-aware face-recognition services in Web 3.0 applications.
Downloads
References
Zhang, Y., Y. Mirei, S. Masahiro, and K. Shoji. 2020. Attribute-Based Access Control for Smart Cities: A Smart Contract-Driven Framework. IEEE Internet of Things Journal. 8(8): 6372–6384. doi: nolinkurl10.1109/JIOT.2020.3033434.
Nita, S. L., and M. I. Mihailescu. 2024. A Novel Authentication Scheme Based on Verifiable Credentials Using Digital Identity in the Context of Web 3.0. Electronics. 13(6): 1137. doi: nolinkurl10.3390/electronics13061137.
Yu, T., M. P. Kowalski, L. T. Shackelford, D. C. Brooks and J. M. Ennis. 2024. Using Web3 technologies to represent personalized consumer taste preferences in whiskies. Food Quality and Preference. 118:105–201. doi: nolinkurl10.1016/j.foodqual.2024.105201.
Asif, M., Z. Aziz, M. B. Ahmad, A. Khalid, H. A. Waris and A. Gilani. 2022. Blockchain-Based Authentication and Trust Management Mechanism for Smart Cities. Sensors. 22(7): 2604. doi: 10.3390/s22072604
Karkkainen, K., and J. Jungseock. 2021. FairFace: Face attribute dataset for balanced race, gender, and age for bias measurement and mitigation. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). p. 1547–1557.
Wang, M., W. Deng, J. Hu, X Tao, and Y. Huang. 2020. Racial faces in the wild: Reducing racial bias by information maximization adaptation network. In: Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). p. 692–702.
Dobbala, M. K., and M. S. S. Lingolu. 2024. Enhancing usability for everyone through web accessibility compliance. J Comput Sci Software Dev. 3: 1–13. doi: nolinkurl10.17303/jcssd.2024.3.105.
Gorodishchev, A. V., A. N. Gorodishcheva, D. O. Baigozhina, and G. P. Kovalev. 2023 Web 3.0 Credibility: Principles for Ranking Media Sources. In Proceeding of the 2023 Communication Strategies in Digital Society Seminar (ComSDS). p. 184–188.
Wang, D., E. Shelhamer, S. Liu, B. Olshausen, and T. Darrell. 2021. Tent: Fully test-time adaptation by entropy minimization. Poster session presented at: The Ninth International Conference on Learning Representations (ICLR)
Taigman, Y., M. Yang, MA. Ranzato, and L. Wolf. 2014. DeepFace: Closing the gap to human-level performance in face verification. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). p. 1701–1708.
Sun, Y., Y. Chen, X. Wang, and X. Tang. 2014. Deep learning face representation by joint identification-verification. In Proceedings of international Conference on Neural Information Processing Systems (NIPS). p. 1988–1996.
Schroff, F., D. Kalenichenko, and J. Philbin. 2015 FaceNet: A unified embedding for face recognition and clustering. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). p. 815–823.
Sohn, K. 2016. Improved deep metric learning with multi-class n-pair loss objective. In Proceedings of the 30th International Conference on Neural Information Processing Systems (NIPS) p. 1857–1867.
Wen, Y., K. Zhang, Z. Li, and Y. Qiao. 2016. A discriminative feature learning approach for deep face recognition. In Computer Vision – ECCV 2016 : 14th European Conference (ECCV). p. 499–515.
Liu, W., Y. Wen, Z. Yu, and M. Yang. 2016. Large-margin softmax loss for convolutional neural networks. In Proceedings of the 33rd International Conference on Machine Learning (ICML). p. 507–516.
Wang, F., W. Liu, H. Liu, and J. Cheng. 2018 Additive margin softmax for face verification. IEEE Signal Process Letters. 25(7): 926–-930. doi: nolinkurl10.1109/lsp.2018.2822810.
Liu. W., Y. Wen, Z. Yu, M Li, B. Raj, and L. Song. 2017. SphereFace: Deep hypersphere embedding for face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR). p. 6738–6746.
Wang, H., Y. Wang, Z. Zhou, X. Ji, D. Gong, J. Zhou, Z. Li, and W. Liu. 2018. CosFace: Large margin cosine loss for deep face recognition. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition (CVPR) p. 5265–5274.
Deng, J., J. Guo, N. Xue, and S. Zafeiriou. 2020. ArcFace: Additive angular margin loss for deep face recognition. In Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 4685–4694.
Dan, J., Y. Liu, H. Xie, J. Deng, H. Xie, X. Xie, and B. Sun. 2024. Transface: Calibrating transformer training for face recognition from a data-centric perspective. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV) p. 20585–20596.
Dosovitskiy, A., L. Beyer, A. Kolesnikov, D. Weissenborn, X. Zhai, T. Unterthiner, et al. 2020. An image is worth 16×16 words: Transformers for image recognition at scale. In Proceedings of the Eighth International Conference on Learning Representations (ICLR). p. 11960–11973.
Dan, J., Y. Liu, J. Deng, H. Xie, S. Li, B. Sun, et al. 2024. TopoFR: A closer look at topology alignment on face recognition. In Advances in Neural Information Processing Systems (NeurIPS). 34: 37213–37240.
Kemelmacher-Shlizerman, I., S. Seitz, D. Miller, and E. Brossard. 2016. MegaFace: A million faces for recognition at scale. arXiv preprint arXiv:1505.02108.
Sharma, S., A. Saini, and S. Chaudhury. 2024. Multimodal biometric user authentication using improved decentralized fuzzy vault scheme based on Blockchain network. Journal of Information Security and Applications (JISA). 82:103740. doi: nolinkurl10.1016/j.jisa.2024.103740.
Saypadith, S., and S. Aramvith. 2018. Real-Time Multiple Face Recognition using Deep Learning on Embedded GPU System. In Proceedings of the 2018 Asia-Pacific Signal and Information Processing Association Annual Summit and Conference (APSIPA ASC). p. 1318–1324.
Bai, J., X. Zhang, X. Song, H. Shao, Q. Wang, S. Cui, and G. Russello. 2023. CryptoMask: Privacy-preserving face recognition. In Proceedings of the International Conference on Information and Communications Security (ICICS). p. 333–350.
Bradwell, H. L., K. J. Edwards, R. Baines, T. Page, A. Chatterjee, and R. B. Jones. 2023. Facial recognition lock technology for social care settings: A qualitative evaluation of implementation of facial recognition locks at two residential care sites. Frontiers in Digital Health. 5:1066327. doi: nolinkurl10.3389/fdgth.2023.1066327.
Hsieh, W.-B. 2025. BF-ACS—Intelligent and Immutable Face Recognition Access Control System. IET Information Security. 2025(1):6755170. doi: 10.1049/ise2.2025.0012.
Garg, A., D. Dixit, A. Khanna, and S. Doss. 2020. UI Personalization using Face Recognition and Machine Learning. In Proceedings of the International Conference on Innovative Computing & Communications (ICICC).
Huang, G. B., M. Mattar, T. Berg, and E. Learned-Miller. 2008. Labeled Faces in the Wild: A database for studying face recognition in unconstrained environments. Paper session presented at: Workshop on Faces in ‘Real-Life’ Images: Detection, Alignment, and Recognition;
Maze, B., Adams, J. A. Duncan J, N. Kalka, T. Miller, C. Otto, et al. 2018. IARPA Janus Benchmark-C: Face dataset and protocol. In: Proceedings of the 2018 International Conference on Biometrics (ICB); p. 158–165.
Zheng T, W. Deng, and J. Hu. 2017. Cross-age LFW: A database for studying cross-age face recognition in unconstrained environments. arXiv preprint arXiv:1708.08197.
Zheng, T., and W. Deng. 2018. Cross-pose LFW: A database for studying cross-pose face recognition in unconstrained environments. :Beijing University of Posts and Telecommunications. 5(7); 5.
Moschoglou, S., A. Papaioannou, C. Sagonas, J. Deng, I. Kotsia, S. Zafeiriou. 2017. AgeDB: The first manually collected, in-the-wild age database. In Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops (CVPRW). p. 1997–2005.
Sengupta, S., JC. Chen, C. Castillo, V. M. Patel, R. Chellappa, and D. W. Jacobs. 2016. Frontal to profile face verification in the wild. In Proceedings of the IEEE winter conference on applications of computer vision (WACV). p. 1–9.
Ryu, H. J., H. Adam, and M. Mitchell. 2017. InclusiveFaceNet: Improving face attribute detection with race and gender diversity. arXiv preprint arXiv:1712.00193.
Venkatasalam, K., K. K. Dhanush, S. Dharun, J. Hemanthraj, and S. Irayanbu. 2024. AI-assisted missing person finder and face recognition using FaceNet algorithm. ShodhKosh: Journal of Visual and Performing Arts. 5(5): 1027–1033. doi: nolinkurl10.29121/shodhkosh.v5.i5.2024.2670.
Phan, H., C. X. Le, V. Le, Y. He, and A. T. Nguyen. 2024. Fast and interpretable face identification for out-of-distribution data using vision transformers. In Proceedings of the IEEE/CVF Winter Conference on Applications of Computer Vision (WACV). p. 6289–6299.
Hendrycks, D., and T. Dietterich. 2019. Benchmarking neural network robustness to common corruptions and perturbations. Poster session presented at: The Seventh International Conference on Learning Representations (ICLR).
Kang, C., Y. Chen, S. Ruan, S. Zhao, R. Zhang, J. Wang, S. Fu, and X. Wei. 2024. OODFace: Benchmarking robustness of face recognition under common corruptions and appearance variations. arXiv preprint arXiv:2412.02479.
Neto, P. C., J. R. Pinto, F. Boutros, N. Damer, A. F. Sequeira, and J. S. Cardoso. 2022. Beyond masks: On the generalization of masked face recognition models to occluded face recognition. IEEE Access. 10: 86222–-86233. doi: nolinkurl10.1109/ACCESS.2022.3199014.
Mirza, M. J., J. Micorek, H. Possegger, and H. Bischof. 2022. The norm must go on: Dynamic unsupervised domain adaptation by normalization. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 14765–14775.
Jang, M., S. Y. Chung, and H. W. Chung. 2023. Test-time adaptation via self-training with nearest neighbor information. The Eleventh International Conference on Learning Representations (ICLR).
Chen, C., P. Zhang, H. Zhang, J. Dai, Y. Yi, H. Zhang, and Y. Zhang. 2020. Deep learning on computational-resource-limited platforms: A survey. Mobile Information Systems. 2020: 8454327. doi: nolinkurl10.1155/2020/8454327.
Hu, K., S. Gong, Q. Zhang, C. Seng, M. Xia, and S. Jiang. 2024. An overview of implementing security and privacy in federated learning. Artificial Intelligence Review. 57: 204. doi: nolinkurl10.1007/s10462-024-10846-8.
Sumsion, A, S. Torrie, D. J. Lee, and Z. Sun. 2024. Surveying racial bias in facial recognition: Balancing datasets and algorithmic enhancements. Electronics. 13(12): 2317. doi: nolinkurl10.3390/electronics13122317.
Deng, J., J. Guo, E. Ververas, I. Kotsia, and S. Zafeiriou. 2020. RetinaFace: Single-Shot Multi-Level Face Localisation in the Wild. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 5203–5212.
Jing, M., X. Zhen, J. Li, and C. G. M. Snoek. 2022. Variational model perturbation for source-free domain adaptation. Advances in Neural Information Processing Systems. 35: 17173–17187.
Yi, L., G. Xu, P. Xu, J. Li, R. Pu, C. Ling, L. McLeod, and B. Wang. 2023. When source-free domain adaptation meets learning with noisy labels The Eleventh International Conference on Learning Representations (ICLR).
Kim, M., A. K. Jain, and X. Liu. 2022. AdaFace: Quality adaptive margin for face recognition. Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition (CVPR). p. 18750–18759.
Guo, Y., L. Zhang, Y. Hu, X. He, and J. Gao. 2016. MS-Celeb-1M: A dataset and benchmark for large-scale face recognition. In Computer Vision – ECCV 2016 : 14th European Conference (ECCV). p. 87–102.
He, K., X. Zhang, S. Ren, and J. Sun. 2016. Deep residual learning for image recognition. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 770–778.
Duta, I. C., L. Liu, F. Zhu, and L. Shao. 2021. Improved residual networks for image and video recognition. Proceedings of International Conference on Pattern Recognition (ICPR). p. 9415–9422.
Michaelis, C., B. Mitzkus, R. Geirhos, E. Rusak, O. Bringmann, A. S. Ecker, M. Bethge, and W. Brendel. 2019. Benchmarking Robustness in Object Detection: Autonomous Driving when Winter is Coming. arXiv preprint arXiv:1907.07484.
Yang, S., P. Luo, C. C. Loy, and X. Tang. 2016. Wider face: A face detection benchmark. Proceedings of the IEEE/CVF conference on computer vision and pattern recognition (CVPR). p. 5525–5533.
Zhang, S., X. Zhu, Z. Lei, H. Shi, X. Wang, and S. Z. Li. 2017. S3fd: Single shot scale-invariant face detector. In Proceedings of the IEEE/CVF International Conference on Computer Vision (ICCV). p. 192–201.
Anwar, A., and A. Raychowdhury. 2020. Masked face recognition for secure authentication. arXiv preprint arXiv:2008.11104.
