Visual Quality Assessment of Point Clouds Compared to Natural Reference Images
DOI:
https://doi.org/10.13052/jwe1540-9589.2232Keywords:
Point cloud, quality evaluation, visual quality assessmentAbstract
This paper proposes a point cloud (PC) visual quality assessment (VQA) framework that reflects the human visual system (HVS). The proposed framework compares natural images acquired using a digital camera and PC images generated via 2D projection in terms of appropriate objective quality evaluation metrics. Humans primarily consume natural images; thus, human knowledge is typically formed from natural images. Thus, natural images can be more reliable reference data than PC data. The proposed framework performs an image alignment process based on feature matching and image warping to use the natural images as a reference which enhances the similarities of the acquired natural and corresponding PC images. The framework facilitates identifying which objective VQA metrics can be used to reflect the HVS effectively. We constructed a database of natural images and three PC image qualities, and objective and subjective VQAs were conducted. The experimental result demonstrates that the acceptable consistency among different PC qualities appears in the metrics that compare the global structural similarity of images. We found that the SSIM, MAD, and GMSD achieved remarkable Spearman rank-order correlation coefficient scores of 0.882, 0.871, and 0.930, respectively. Thus, the proposed framework can reflect the HVS by comparing the global structural similarity between PC and natural reference images.
Downloads
References
D. J. Brady, W. Pang, H. Li, et al., “Parallel cameras,” Optica, vol. 5, no. 2, pp. 127–137, Feb 2018.
S. Schwarz, M. Preda, V. Baroncini, et al., “Emerging mpeg standards for point cloud compression,” IEEE J. Emerg. Sel. Topics in Circuits and Systems, vol. 9, no. 1, pp. 133–148, Dec 2018.
K. Khoshelham, and S. O. Elberink, “Accuracy and resolution of kinect depth data for indoor mapping applications,” Sensors (Basel), vol. 12, no. 2, pp. 1437–1454, 2012.
H.-G. Maas, A. Bienert, S. Scheller, E. Keane, “Automatic forest inventory parameter determination from terrestrial laser scanner data,” Int. J. Remote Sens., vol. 29, no. 5, pp. 1579–1593, 2008.
D. D. Lichti, and M. G. Licht, “Experiences with terrestrial laser scanner modelling and accuracy assessment,” Int. Arch. Photogramm. Remote Sens, Spat. Inf. Sci., vol. 36, no. 5, pp. 155–160, 2006.
M. Berger, A. Tagliasacchi, L. M. Seversky, et al., “A survey of surface reconstruction from point clouds,” Comput. Graph. Forum, vol. 36, no. 1, 301–329, 2017.
Available at: http://graphics.stanford.edu/data/3Dscanrep/.
G. Turk, and M. Levoy, “Zippered polygon meshes from range images,” Proc. SIGGRAPH ‘94, Computer Graphics Proceedings, Annual Conference Series, 1994, pp. 311–318.
E. Dumic, C. R. Duarte and L. A. da Silva Cruz, “Subjective evaluation and objective measures for point clouds – State of the art,” 2018 First International Colloquium on Smart Grid Metrology (SmaGriMet), 2018, pp. 1–5.
K. Mammou, P. A. Chou, D. Flynn, et al., G-PCC codec description v2 ISO/IEC JTC1/SC29/WG11, vol. N18189, 2019.
M. Committee, “V-PCC Codec Description JTC1/SC29/WG11 w19526,” Italy: Virtual, Document ISO/IEC, Sep. 2020.
JPEG Pleno Point Cloud Coding Common Test Conditions ISO/IEC JTC1/SC29/WG11, vol. N18474, 2019.
“Final call for evidence on JPEG pleno point cloud coding” in ISO/IEC JTC1/SC29/WG1 JPEG Output Document, vol. N88014, Jul. 2020.
S. Park, J. Kim, Y. Hwang, et al., “Enhancement of 3D point cloud contents using 2D image super resolution network,” Journal of Web Engineering, vol. 21, pp. 425–442, 2022.
K. Wolff et al., “Point cloud noise and outlier removal for image-based 3D reconstruction,” IEEE Fourth Int. Conf. on 3D Vis., Stanford, CA, USA, Oct. 2016.
M. Berger, A. Tagliasacchi, L. Seversky, et al., “A survey of surface reconstruction from point clouds,” Comput. Graph. Forum, vol. 36, no. I, Mar 2016.
A. Javaheri, C. Brites, F. Pereira, and J. Ascenso, “Subjective and objective quality evaluation of 3d point cloud denoising algorithms,” ISO/IEC JTC m75024. Sydney, Australia, Mar. 2017.
H. R. Sheikh, and A. C. Bovik, “Image information and visual quality,” IEEE Trans. Image Process., vol. 15, no. 2, pp. 4–2298, Sep 2007.
Z. Wang, and A. C. Bovik, “Mean squared error: Love it or leave it? – A new look at signal fidelity measures,” IEEE Signal Process. Mag., vol. 26, no. 1, pp. 98–117, Jan 2009.
E. Alexiou, “On subjective and objective quality evaluation of point cloud geometry,” in 9th Int. Conf. Qual. of Multimedia Experience (QoMEX’17). IEEE, pp. 1–3, 2017.
Z. Wang, A. C. Bovik, H. R. Sheikh, E. P. Simoncelli, “Image quality assessment: From error visibility to structural similarity,” IEEE Trans. Image Process., vol. 13, no. 4, pp. 600–612, 2004.
D. Tiang, H. Ochimizu, C. Feng, et al., “Geometric distortion metrics for point cloud compression,” IEEE Intl Conf. on Image Process., Sep. 2017.
S. Perry, A. Pinheiro, E. Dumic, et al., “Study of subjective and objective quality evaluation of 3D point cloud data by the JPEG committee,” Electronic Imaging, vol. 31, no. 10, pp. 312–316, 2019.
A. Javaheri, C. Brites, F. Pereira, J. Ascenso, “Subjective and objective quality evaluation of 3D point cloud denoising algorithms,” IEEE Intl Conf. on Multimedia Expo Workshops (ICMEW), pp. 1–6, Jul. 2017.
Q. Yang, Z. Ma, Y. Xu, et al., “Inferring point cloud quality via graph similarity,” arXiv Preprint ArXiv:2006.00497, 2020.
S. Chen, D. Tian, C. Feng, et al., “Fast resampling of three-dimensional point clouds via graphs,” IEEE Trans. Signal Process., vol. 66, no. 3, pp. 666–681, 2017.
S. Yan, Y. Peng, G. Wang, et al., “Weakly supported plane surface reconstruction via plane segmentation guided point cloud enhancement,” IEEE Access, vol. 8, 60491–60504, 2019.
Y. Regaya, F. Fadli, A. Amira, “3d point cloud enhancement using unsupervised anomaly detection,” Int. Symp. on Syst. Eng. (ISSE”19), vol. 2019. IEEE, 2019, pp. 1–6.
T. Zheng, C. Chen, J. Yuan, et al., “Pointcloud saliency maps,” Proc. IEEE Int. Conf. Computer Vision (ICCV’19), 2019, pp. 1598–1606.
T. Hackel, N. Savinov, L. Ladicky, et al., “Semantic3d. net: A new large-scale point cloud classification benchmark,” arXiv Preprint ArXiv:1704.03847, 2017.
Z. Zhang, L. Zhang, X. Tong, et al., “A multilevel point-cluster-based discriminative feature for als point cloud classification,” IEEE Trans. Geosci. Remote Sens., vol. 54, no. 6, pp. 3309–3321, 2016.
T. Rabbani, F. Van Den Heuvel, G. Vosselmann, “Segmentation of point clouds using smoothness constraint,” Int. Arch. Photogramm. Remote Sens. Spat. Inf. Sci., vol. 36, no. 5, pp. 248–253, 2006.
R. Schnabel and R. Klein, “Octree-based point-cloud compression,” SPBG, vol. 6, pp. 111–120, 2006.
Y. Shao, Q. Zhang, G. Li, et al., “Hybrid point cloud attribute compression using slice-based layered structure and block-based intra prediction,” Proc. 26th ACM Int. Conf. Multimedia, 2018, pp. 1199–1207.
S. Gumhold, Z. Kami, M. Isenburg, H.-P. Seidel, “Predictive point-cloud compression,” ACM SIGGRAPH, 2005 Sketches, pp. 137–es, 2005.
L. Li, Z. Li, S. Liu, H. Li, “Occupancy-map-based rate distortion optimization and partition for video-based point cloud compression,” IEEE Trans. Circuits Syst. Video Technol., vol. 31, no. 1, 326–338, 2020.
E. Alexiou and T. Ebrahimi, “Towards a point cloud structural similarity metric,” Proc. IEEE Int. Conf. Multimedia Expo Workshops, pp. 1–6, 2020.
B. An and Y. Kim, “Image link through adaptive encoding data base and optimized GPU algorithm for real-time image processing of artificial intelligence,” Journal of Web Engineering, 2022, vol. 21, pp. 459–496.
W. Zhou, Q, Yang, Q. Jiang, et al., “Blind quality assessment of 3D dense point clouds with structure guided resampling,” arXiv 2022, arXiv:2208.14603.
M. H. Pinson and S. Wolf, “A new standardized method for objectively measuring video quality,” IEEE Trans. Broadcast., vol. 3, pp. 312–322, Sep. 2004.
A. K. Moorthy, C. Su, A. Mittal, and A. C. Bovik, “Subjective evaluation of stereoscopic image quality,” Signal Processing: Image Communication, vol. 28, no. 8, 870–883, 2013.
E. Dumic and S. Grgic, “Reduced video quality measure based on 3D steerable wavelet transform and modified structural similarity index,” Proc. of the 55th Intl Symp. ELMAR-2013, 2013, pp. 65–69.
R. Soundararajan and A. C. Bovik, “Video quality assessment by reduced reference spatio-temporal entropic differencing,” IEEE Trans. Circuits Syst. Video Technol., vol. 23, no. 4, pp. 684–694, Apr. 2013.
F. Battisti, E. Bosc, M. Carli, et al., “Objective image quality assessment of 3D synthesized views,” Signal Process. Image Commun., vol. 30, pp. 78–88, Jan 2015.
E. Dumić, S. Grgić, K. Šakic, P. M. R. Rocha, L. A. da Silva Cruz, “3D video subjective quality: A new database and grade comparison study,” Multimedia Tool. Appl., vol. 76, no. 2, pp. 2087–2109, Jan. 2017.
E. M. Torlig, E. Alexiou, T. A. Fonseca, et al., “A novel methodology for quality assessment of voxelized point clouds,” SPIE vol. 10752, Applications of Digital Image Processing XLI. San Diego, CA, USA, 2018.
Q. Liu, H. Yuan, H. Su, et al., “PQA-Net: Deep no reference point cloud quality assessment via multi-view projection,” IEEE Transactions on Circuits and Systems for Video Technology 2021, 31, 4645–4660.
A. Artusi, F. Banterle, F. Carra, A. Moreno, “Efficient evaluation of image quality via deep-learning approximation of perceptual metrics,” IEEE Transactions on Image Processing, vol. 29, pp. 1843–1855, 2019.
R. Mantiuk, K. Kim, A. G. Rempel, W. Heidrich, “HDR-VDP-2: A calibrated visual metric for visibility and quality pre-dictions in all luminance conditions,” ACM Transactions on Graphics, vol. 30, pp. 1–14, 2011.
ITU-R BT.500-13 “Methodology for the subjective assessment of the quality of television pictures,” International Telecommunication Union. ITU Radiocommunication Sector, 2012.
ITU-R BT.2021, “Subjective methods for the assessment of stereoscopic 3DTV systems,” International Telecommunication Union. ITU Radiocommunication Sector, 2012.
D. G. Lowe, “Distinctive image features from scale-invariant Keypoints,” Intl. J. Comput. Vis., vol. 60, no. 2, pp. 91–110, 2004.
H. Bay, T. Tuytelaars, L. Van Gool, “Surf: Speeded up robust features,” Eur. Conf. on Comput. Vis., May 2006.
G. Bjøntegaard, “Calculation of average PSNR differences between RD curves,” ITU-T, vol. Q6, no. 16, Apr, document VCEG-M33, 2001.
Z. Wang, E. P. Simoncelli, A. C. Bovik, “Multiscale structural similarity for image quality assessment,” Proc. IEEE Conf. Rec. 37th Asilomar Conf. Signals, Syst. Comput., Nov 2003, vol. 2, pp.1398-1402.
Z. Wang and Q. Li, “Information content weighting for perceptual image quality assessment,” IEEE Trans. Image Process., vol. 20, no. 5, pp. 1185– 1198, Nov 2011.
L. Zhang, L. Zhang, X. Mou, D. Zhang, “FSIM: A feature similarity index for image quality assessment,” IEEE Trans. Image Process., vol. 20, no. 8, pp. 2378–2386, Aug 2011.
ITU-R Recommendation BT. 1788, “Methodology for the subjective assessment of video quality in multimedia application,” 2007.
X. Qin, Z. Zhang, C. Huang, et al., “BASNet: Boundary-aware salient object detection,” Proc. IEEE/CVF Conf. Comput. Vis., Pattern Recognit. (CVPR), Jun. 2019, pp. 7479–7489.
X. Qin, Z. Zhang, C. Huang, et al., “U2-net: Going deeper with nested U-structure for salient object detection,” Pattern Recognit., vol. 106, no. Oct, Art. no. 107404, 2020.
E. C. Larson and D. M. Chandler, “Most apparent distortion: Full-reference image quality assessment and the role of strategy,” J. Electron. Imag., vol. 19, pp. 011006:1-011006:21, Jan 2010.
W. Xue, L. Zhang, X. Mou, A. C. Bovik, “Gradient magnitude similarity deviation: A highly efficient perceptual image quality index,” IEEE Trans. Image Process., vol. 23, no. 2, pp. 684–695, Feb. 2014.
L. Zhang, Y. Shen, H. Li, “VSI: A visual saliency-induced index for perceptual image quality assessment,” IEEE Trans. Image Process., vol. 23, no. 10, pp. 4270–4281, Aug 2014.
V. Laparra, J. Ballé, A. Berardino, E. P. Simoncelli, “Perceptual image quality assessment using a normalized Laplacian pyramid,” Electronic Imaging, vol. 28, no. 16, pp. 1–6, Feb 2016.
Y. Jin, M. Chen, T. Goodall, et al., “Subjective and objective quality assessment of 2d and 3d foveated video compression in virtual reality,” IEEE Transactions on Image Processing, vol. 30, pp. 5905–5919, 2021.
ITU-T P.1401, “Methods, metrics and procedures for statistical evaluation, qualification and comparison of objective quality prediction models,” International Telecommunication Union, Jul 2012.
C. Loscos, P. Souchet, T. Barrios, et al., “From capture to immersive viewing of 3D HDR point clouds,” Eurographics, Reims, France, 25-29 April 2022.
