Structure Preserving Large Imagery Reconstruction

Authors

  • Ju Shen Department of Computer Science, University of Dayton, 300 College Park, Dayton, OH 45469, USA
  • Jianjun Yang Department of Computer Science and Information Systems, University of North Georgia, Oakwood, GA 30566, USA
  • Sami Taha-abusneineh Computer Science Department, Palestine Polytechnic University (PPU), Ein Sara Street, Hebron, Palestine
  • Bryson Payne Department of Computer Science and Information Systems, University of North Georgia, Dahlonega, GA 30597, USA
  • Markus Hitz Department of Computer Science and Information Systems, University of North Georgia, Dahlonega, GA 30597, USA

DOI:

https://doi.org/10.13052/jcsm2245-1439.332

Keywords:

Structure Preserving Large Imagery Reconstruction

Abstract

With the explosive growth of web-based cameras and mobile devices, billions of photographs are uploaded to the internet. We can trivially collect a huge number of photo streams for various goals, such as image clustering, 3D scene reconstruction, and other big data applications. However, such tasks are not easy due to the fact the retrieved photos can have large variations in their view perspectives, resolutions, lighting, noises, and distortions. Furthermore, with the occlusion of unexpected objects like people, vehicles, it is even more challenging to find feature correspondences and reconstruct realistic scenes. In this paper, we propose a structure-based image completion algorithm for object removal that produces visually plausible content with consistent structure and scene texture. We use an edge matching technique to infer the potential structure of the unknown region. Driven by the estimated structure, texture synthesis is performed automatically along the estimated curves. We evaluate the proposed method on different types of images: from highly structured indoor environment to natural scenes. Our experimental results demonstrate satisfactory performance that can be potentially used for subsequent big data processing, such as image localization, object retrieval, and scene reconstruction. Our experiments show that this approach achieves favorable results that outperform existing state-of-the-art techniques.

Downloads

Download data is not yet available.

Author Biographies

Ju Shen, Department of Computer Science, University of Dayton, 300 College Park, Dayton, OH 45469, USA

Ju Shen is an Assistant Professor from the Department of Computer Science, University of Dayton (UD), Dayton, Ohio, USA. He received his Ph.D. degree from University of Kentucky, Lexington, KY, in USA and his M.Sc degree from University of Birmingham, Birmingham, United Kingdom. His work spans a number of different areas in computer vision, graphics, multimedia, and image processing.

Jianjun Yang, Department of Computer Science and Information Systems, University of North Georgia, Oakwood, GA 30566, USA

Jianjun Yang Jianjun Yang received his B.S. degree and his first M.S. degree in Computer Science in China, his second M.S. degree in Computer Science during his doctoral study from University of Kentucky, USA in May 2009, and his Ph.D degree in Computer Science from University of Kentucky, USA in 2011. He is currently an Assistant Professor in the Department of Computer Science and Information Systems at the University of North Georgia. His research interest includes wireless networking, computer networks, and image processing.

Sami Taha-abusneineh, Computer Science Department, Palestine Polytechnic University (PPU), Ein Sara Street, Hebron, Palestine

Sami Taha Abu Sneineh is currently an assistant professor at Palestine Polytechnic University (PPU). He received his Ph.D. degree in May 2013 in Computer Vision at the University of Kentucky (UK) under the supervision of Dr. Brent Seales. Sami earned a B.Sc in Electrical Engineering from Palestine Polytechnic University in 2001 and M.Sc in Computer Science from Maharishi University of Management in 2006. His research focus is on computer vision in minimally invasive surgery to improve the performance assessment. Sami has worked for three years as consultant software engineer at Lexmark Inc. and four years as consultant programmer at IBM before joining UK. He started his teaching experience as TA in the computer science department. He received a certificate and an award in college teaching and learning in 2012. His goal is to contribute in improving the higher education system and raise the education standards.

Bryson Payne, Department of Computer Science and Information Systems, University of North Georgia, Dahlonega, GA 30597, USA

Bryson Payne received his B.S. degree in Mathematics from North Georgia College & State University, USA in 1997, his M.S. degree Mathematics from North Georgia College & State University, USA in 1999, and his Ph.D degree in computer science from Georgia State University, USA in 2004. He was the (CIO) Chief Information Officer in the North Georgia College & State University (former University of North Georgia). He is currently an Associate Professor in the Department of Computer Science and Information Systems at the University of North Georgia. He is also a CISSP (Certified Information Systems Security Professional). His research interest includes image processing, web application and communications.

Markus Hitz, Department of Computer Science and Information Systems, University of North Georgia, Dahlonega, GA 30597, USA

Markus Hitz received his B.S. degree and M.S. degree of Computer Science in Europe and his Ph.D degree of Computer Science in USA. He is currently a Professor and Acting Head in the Department of Computer Science and Information Systems at the University of North Georgia. His research interest includes networking, simulation and communications.

References

Jim Jing-Yan Wang, Halima Bensmail, and Xin Gao. Joint learning and weighting of visual vocabulary for bag-of-feature based tissue classification. Pattern Recognition, 46(12):3249–3255, 2013.

Jim Jing-Yan Wang, Halima Bensmail, and Xin Gao. Multiple graph regularized nonnegative matrix factorization. Pattern Recognition, 46(10):2840–2847, 2013.

Q. Sun, P. Wu, and Y. Wu. Unsupervised multi-level non-negative matrix factorization model: Binarydata case. International Journal of Information Security, 3(4):245–250, 2012.

Jing-Yan Wang, Islam Almasri, and Xin Gao. Adaptive graph regularized nonnegative matrix factorization via feature selection. In Pattern Recognition (ICPR), 2012 21st International Conference on}, pages 963–966. IEEE, 2012.

Ju Shen and S.-C.S. Cheung. Layer depth denoising and completion for structured-light rgb-d cameras. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 1187–1194, 2013.

Yuzhu Zhou, Le Li, Tong Zhao, and Honggang Zhang. Region-based high-level semantics extraction with cedd. In Network Infrastructure and Digital Content, 2010 2nd IEEE International Conference on}, pages 404–408. IEEE, 2010.

Ju Shen and Wai tian Tan. Image-based indoor place-finder using image to plane matching. In Multimedia and Expo (ICME), 2013 IEEE International Conference on, pages 1–6, 2013.

Yuzhu Zhou, Le Li, and Honggang Zhang. Adaptive learning of region-based plsa model for total scene annotation. arXiv preprint arXiv:1311.5590, 2013.

Le Li, Jianjun Yang, Kaili Zhao, Yang Xu, Honggang Zhang, and Zhuoyi Fan. Graph regularized non-negative matrix factorization by maximizing correntropy. arXiv preprint arXiv:1405.2246, 2014.

Q. Sun, F. Hu, and Q. Hao. Context awareness emergence for distributed binary pyroelectric sensors. In Proceeding of 2010 IEEE International Conference of Multi-sensor Fusion and Integration of Intelligent Systems, pages 162–167, 2010.

Le Li, Jianjun Yang, Yang Xu, Zhen Qin, and Honggang Zhang. Document clustering based on max-correntropy non-negative matrix factorization. 2014.

Q. Sun, F. Hu, and Q. Hao. Mobile target scenario recognition via low-cost pyroelectric sensing system: Toward a context-enhanced accurate identification. IEEE Transactions on Systems, Man and Cybernetics: Systems, 4(3):375–384, 2014.

Ju Shen, Anusha Raghunathan, Sen ching Cheung, and Rita Patel. Automatic content generation for video self modeling. In Multimedia and Expo (ICME), 2011 IEEE International Conference on, pages 1–6, 2011.

Ju Shen, Changpeng Ti, Sen ching Cheung, and R.R. Patel. Automatic lip-synchronized video-self-modeling intervention for voice disorders. In e-Health Networking, Applications and Services (Healthcom), 2012 IEEE 14th International Conference on}, pages 244–249, 2012.

Ju Shen, Changpeng Ti, Anusha Raghunathan, Sen ching Cheung, and Rita Patel. Automatic video self modeling for voice disorder. In Computer Vision and Pattern Recognition (CVPR), 2013 IEEE Conference on, pages 1187–1194, 2013.

Li Xu, Zhenxin Zhan, Shouhuai Xu, and Keying Ye. Cross-layer detection of malicious websites. In Proceedings of the third ACM conference on Data and application security and privacy, pages 141–152. ACM, 2013.

Jianjun Yang and Zongming Fei. Hdar: Hole detection and adaptive geographic routing for ad hoc networks. In Computer Communications and Networks (ICCCN), 2010 Proceedings of 19th International Conference on, August 2010.

Li Xu, Zhenxin Zhan, Shouhuai Xu, and Keying Ye. An evasion and Counter-Evasion study in malicious websites detection. In 2014 IEEE Conference on Communications and Network Security (CNS) (IEEE CNS 2014), San Francisco, USA, 2014.

Jianjun Yang and Zongming Fei. Broadcasting with prediction and selective forwarding in vehicular networks. In International Journal of Distributed Sensor Networks, 2013.

Yi Wang, Arnab Nandi, and Gagan Agrawal. SAGA: Array Storage as a DB with Support for Structural Aggregations. In Proceedings of SSDBM, June 2014.

Hong Zhang, Fengchong Kong, Lei Ren, and Jian-Yue Jin. An inter-projection interpolation (ipi) approach with geometric model restriction to reduce image dose in cone beam ct (cbct) computational modeling of objects presented in images. In Fundamentals, Methods, and Applications Lecture Notes in Computer Science, volume 8641, pages 12–23, 2014.

Yi Wang, Wei Jiang, and Gagan Agrawal. Scimate: A novel mapreduce-like framework for multiple scientific data formats. In Cluster, Cloud and Grid Computing (CCGrid), 2012 12th IEEE/ACM International Symposium on, pages 443–450. IEEE, 2012.

Hong Zhang, Vitaly Kheyfets, and Ender Finol. Robust infrarenal aortic aneurysm lumen centerline detection for rupture status classication. In Medical Engineering and Physics, volume 35(9), pages 1358–1367, 2013.

Hong Zhang, Lin Yang, David J. Foran, John L. Nosher, and Peter J. Yim. 3d segmentation of the liver using free-form deformation based on boosting and deformation gradients. In IEEE International Symposium on Biomedical Imaging (ISBI), 2009.

Bertalmio M. Sapiro G. Ballester C. and Caselles V. Image inpainting. Proceedings of ACM SIGGRAPH 2000, pages 417–424, 2000.

A. Criminisi P. Prez and K. Toyama. Region filling and object removal by exemplar-based inpainting. IEEE Trans. Image Process, 13(9): 1200–1212, Sep 2004.

M. Ashikhmin. Synthesizing natural textures. Proc. ACM Symposium on Interactive 3D Graphics, pages 217–226, March 2001.

A. Efros and W.T. Freeman. Image quilting for texture synthesis and transfer. Proc. ACM Conf. Comp. Graphics (SIGGRAPH), pages 341–346, August 2001.

A. Efros and T. Leung. Texture synthesis by non-parametric sampling. Proc. Int. Conf. Computer Vision, pages 1033–1038, September 1999.

M. Bertalmio L. Vese G. Sapiro and S. Osher. Simultaneous structure and texture image inpainting. Proc. Conf. Comp. Vision Pattern Rec., 2003.

I. Drori D. Cohen-Or and H. Yeshurun. Fragment-based image completion. ACM Trans. on Graphics, 22(3):303–312, 2003.

J. Jia and C.-K. Tang. Image repairing: Robust image synthesis by adaptive nd tensor voting. Proc. Conf. Comp. Vision Pattern Rec., 2003.

Jian Sun. Lu Yuan. Jiaya Jia. and Heung-Yeung Shum. Image completion with structure propagation. ACM Transactions on Graphics (TOG) - Proceedings of ACM SIGGRAPH, 2005.

Jianjun Yang, Yin Wang, Honggang Wang, Kun Hua, Wei Wang, and Ju Shen. Automatica objects removal for scene completion. IEEE INFOCOM Workshop on Security and Privacy in Big Data, Toronto, Canada, 2014.

Pablo Arbeláez. Michael Maire. Charless Fowlkes and Jitendra Malik. Contour detection and hierarchical image segmentation. IEEE Transactions on Pattern Analysis and Machine Intelligence, 33:898–916, May 2011.

Lab. d’Anal. Numerique. Univ. Pierre et Marie Curie. Paris. Disocclusion: a variational approach using level lines. IEEE Transactions on Image Processing, 11:68–76, Feb 2002.

Pablo Arbeláez. Boundary extraction in natural images using ultrametric contour maps. Conference on Computer Vision and Pattern Recognition Workshop, 2006. CVPRW ’06, pages 182–190, Jun 2006.

P. W. Lamberti, A. P. Majtey, A. Borras, M. Casas, and A. Plastino. Metric character of the quantum jensen-shannon divergence. Phys. Rev. A, 77:052311, May 2008.

James McCrae and Karan Singh. Sketching piecewise clothoid curves. Computers and Graphics, 33:452–461, Aug 2009.

Mullinex G and Robinson ST. Fairing point sets using curvature. Computer Aided Design, 39:27–34, 2007.

Downloads

Published

2014-10-15

How to Cite

1.
Shen J, Yang J, Taha-abusneineh S, Payne B, Hitz M. Structure Preserving Large Imagery Reconstruction. JCSANDM [Internet]. 2014 Oct. 15 [cited 2024 Mar. 29];3(3):263-88. Available from: https://journals.riverpublishers.com/index.php/JCSANDM/article/view/6189

Issue

Section

Articles