Abstract
In a Visual Internet of Things (VIoT), the video sequences of different viewpoints are captured by different visual sensors and transmitted simultaneously, which puts a huge burden on storage and bandwidth resources. Mixed-resolution multi-view video format can alleviate the burden on the limited storage and bandwidth resources. However, the low resolution view need to be up-sampled to provide high quality visual experiences to the users. Therefore, a super resolution (SR) algorithm to reconstruct the low resolution view is highly desirable. In this paper, we propose a new two-stage super resolution method. In the first depth-assisted high frequency synthesis stage, depth image based rendering (DIBR) is used to project a high resolution view to a low resolution view to estimate the super resolution result. Then in the second high frequency compensation stage, the local block matching model based on manifold learning is used to enhance the super resolution result. The experimental results demonstrate that our method is capable to achieving a PSNR gain up to 4.76 dB over bicubic baseline and recover details in edge regions, without sacrificing the quality of smooth areas.
Supported by organization NSFC61571326.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Aflaki, P., et al.: Coding of mixed-resolution multiview video in 3D video application. In: 2013 20th IEEE International Conference on Image Processing (ICIP), pp. 1704–1708. IEEE (2013)
Blake, R.: Threshold conditions for binocular rivalry. J. Exp. Psychol. Hum. Percept. Perform. 3(2), 251 (1977)
Brust, H., Smolic, A., Mueller, K., Tech, G., Wiegand, T.: Mixed resolution coding of stereoscopic video for mobile devices. In: 3DTV Conference, pp. 1–4. Citeseer (2009)
Chang, H., Yeung, D.Y., Xiong, Y.: Super-resolution through neighbor embedding. In: Proceedings of the 2004 IEEE Computer Society Conference on Computer Vision and Pattern Recognition CVPR 2004, p. I (2004)
Cruz, M.A.A.D., Rodrigues, J.J.P.C., Al-Muhtadi, J., Korotaev, V., Albuquerque, V.H.C.: A reference model for internet of things middleware. IEEE Internet Things J. 99, 1 (2018)
Fehn, C.: Depth-image-based rendering (DIBR), compression, and transmission for a new approach on 3D-TV. In: Stereoscopic Displays and Virtual Reality Systems XI, vol. 5291, pp. 93–105. International Society for Optics and Photonics (2004)
Garcia, D.C., Dorea, C., de Queiroz, R.L.: Super resolution for multiview images using depth information. IEEE Trans. Circuits Syst. Video Technol. 22(9), 1249–1256 (2012)
Garcia, D.C., Drea, C., Queiroz, R.L.D.: Super-resolution for multiview images using depth information. In: IEEE International Conference on Image Processing, pp. 1793–1796 (2010)
Jain, A.K., Nguyen, T.Q.: Video super-resolution for mixed resolution stereo. In: 2013 20th IEEE International Conference on Image Processing (ICIP), pp. 962–966. IEEE (2013)
Jin, Z., Tillo, T., Yao, C., Xiao, J., Zhao, Y.: Virtual-view-assisted video super-resolution and enhancement. IEEE Trans. Circuits Syst. Video Technol. 26(3), 467–478 (2016)
Merkle, P., Smolic, A., Muller, K., Wiegand, T.: Multi-view video plus depth representation and coding. In: IEEE International Conference on Image Processing. ICIP 2007, vol. 1, pp. 1–201. IEEE (2007)
Roweis, S.T., Saul, L.K.: Nonlinear dimensionality reduction by locally linear embedding. Science 290(5500), 2323 (2000)
Sawhney, H.S., Guo, Y., Hanna, K., Kumar, R., Adkins, S., Zhou, S.: Hybrid stereo camera: an IBR approach for synthesis of very high resolution stereoscopic image sequences. In: Proceedings of the 28th Annual Conference on Computer Graphics and Interactive Techniques. pp. 451–460. ACM (2001)
Sezer, O.B., Dogdu, E., Ozbayoglu, A.M.: Context aware computing, learning and big data in internet of things: a survey. IEEE Internet Things J. 5(1), 1–27 (2017)
Yang, C.Y., Huang, J.B., Yang, M.H.: Exploiting self-similarities for single frame super-resolution. In: Asian Conference on Computer Vision, pp. 497–510 (2010)
Zhang, J., Cao, Y., Wang, Z.: A simultaneous method for 3D video super-resolution and high-quality depth estimation. In: ICIP, pp. 1346–1350 (2013)
Zhang, J., Cao, Y., Zha, Z.J., Zheng, Z., Chen, C.W., Wang, Z.: A unified scheme for super-resolution and depth estimation from asymmetric stereoscopic video. IEEE Trans. Circuits Syst. Video Technol. 26(3), 479–493 (2016)
Zhang, X., Wu, X.: Image interpolation by adaptive 2-D autoregressive modeling and soft-decision estimation. IEEE Trans. Image Process. 17(6), 887–896 (2008)
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2019 ICST Institute for Computer Sciences, Social Informatics and Telecommunications Engineering
About this paper
Cite this paper
Zhou, Y., Wang, Y., Zhang, Y., Du, X., Liu, H., Li, C. (2019). Manifold Learning Based Super Resolution for Mixed-Resolution Multi-view Video in Visual Internet of Things. In: Han, S., Ye, L., Meng, W. (eds) Artificial Intelligence for Communications and Networks. AICON 2019. Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, vol 287. Springer, Cham. https://doi.org/10.1007/978-3-030-22971-9_41
Download citation
DOI: https://doi.org/10.1007/978-3-030-22971-9_41
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-22970-2
Online ISBN: 978-3-030-22971-9
eBook Packages: Computer ScienceComputer Science (R0)