Abstract
Depth image based rendering (DIBR) has been proposed to create content for 3D-TV. In DIBR, stereoscopic images are created from monoscopic images and associated depth data. Since for most of the available video content sensor depth data are lacking, methods to create artificial depth data for video content have been developed. Yet artificial as well as sensor depth data may contain misalignments with respect to video data. Misaligned depth data are a source of artifacts observable in rendered 3D views. We show that by using an edge-based registration method, the spatial alignment of depth and video data can be improved, leading to an alleviation of the observed artifacts.
Similar content being viewed by others
References
Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. ECCV (4), Lecture Notes in Computer Science, vol. 3024, pp. 25–36. Springer, New York (2004)
Bruhn, A., Weickert, J., Schnörr, C.: Lucas/kanade meets horn/schunck: combining local and global optic flow methods. International Journal of Computer Vision 61(3), pp. 211–231 (2005). doi:10.1023/B:VISI.0000045324.43199.43. http://www.mia.uni-saarland.de/Publications/bruhn-ijcv05c.pdf
Christensen G.E., Johnson H.J.: Consistent image registration. IEEE Transactions on Medical Imaging 20(7), 568–582 (2001). doi:10.1109/42.932742
Droske, M., Ring, W.: A mumford-shah level-set approach for geometric image registration. SIAM journal on applied mathematics 66(6), 2127–2148 (2006). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.61.8563
Droske, M., Ring, W., Rumpf, M.: Mumfordshah based registration: a comparison of a level set and a phase field approach. Computing and Visualization in Science Online first (2008). doi:10.1007/s00791-008-0084-2
Fehn, C., de la Barré, R., Pastoor, S.: Interactive 3DTV – Concepts and key technologies. In: Proceedings of the IEEE 94(3), pp. 524–538 (2006)
Fieseler, M., Jiang, X.: Registration of depth and video data in depth image based rendering. In: 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, 2009, pp. 1–4 (2009). doi:10.1109/3DTV.2009.5069677
Han J., Berkels B., Droske M., Hornegger J., Rumpf M., Schaller C., Scorzin J., Urbach H.: Mumford-Shah model for one-to-one edge matching. IEEE Trans. Image Process. 16(11), 2720–2732 (2007). doi:10.1109/TIP.2007.906277
Jiang, X., Bunke, H.: Edge detection in range images based on scan line approximation. Comput. Vis. Image Unders. 73(2), 183–199 (1999). http://www.ingentaconnect.com/content/ap/iv/1999/00000073/00000002/art00715
Jiang, X., Lambers, M.: DIBR-based 3D videos using non video rate range image stream. In: Proceedings of International Conference on Multimedia & Expo, pp. 1873–1876 (2006). doi:10.1109/ICME.2006.262920
Li H., Manjunath B.S., Mitra S.K.: A contour-based approach to multisensor image registration. IEEE Trans. Image Process. 4(3), 320–334 (2002). doi:10.1109/83.366480
Maintz , van den Elsen P.A., Viergever M.A.: Comparison of edge-based and ridge-based registration of CT and MR brain images. Med. Image Anal. 1(2), 151–161 (1996). doi:10.1016/S1361-8415(96)80010-7
Min J., Powell M., Bowyer K.W.: Automated performance evaluation of range image segmentation algorithms. IEEE Trans. Syst. Man Cybern. B 34(1), 263–271 (2004). doi:10.1109/TSMCB.2003.811118
Modersitzki J.: Fair: Flexible Algorithms for Image Registration (Fundamentals of Algorithms). Society for Industrial and Applied Mathematics, Philadelphia (2009)
Rothaus, S., Rothaus, K., Jiang, X.: Synthesizing 3D videos by a motion-conditioned background mosaic. In: Proceedings of International Conference on Pattern Recognition (2008)
Viola P., Wells W.M. III: Alignment by maximization of mutual information. Int. J. Comput. Vis. 24(2), 137–154 (1997). doi:10.1023/A:1007958904918
Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. In: SIGGRAPH ’04, pp. 600–608. ACM, New York, NY, USA (2004). doi:10.1145/1186562.1015766
Zitova B., Flusser J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003). doi:10.1016/S0262-8856(03)00137-9
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Fieseler, M., Jiang, X. Discontinuity-based registration of depth and video data in depth image based rendering. SIViP 5, 353–361 (2011). https://doi.org/10.1007/s11760-010-0199-z
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11760-010-0199-z