Skip to main content
Log in

Discontinuity-based registration of depth and video data in depth image based rendering

  • Original Paper
  • Published:
Signal, Image and Video Processing Aims and scope Submit manuscript

Abstract

Depth image based rendering (DIBR) has been proposed to create content for 3D-TV. In DIBR, stereoscopic images are created from monoscopic images and associated depth data. Since for most of the available video content sensor depth data are lacking, methods to create artificial depth data for video content have been developed. Yet artificial as well as sensor depth data may contain misalignments with respect to video data. Misaligned depth data are a source of artifacts observable in rendered 3D views. We show that by using an edge-based registration method, the spatial alignment of depth and video data can be improved, leading to an alleviation of the observed artifacts.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Similar content being viewed by others

References

  1. Brox, T., Bruhn, A., Papenberg, N., Weickert, J.: High accuracy optical flow estimation based on a theory for warping. In: Pajdla, T., Matas, J. ECCV (4), Lecture Notes in Computer Science, vol. 3024, pp. 25–36. Springer, New York (2004)

  2. Bruhn, A., Weickert, J., Schnörr, C.: Lucas/kanade meets horn/schunck: combining local and global optic flow methods. International Journal of Computer Vision 61(3), pp. 211–231 (2005). doi:10.1023/B:VISI.0000045324.43199.43. http://www.mia.uni-saarland.de/Publications/bruhn-ijcv05c.pdf

    Google Scholar 

  3. Christensen G.E., Johnson H.J.: Consistent image registration. IEEE Transactions on Medical Imaging 20(7), 568–582 (2001). doi:10.1109/42.932742

    Article  Google Scholar 

  4. Droske, M., Ring, W.: A mumford-shah level-set approach for geometric image registration. SIAM journal on applied mathematics 66(6), 2127–2148 (2006). http://citeseerx.ist.psu.edu/viewdoc/summary?doi=10.1.1.61.8563

  5. Droske, M., Ring, W., Rumpf, M.: Mumfordshah based registration: a comparison of a level set and a phase field approach. Computing and Visualization in Science Online first (2008). doi:10.1007/s00791-008-0084-2

  6. Fehn, C., de la Barré, R., Pastoor, S.: Interactive 3DTV – Concepts and key technologies. In: Proceedings of the IEEE 94(3), pp. 524–538 (2006)

  7. Fieseler, M., Jiang, X.: Registration of depth and video data in depth image based rendering. In: 3DTV Conference: The True Vision - Capture, Transmission and Display of 3D Video, 2009, pp. 1–4 (2009). doi:10.1109/3DTV.2009.5069677

  8. Han J., Berkels B., Droske M., Hornegger J., Rumpf M., Schaller C., Scorzin J., Urbach H.: Mumford-Shah model for one-to-one edge matching. IEEE Trans. Image Process. 16(11), 2720–2732 (2007). doi:10.1109/TIP.2007.906277

    Article  MathSciNet  Google Scholar 

  9. Jiang, X., Bunke, H.: Edge detection in range images based on scan line approximation. Comput. Vis. Image Unders. 73(2), 183–199 (1999). http://www.ingentaconnect.com/content/ap/iv/1999/00000073/00000002/art00715

  10. Jiang, X., Lambers, M.: DIBR-based 3D videos using non video rate range image stream. In: Proceedings of International Conference on Multimedia & Expo, pp. 1873–1876 (2006). doi:10.1109/ICME.2006.262920

  11. Li H., Manjunath B.S., Mitra S.K.: A contour-based approach to multisensor image registration. IEEE Trans. Image Process. 4(3), 320–334 (2002). doi:10.1109/83.366480

    Article  Google Scholar 

  12. Maintz , van den Elsen P.A., Viergever M.A.: Comparison of edge-based and ridge-based registration of CT and MR brain images. Med. Image Anal. 1(2), 151–161 (1996). doi:10.1016/S1361-8415(96)80010-7

    Article  Google Scholar 

  13. Min J., Powell M., Bowyer K.W.: Automated performance evaluation of range image segmentation algorithms. IEEE Trans. Syst. Man Cybern. B 34(1), 263–271 (2004). doi:10.1109/TSMCB.2003.811118

    Article  Google Scholar 

  14. Modersitzki J.: Fair: Flexible Algorithms for Image Registration (Fundamentals of Algorithms). Society for Industrial and Applied Mathematics, Philadelphia (2009)

    MATH  Google Scholar 

  15. Rothaus, S., Rothaus, K., Jiang, X.: Synthesizing 3D videos by a motion-conditioned background mosaic. In: Proceedings of International Conference on Pattern Recognition (2008)

  16. Viola P., Wells W.M. III: Alignment by maximization of mutual information. Int. J. Comput. Vis. 24(2), 137–154 (1997). doi:10.1023/A:1007958904918

    Article  Google Scholar 

  17. Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S., Szeliski, R.: High-quality video view interpolation using a layered representation. In: SIGGRAPH ’04, pp. 600–608. ACM, New York, NY, USA (2004). doi:10.1145/1186562.1015766

  18. Zitova B., Flusser J.: Image registration methods: a survey. Image Vis. Comput. 21(11), 977–1000 (2003). doi:10.1016/S0262-8856(03)00137-9

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Xiaoyi Jiang.

Rights and permissions

Reprints and permissions

About this article

Cite this article

Fieseler, M., Jiang, X. Discontinuity-based registration of depth and video data in depth image based rendering. SIViP 5, 353–361 (2011). https://doi.org/10.1007/s11760-010-0199-z

Download citation

  • Received:

  • Revised:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11760-010-0199-z

Keywords

Navigation