Skip to main content
View expanded cover

German Conference on Pattern Recognition

GCPR 2014: Pattern Recognition pp 305–316Cite as

Spatial and Temporal Interpolation of Multi-view Image Sequences

Part of the Lecture Notes in Computer Science book series (LNIP,volume 8753)

Abstract

We propose a simple and effective framework for multi-view image sequence interpolation in space and time. For spatial view point interpolation we present a robust feature-based matching algorithm that allows for wide-baseline camera configurations. To this end, we introduce two novel filtering approaches for outlier elimination and a robust approach for match extrapolations at the image boundaries. For small-baseline and temporal interpolations we rely on an established optical flow based approach. We perform a quantitative and qualitative evaluation of our framework and present applications and results. Our method has a low runtime and results can compete with state-of-the-art methods.

Keywords

  • Optical Flow
  • Image Boundary
  • Image Interpolation
  • Temporal Interpolation
  • Epipolar Constraint

These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

This work was supported by the ERC Starting Grant “Convex Vision”.

This is a preview of subscription content, access via your institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-319-11752-2_24
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   84.99
Price excludes VAT (USA)
  • ISBN: 978-3-319-11752-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   109.99
Price excludes VAT (USA)
Learn about institutional subscriptions
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.

Notes

  1. 1.

    http://www.tobiasgurdan.de/vision/imageinterpolation/

References

  1. Ballan, L., Brostow, G.J., Puwein, J., Pollefeys, M.: Unstructured video-based rendering: interactive exploration of casually captured videos. ACM Trans. Graph. 29(4) (2010). http://dblp.uni-trier.de/db/journals/tog/tog29.html#BallanBPP10

  2. Chen, K., Lorenz, D.A.: Image sequence interpolation based on optical flow, segmentation, and optimal control. IEEE Trans. Image Process. 21(3), 1020–1030 (2012). http://dblp.uni-trier.de/db/journals/tip/tip21.html#ChenL12

    MathSciNet  CrossRef  Google Scholar 

  3. Debevec, P.: The campanile movie. In: SIGGRAPH 97 Electronic Theater (1997). http://www.debevec.org/Campanile/ (visited: May 2014)

  4. Fehn, C.: Depth-Image-Based Rendering (DIBR), compression and transmission for a new approach on 3D-TV. In: Proceedings of SPIE Stereoscopic Displays and Virtual Reality Systems XI, pp. 93–104 (2004)

    Google Scholar 

  5. Fischler, M., Bolles, R.: Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun. ACM 24(6), 381–395 (1981)

    MathSciNet  CrossRef  Google Scholar 

  6. Fragneto, P., Fusiello, A., Rossi, B., Magri, L., Ruffini, M.: Uncalibrated view synthesis with homography interpolation. In: 3DIMPVT, pp. 270–277. IEEE (2012). http://dblp.uni-trier.de/db/conf/3dim/3dimpvt2012.html#FragnetoFRMR12

  7. Germann, M., Hornung, A., Keiser, R., Ziegler, R., Würmlin, S., Gross, M.: Articulated billboards for video-based rendering. Comput. Graph. Forum (Proc. Eurographics) 29(2), 585–594 (2010)

    CrossRef  Google Scholar 

  8. Goesele, M., Ackermann, J., Fuhrmann, S., Haubold, C., Klowsky, R., Steedly, D., Szeliski, R.: Ambient point clouds for view interpolation. ACM Trans. Graph. 29(4), 95:1–95:6 (2010). http://doi.acm.org/10.1145/1778765.1778832

    CrossRef  Google Scholar 

  9. Hasler, N., Rosenhahn, B., Thormählen, T., Wand, M., Gall, J., Seidel, H.P.: Markerless motion capture with unsynchronized moving cameras. In: CVPR, pp. 224–231 (2009)

    Google Scholar 

  10. Inamoto, N., Saito, H.: Free viewpoint video synthesis and presentation from multiple sporting videos. In: ICME, pp. 322–325. IEEE (2005). http://dblp.uni-trier.de/db/conf/icmcs/icme2005.html#InamotoS05

  11. Lipski, C.: Virtual video camera: a system for free viewpoint video of arbitrary dynamic scenes. Ph.D. thesis, TU Braunschweig, June 2013

    Google Scholar 

  12. Lipski, C., Linz, C., Berger, K., Magnor, M.A.: Virtual video camera: image-based viewpoint navigation through space and time. In: SIGGRAPH Posters. ACM (2009). http://dblp.uni-trier.de/db/conf/siggraph/siggraph2009posters.html#LipskiLBM09

  13. Lowe, D.G.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 20, 91–110 (2003)

    Google Scholar 

  14. Mahajan, D., Huang, F.C., Matusik, W., Ramamoorthi, R., Belhumeur, P.N.: Moving gradients: a path-based method for plausible image interpolation. ACM Trans. Graph. 28(3), 42:1–42:11 (2009). doi:10.1145/1531326.1531348

    CrossRef  Google Scholar 

  15. Morel, J.M., Yu, G.: Asift: a new framework for fully affine invariant image comparison. SIAM J. Imaging Sci. 2(2), 438–469 (2009). http://dblp.uni-trier.de/db/journals/siamis/siamis2.html#MorelY09

    MathSciNet  CrossRef  MATH  Google Scholar 

  16. Muja, M., Lowe, D.G.: Fast approximate nearest neighbors with automatic algorithm configuration. In: Ranchordas, A., Arajo, H. (eds.) VISAPP (1), pp. 331–340. INSTICC Press (2009). http://dblp.uni-trier.de/db/conf/visapp/visapp2009-1.html#MujaL09

  17. Replay Technologies Inc.: freeD\(^{\rm {TM}}\) technology (2013). http://replay-technologies.com/ (visited: May 2014)

  18. Scharstein, D., Pal, C.: Learning conditional random fields for stereo. In: CVPR (2007)

    Google Scholar 

  19. Seitz, S.M., Dyer, C.R.: Physically-valid view synthesis by image interpolation. In: Proceedings of the IEEE Workshop on Representations of Visual Scenes, pp. 18–25 (1995)

    Google Scholar 

  20. Seitz, S.M., Dyer, C.R.: View morphing. In: SIGGRAPH, pp. 21–30 (1996). http://dblp.uni-trier.de/db/conf/siggraph/siggraph1996.html#SeitzD96

  21. Snavely, N., Garg, R., Seitz, S.M., Szeliski, R.: Finding paths through the world’s photos. ACM Trans. Graph. (Proceedings of SIGGRAPH 2008) 27(3), 11–21 (2008)

    Google Scholar 

  22. Strecha, C., Tuytelaars, T., Gool, L.J.V.: Dense matching of multiple wide-baseline views. In: ICCV, pp. 1194–1201 (2003)

    Google Scholar 

  23. Vedula, S., Baker, S., Kanade, T.: Image-based spatio-temporal modeling and view interpolation of dynamic events. ACM Trans. Graph. 24(2), 240–261 (2005)

    CrossRef  Google Scholar 

  24. Vlad, A.: Image morphing techniques. JIDEG 5(1) (2010). http://www.sorging.ro/ro/member/serveFile/format/pdf/slug/image-morphing-techniques

  25. Warn, S., Apon, A., Cothren, J.: Accelerating sift on hybrid clusters. In: Proceedings of the ACM SIGSPATIAL Second International Workshop on High Performance and Distributed Geographic Information Systems, HPDGIS ’11, pp. 2–9. ACM, New York (2011). http://doi.acm.org/10.1145/2070770.2070771

  26. Werlberger, M., Pock, T., Unger, M., Bischof, H.: Optical flow guided tv-l1 video interpolation and restoration. In: Energy Minimization Methods in Computer Vision and Pattern Recognition (2011)

    Google Scholar 

  27. Wolberg, G.: Image morphing: a survey. Vis. Comput. 14, 360–372 (1998). http://ci.nii.ac.jp/naid/80010827845/en/

    CrossRef  Google Scholar 

  28. Zhang, Z., Deriche, R., Faugeras, O.D., Luong, Q.T.: A robust technique for matching two uncalibrated images through the recovery of the unknown epipolar geometry. Artif. Intell. 78(1–2), 87–119 (1995). http://dblp.uni-trier.de/db/journals/ai/ai78.html#ZhangDFL95

    CrossRef  Google Scholar 

  29. Zitnick, C.L., Kang, S.B., Uyttendaele, M., Winder, S.A.J., Szeliski, R.: High-quality video view interpolation using a layered representation. ACM Trans. Graph. 23(3), 600–608 (2004). http://dblp.uni-trier.de/db/journals/tog/tog23.html#ZitnickKUWS04

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Department of Computer Science, Technische Universität München, Munich, Germany

    Tobias Gurdan, Martin R. Oswald & Daniel Cremers

  2. Ascending Technologies GmbH, Krailing, Germany

    Tobias Gurdan & Daniel Gurdan

Authors
  1. Tobias Gurdan
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Martin R. Oswald
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Daniel Gurdan
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Daniel Cremers
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Tobias Gurdan .

Editor information

Editors and Affiliations

  1. Department of Mathematics and Computer Science, University of Münster, Münster, Germany

    Xiaoyi Jiang

  2. Computer Science Department 5, University of Erlangen-Nürnberg, Erlangen, Germany

    Joachim Hornegger

  3. Department of Computer Science, University of Kiel, Kiel, Germany

    Reinhard Koch

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer International Publishing Switzerland

About this paper

Cite this paper

Gurdan, T., Oswald, M.R., Gurdan, D., Cremers, D. (2014). Spatial and Temporal Interpolation of Multi-view Image Sequences. In: Jiang, X., Hornegger, J., Koch, R. (eds) Pattern Recognition. GCPR 2014. Lecture Notes in Computer Science(), vol 8753. Springer, Cham. https://doi.org/10.1007/978-3-319-11752-2_24

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-11752-2_24

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-11751-5

  • Online ISBN: 978-3-319-11752-2

  • eBook Packages: Computer ScienceComputer Science (R0)