Real-time correction of panoramic images using hyperbolic Möbius transformations

Abstract

Wide-angle images gained a huge popularity in the last years due to the development of computational photography and imaging technological advances. They present the information of a scene in a way which is more natural for the human eye but, on the other hand, they introduce artifacts such as bent lines. These artifacts become more and more unnatural as the field of view increases. In this work, we present a technique aimed to improve the perceptual quality of panorama visualization. The main ingredients of our approach are, on one hand, considering the viewing sphere as a Riemann sphere, what makes natural the application of Möbius (complex) transformations to the input image, and, on the other hand, a projection scheme which changes in function of the field of view used. We also introduce an implementation of our method, compare it against images produced with other methods and show that the transformations can be done in real time, which makes our technique very appealing for new settings, as well as for existing interactive panorama applications.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12

Notes

  1. 1.

    For us, real time means that the computations are done quicker than the rendering of a frame in the application. Even if the other methods are fast, they are not suitable to be implemented in real time.

  2. 2.

    Recall that, in our method, the viewing sphere is rotated to make coincide the north pole with the center of stretching, and rotated back after the stretching. For simplicity, we will assume here that the shrinking occurs with the north pole as center, without considering rotations.

References

  1. 1.

    D’Angelo, P., et al.: Hugin—panorama photo stitcher, version 2014.0.0. http://hugin.sourceforge.net. Accessed 24 March 2015

  2. 2.

    Pfeil, J., Hildebrand, K., Gremzow, C., Bickel, B., Alexa, M.: Throwable panoramic ball camera. In: SIGGRAPH Asia 2011 Emerging Technologies, SA ’11, pp. 4:1. ACM (2011)

  3. 3.

    Kingslake, R.: A History of the Photographic Lens. Academic Press, London (1989)

    Google Scholar 

  4. 4.

    Zorin, D., Barr, A.H.: Correction of geometric perceptual distortions in pictures. In: 22nd Annual Conference on Computer Graphics and Interactive Techniques, SIGGRAPH ’95, pp. 257–264, New York. ACM (1995)

  5. 5.

    Carroll, R., Agrawal, M., Agarwala, A.: Optimizing content-preserving projections for wide-angle images. ACM Trans. Graph. 28(3), 43:1–43:9 (2009)

    Article  Google Scholar 

  6. 6.

    Sacht, L.: Content-based projections for panoramic images and videos. Master’s thesis, IMPA (2010)

  7. 7.

    Sacht, L., Velho, L.: Complex plane transformations for manipulation and visualization of panoramas. In: International Conference on Computer Graphics Theory and Applications (GRAPP 2013) (2013)

  8. 8.

    Zelnik-Manor, L., Peters, G., Perona, P.: Squaring the circles in panoramas. In: 10th IEEE International Conference on Computer Vision, vol. 2, ICCV ’05, pp. 1292–1299, Washington. IEEE (2005)

  9. 9.

    Sharpless, T.K., Postle, B., Germán, D.M.: Pannini: a new projection for rendering wide angle perspective images. In: 6th International Conference on Computational Aesthetics in Graphics, Visualization and Imaging, Computational Aesthetics’10, pp. 9–16. Aire-la-Ville, Switzerland. Eurographics Association (2010)

  10. 10.

    Germán, D.M., Burchill, L., Duret-Lutz, A., Pérez-Duarte, S., Pérez-Duarte, E., Sommers, J.: Flattening the viewable sphere. In: 3rd Eurographics Conference on Computational Aesthetics in Graphics, Visualization and Imaging, Computational Aesthetics ’07, pp. 23–28. Eurographics (2007)

  11. 11.

    Kopf, J., Lischinski, D., Deussen, O., Cohen-Or, D., Cohen, M.: Locally adapted projections to reduce panorama distortions. Comput. Graph. Forum 28(4), 1083–1089 (2009)

    Article  Google Scholar 

  12. 12.

    Wei, J., Li, C.-F., Shi-Min, H., Martin, R.R., Tai, C.-L.: Fisheye video correction. IEEE Trans. Vis. Comput. Graph. 18(10), 1771–1783 (2012)

    Article  Google Scholar 

  13. 13.

    Kopf, J., Uyttendaele, M., Deussen, O., Cohen, M.F.: Capturing and viewing gigapixel images. ACM Trans. Graph. 26(3) (2007)

  14. 14.

    Snyder, J.P.: Map Projections-A Working Manual, vol. 1395 of Geological Survey Bulletin Series. U.S. G.P.O. (1987)

  15. 15.

    Needham, T.: Visual Complex Analysis. Clarendon Press (1997)

  16. 16.

    Cambridge in colour: Cameras vs. the human eye. http://www.cambridgeincolour.com/tutorials/cameras-vs-human-eye.htm. Accessed 24 Sept 2013

  17. 17.

    Ware, C.: Information Visualization: Perception for Design, 2nd edn. Morgan Kaufman, Massachusetts (2004)

    Google Scholar 

  18. 18.

    Milnor, J.: A problem in cartography. Am. Math. Mon. 76(10), 1101–1112 (1969)

    MathSciNet  Article  Google Scholar 

  19. 19.

    Stein, W.A., et al.: Sage Mathematics Software (Version 5.7). The Sage Development Team (2013). http://www.sagemath.org. Accessed 9 Sept 2013

  20. 20.

    Dersch, H., et al.: Panorama tools, version 2.9.19. http://panotools.sourceforge.net. Accessed 24 March 2015

  21. 21.

    Sharpless, T.: Panini perspective tool, version 0.71.104. http://pvqt.sourceforge.net. Accessed 24 March 2015

  22. 22.

    Google street view. http://www.google.com/streetview. Accessed 24 March 2015

  23. 23.

    Anguelov, D., Dulong, C., Filip, D., Frueh, C., Lafon, S., Lyon, R., Ogale, A., Vincent, L., Weaver, J.: Google street view: capturing the world at street level. Computer 43 (2010)

  24. 24.

    Bourke, P.D.: Spherical mirror: a new approach to hemispherical dome projection. In: 3rd International Conference on Computer Graphics and Interactive Techniques in Australasia and Southeast Asia, pp. 281–284. ACM (2005)

  25. 25.

    Bourke, P.D., Felinto, D.Q.: Blender and immersive gaming in a hemispherical dome. In: Computer Games, Multimedia and Allied Technology Conference (2010)

Download references

Acknowledgments

The authors thank the Flickr and Wikimedia Commons users who made available their equirectangular images under the Creative Commons license, used to obtain some figures of the paper: Gadl (Figs. 6, 10, 11, 12), Luca Biada (Fig. 7), DXR (Fig. 8) and HamburgerJung (Fig. 9). L. Sacht acknowledges the doctoral scholarship from CNPq. L. Peñaranda acknowledges financial support from IMPA during years 2012 to 2014.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Luis Peñaranda.

Electronic supplementary material

Below is the link to the electronic supplementary material.

Supplementary material 1 (mp4 23413 KB)

Appendix: Möbius transformations

Appendix: Möbius transformations

We recall in this appendix some properties of Möbius transformations. The information here should be enough to understand the paper. We refer to the book by Needham [15] for proofs and details.

A Möbius transformation is a mapping of the form

$$\begin{aligned} M(z)=\frac{az+b}{cz+d}, \end{aligned}$$

where \(a\), \(b\), \(c\) and \(d\) are complex constants. Multiplying these coefficients by a constant yields the same mapping, thus what matter are the ratios of the coefficients (usually the coefficients are normalized by scaling them to satisfy \(ad-bc=1\)). This fact shows that only three complex numbers are sufficient to determine uniquely a mapping. It can be shown that there exist a unique Möbius transformation which sends any three points to any other three points.

Fig. 13
figure13

Four kinds of Möbius transformations (drawings taken from [15]). a Elliptic, b hyperbolic, c loxodromic, d parabolic

Möbius transformations have fixed points, computed through the equation \(z=M(z)\). Since this equation is quadratic, it has at most two solutions (except for the identity mapping). When \(c\ne 0\), both fixed points lie in the finite plane. When \(c=0\), at least one of the fixed points lies at infinity (as it can be deduced from Fig. 13, infinity is represented by the north pole \(N\) of the Riemann sphere). In this case, the transformation is simplified, taking the form \(M(z)=Az+B\). We can write \(A=\rho e^{i\alpha }\) to view \(M(z)\) as the composition of a rotation of \(\alpha \) centered in the origin, plus an expansion by \(\rho \) and a translation of \(B\). This interpretation lets us view geometrically the Möbius transformation, as depicted in Fig. 13.

When \(\alpha >0\), \(\rho =1\) and \(B=0\), \(M(z)\) is a rotation of the complex plane (i.e., a rotation of the sphere), as shown if Fig. 13a. The fixed points of this transformation are the two poles of the sphere, which correspond, in the complex plane, to the origin and infinity. This is called an elliptic Möbius transformation.

Figure 13b illustrates the transformation when \(\alpha =0\), \(\rho >1\) and \(B=0\). \(M(z)\) is, in this case, an expansion centered in the origin. The two fixed points are the same as in the previous case. If \(\alpha =0\), \(\rho <1\) and \(B=0\), it is an origin-centered contraction. These Möbius transformations are called hyperbolic.

When \(\alpha \ne 0\), \(\rho \ne 1\) and \(B=0\), the resulting transformation is a combination of the two former cases. This is called a loxodromic Möbius transform and it is illustrated in Fig. 13c. The two fixed points of this transform are the same as the above cases.

The last case study is the translation, occurring when \(A=0\) and \(B\ne 0\). The only fixed point of this transformation is infinity, represented on the sphere by the north pole. This transformation, the parabolic Möbius transformation, is depicted in Fig. 13d.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Peñaranda, L., Velho, L. & Sacht, L. Real-time correction of panoramic images using hyperbolic Möbius transformations. J Real-Time Image Proc 15, 725–738 (2018). https://doi.org/10.1007/s11554-015-0502-x

Download citation

Keywords

  • Panorama
  • Perspective projection
  • Möbius transformation
  • Real time
  • Implementation