A Framework for Augmented Reality using Non-Central Catadioptric Cameras


This paper addresses the problem of augmented reality on images acquired from non-central catadioptric systems. We propose a solution which allows the projection of textured objects to images of these type of systems and, depending on the complexity of the objects, can run up to 20 fps, using a 1328×1048 image resolution. The main contributions are related with the image formation of the non-central catadioptric cameras: projection of the 3D segments onto the image of non-central catadioptric cameras; occlusions; and illumination/shading. To validate the proposed solution, we used a non-central catadioptric camera formed with a perspective camera and a spherical mirror. Also, to test the robustness of the proposed method, we used a regular object (a parallelepiped) and three well known irregular objects in computer graphics: “bunny”, “happy buddha” and “dragon”, from Stanford database.

This is a preview of subscription content, access via your institution.


  1. 1.

    Appel, A.: Some techniques for shading machine renderings of solids. In: Proceeding of the AFIPS (1968)

  2. 2.

    Azuma, R.T.: A survey of augmented reality. In: Presence: Teleoperators and virtual environments: MIT Press Journal (1997)

  3. 3.

    Fournier, A., Gunawan, A.S., Romanzin, C.: Common illumination between real and computer generated scenes. In: Proceeding of Graphics Interface (GI’93) (1993)

  4. 4.

    Sato, I., Sato, Y., Ikeuchi, K.: Acquiring a radiance distribution to superimpose virtual objects onto a real scene. IEEE Trans. Vis. Comput. Graph. 5(1), 1–12 (1999)

    Article  Google Scholar 

  5. 5.

    Debevec, P.: Rendering synthetic objects into real scenes: bridging traditional and image-based graphics with global illumination and high dynamic range photography. ACM SIGGRAPH 2008 classes. ACM, p. 32 (2008)

  6. 6.

    Santos, A.L., Lemos, D., Lindoso, J.E.F., Teichrieb, V.: Real time ray tracing for augmented reality. IEEE Symposium on Virtual and Augmented Reality (SVR) (2012)

  7. 7.

    Grossberg, M.D., Nayar, S.K.: A general imaging model and a method for finding its parameters. In: IEEE Proceedings of the International Conference on Computer Vision (ICCV) (2003)

  8. 8.

    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. Cambridge University Press (2000)

  9. 9.

    Swaminathan, R., Grossberg, M.D., Nayar, S.K.: A perspective on distortions. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2003)

  10. 10.

    Nalwa, V.S.: A True Omni-Directional Viewer. Technichal report, Bell Laboratories (1996)

  11. 11.

    Nayar, S.K., Baker, S.: Catadioptric image formation. In: DARPA Proceedings of the Image Understanding Workshop (1997)

  12. 12.

    Baker, S., Nayar, S.K.: A theory of single-viewpoint catadioptric image formation. Int. J. Comput. Vis. 35(2), 175–196 (1999)

    Article  Google Scholar 

  13. 13.

    Swaminathan, R., Grossberg, M.D., Nayar, S.K.: Caustics of catadioptric cameras. In: IEEE Proceedings of the International Conference on Computer Vision (ICCV) (2001)

  14. 14.

    Micusik, B., Pajdla, T.: Autocalibration & 3D reconstruction with non-central catadioptric cameras. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2004)

  15. 15.

    Gonçalves, N.: Noncentral Catadioptric Systems with Quadric Mirrors: Geometry and Calibration. Ph.D. dissertation, University of Coimbra (2008)

  16. 16.

    Perdigoto, L., Araujo, H.: Calibration of mirror position and extrinsic parameters in axial non-central catadioptric systems. Comput. Vis. Image Underst. 117(8), 909–921 (2013)

    Article  Google Scholar 

  17. 17.

    Agrawal, A., Ramalingam, S.: Single image calibration of multi-axial imaging systems. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2013)

  18. 18.

    Goodrich, M.A., Schultz, A.C.: Human-Robot Interaction: a Survey. Foundations and Trends in Human-Computer Interaction (2007)

  19. 19.

    Chintamani, K., Cao, A., Ellis, R.D., Pandya, A.K.: Improved telemanipulator navigation during display-control misalignments using augmented reality cues. IEEE Trans. Syst. Man Cybern. Part A Syst. Humans 40(1), 29–39 (2010)

    Article  Google Scholar 

  20. 20.

    Chen, I. Y.-H., MacDonald, B., Wunsche, B.: Mixed reality simulation for mobile robots. In: IEEE Proceedings of the International Conference on Robotics and Automation (ICRA) (2009)

  21. 21.

    Fuchs, H., Livingston, M.A., Raskar, R., Colucci, D., Keller, K., State, A., Crawford, J.R., Rademacher, P., Drake, S.H., Meyer, A.A.: Augmented reality visualization for laparoscopic surgery. In: International Conference on Medical Image Computing and Computer Assisted Intervention (MICCAI) (2009)

  22. 22.

    Gonçalves, N.: On the reflection point where light reflects to a known destination in quadric surfaces. Opt. Lett. 35(2), 100–102 (2010)

    Article  Google Scholar 

  23. 23.

    Agrawal, A., Taguchi, Y., Ramalingam, S.: Beyond alhazen’ Problem: Analytical Projection Model for Non-Central Catadioptric Cameras with Quadric Mirrors. In: IEEE Proceedings of the Computer Vision and Pattern Recognition (CVPR) (2011)

  24. 24.

    Hughes, J., Dam, A.V., Mcguire, M., Skylar, D.F., Foley, J.D., Feiner, S.K., Akeley, K.: Computer Graphics: Principles and Practice, 3rd Ed. Addison-Wesley (2014)

  25. 25.

    Carpenter, L.: The A-buffer, an antialiased hidden surface method. ACM SIGGRAPH 18(3), 103–108 (1984)

    MathSciNet  Article  Google Scholar 

  26. 26.

    Gouraud, H.: Continuous shading of curved surfaces. IEEE Trans. Comput. 100(6), 623–629 (1971)

    Article  MATH  Google Scholar 

  27. 27.

    Phong, B.T.: Illumination for Computer Generated Pictures. Commun. ACM 18(6), 311–317 (1975)

    Article  Google Scholar 

  28. 28.

    Stanford University Computer Graphics Laboratory, Stanford Bunny, https://graphics.stanford.edu/data/3Dscanrep/ (1993)

  29. 29.

    Delaunay, B.: Sur la sphere vide. Izv. Akad. Nauk SSSR, Otdelenie Matematicheskii i Estestvennyka Nauk 7(793-800), 1–2 (1934)

    MATH  Google Scholar 

  30. 30.

    Chen, C.-S., Chang, W.-Y.: On pose recovery for generalized visual sensors. IEEE Trans. Pattern Anal. Mach. Intell. 26(7), 848–861 (2004)

    Article  Google Scholar 

  31. 31.

    Schweighofer, G., Pinz, A.: Globally Optimal O(n) Solution to the PnP Problem for General Camera Models. In: Proceedings of the British Machine Vision Conference (BMVC) (2008)

  32. 32.

    Miraldo, P., Araujo, H.: Pose estimation for non-central cameras using planes. In: IEEE International Conference on Autonomous Robot Systems & Competitions –ROBÓTICA (2014)

  33. 33.

    Miraldo, P., Araujo, H.: Planar pose estimation for general cameras using known 3D lines. In: IEEE/RSJ Proceedings of the International Conference on Intelligent Robots & Systems (IROS) (2014)

  34. 34.

    MobileRobots, Inc., Pioneer 3-DX, . http://www.mobilerobots.com/Mobile_Robots.aspx http://www.mobilerobots.com/Mobile_Robots.aspx

Download references

Author information



Corresponding author

Correspondence to Pedro Miraldo.

Electronic supplementary material

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Dias, T., Miraldo, P. & Gonçalves, N. A Framework for Augmented Reality using Non-Central Catadioptric Cameras. J Intell Robot Syst 83, 359–373 (2016). https://doi.org/10.1007/s10846-016-0349-9

Download citation


  • Augmented reality
  • Non-central catadioptric cameras
  • Forward-projection