Abstract
We design a novel “folded” spherical catadioptric rig (formed by two coaxially-aligned spherical mirrors of distinct radii and a single perspective camera) to recover near-spherical range panoramas (about 360° × 153°) from the fusion of depth given by optical flow and stereoscopy. We observe that for rigid motion that is parallel to a plane, optical flow and stereo generate nearly complementary distributions of depth resolution. While optical flow provides strong depth cues in the periphery and near the poles of the view-sphere, stereo generates reliable depth in a narrow band about the equator instead. We exploit this dual-modality principle by modeling (separately) the depth resolution of optical flow and stereo in order to fuse them later on a probabilistic spherical panorama. We achieve a desired vertical field-of-view and optical resolution by deriving a linearized model of the rig in terms of three parameters (radii of the two mirrors plus axial distance between the mirrors’ centers). We analyze the error due to the violation of the single viewpoint constraint and formulate additional constraints on the design to minimize this error. We evaluate our proposed method via a synthetic model and with real-world prototypes by computing dense spherical panoramas of depth from cluttered indoor environments after fusing the two modalities (stereo and optical flow).
Similar content being viewed by others
References
Conroy J., Gremillion G., Ranganathan B., Humbert J.S.: Implementation of wide-field integration of optic flow for autonomous quadrotor navigation. Auton. Robots 27(3), 189 (2009)
Burbridge, C., Spacek, L., Park, W.: Omnidirectional vision simulation and robot localisation. In: Proceedings of TARO, pp. 32–39 (2006)
Kim J.H., Chung M.J.: SLAM with omni-directional stereo vision sensor. Intell. Robots Syst. (2003). Nevada, Las Vegas (2003)
Kuthirummal S., Nayar S.K.: Multiview radial catadioptric imaging for scene capture. Intell. Robots Syst. ACM Trans. Graph. 25(3), 916 (2006)
Baker, S., Nayar, S.: In: A theory of catadioptric image formation. pp. 35–42 (1998)
Nayar, S.: In: Omnidirectional vision. vol. 8, pp. 195–202 (1998)
Su, L., Luo, C., Zhu, F.: In: Obtaining obstacle information by an omnidirectional stereo vision system. pp. 48–52 (2006)
Corrêa, F.R., Guizilini, V.C., Junior, J.O.: Omnidirectional stereovision system with two-lobe hyperbolic mirror for robot navigation. ABCM Symp. Series Mech. 2, 653 (2006)
Nayar, S.K., Peri, V.: In: Folded catadioptric cameras. vol. 2, pp. 217–223 (1999)
Jang, G., Kim, S., Kweon, I.: Single camera catadioptric stereo system. In: Proceedings of Workshop on Omnidirectional Vision, Camera Networks and Nonclassical cameras (OMNIVIS2005) (2005)
Cabral, E.L.L., de Souza Junior, J.C., Hunold, M.C.: Omnidirectional stereo vision with a hiperbolic double lobed mirror. Pattern Recogn. Int. Conf. 1, 1–4 (2004)
Derrien, S., Konolige, K.: Approximating a single viewpoint in panoramic imaging devices. In: Proceedings of ICRA ’00. IEEE International Conference on Robotics and Automation, vol. 4, pp. 3931–3938 (2000)
Nelson R.C., Aloimonos J.: Finding motion parameters from spherical motion fields (or the advantages of having eyes in the back of your head). Biol. Cybern. 58(4), 261–273 (1988)
McCarthy, C., Barnes, N., Srinivasan, M.: Real time biologically-inspired depth maps from spherical flow. In: 2007 IEEE International Conference on Robotics and Automation, pp. 4887–4892 (2007)
Baker, S., Nayar, K.: A tutorial on catadioptric image formation. Spherical mirrors (1998)
Matthies L., Shafer S.: Error modeling in stereo navigation. IEEE J. Robot. Autom. 3(3), 239–248 (1987)
Souhila K., Karim A.: Optical flow based robot obstacle avoidance. Int. J. Adv. Robot. Syst. 4(1), 13–16 (2007)
Li R., Sclaroff S.: Computer vision and image understanding. Comput. Vision Image Underst. 110(1), 75–90 (2008)
Scaramuzza, D.: Omnidirectional vision: from calibration to robot motion estimation. ETH Zurich, PhD thesis, 17635 (2008)
Author information
Authors and Affiliations
Corresponding author
Additional information
This work is supported in part by US Army Research Office under grant No. W911NF-09-1-0565, and US National Science Foundation under grants Nos. CNS-0619577 and IIS-0644127.
Rights and permissions
About this article
Cite this article
Labutov, I., Jaramillo, C. & Xiao, J. Generating near-spherical range panoramas by fusing optical flow and stereo from a single-camera folded catadioptric rig. Machine Vision and Applications 24, 133–144 (2013). https://doi.org/10.1007/s00138-011-0368-2
Received:
Revised:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s00138-011-0368-2