The PANOPTIC Camera: A Plenoptic Sensor with Real-Time Omnidirectional Capability
- 832 Downloads
A new biologically-inspired vision sensor made of one hundred “eyes” is presented, which is suitable for real-time acquisition and processing of 3-D image sequences. This device, named the Panoptic camera, consists of a layered arrangement of approximately 100 classical CMOS imagers, distributed over a hemisphere of 13 cm in diameter. The Panoptic camera is a polydioptric system where all imagers have their own vision of the world, each with a distinct focal point, which is a specific feature of the Panoptic system. This enables 3-D information recording such as omnidirectional stereoscopy or depth estimation, applying specific signal processing. The algorithms dictating the image reconstruction of an omnidirectional observer located at any point inside the hemisphere are presented. A hardware architecture which has the capability of handling these algorithms, and the flexibility to support additional image processing in real time, has been developed as a two-layer system based on FPGAs. The detail of the hardware architecture, its internal blocks, the mapping of the algorithms onto the latter elements, and the device calibration procedure are presented, along with imaging results.
KeywordsOmnidirectional Polydioptric Plenoptic Image rendering Computer vision Image processing hardware 3-D vision system FPGA
The authors thank A. Cevrero, P. Athanasopoulos, S. Hauser, P. Bruehlmeier, D. Raboud, B. Devrim, A. Amini and the staff at the mechanical and electrical workshops of EPFL. The authors gratefully acknowledge the support of Xilinx, Inc., through the Xilinx University Program.
- 1.Richte, I. A. (1980). The notebooks of Leonardo Da Vinci. Oxford University Press.Google Scholar
- 2.Adelson, E. H., & Bergen, J. R. (1991). Computational models of visual processing (pp. 3–20). MIT Press.Google Scholar
- 3.Neumann, J., Fermuller, C., & Aloimonos, Y. (2002). In 3D data processing visualization and transmission, 2002. Proceedings. First international symposium on (pp. 2–11).Google Scholar
- 4.Zbikowski, R. (2005). Spectrum. IEEE, 42(11), 46.Google Scholar
- 7.Cianci, C., Raemy, X., Pugh, J., & Martinoli, A. (2006). In Proceedings of the 2nd international conference on swarm robotics (pp. 103–115). Springer-Verlag.Google Scholar
- 8.ViewPLUS. ASTRO Sensor Series (SOS:Stereo Omnidirectional System, BallCam). Available online at http://www.viewplus.co.jp/product/09/05.html. Accessed on 7 December 2011.
- 11.Spherical trigonometry. Available online at http://en.wikipedia.org/wiki/Spherical_trigonometry. Accessed on 7 December 2011.
- 12.Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd Edn.). Cambridge University Press, ISBN: 0521540518.Google Scholar
- 13.Levoy, M., & Hanrahan, P. (1996). In SIGGRAPH ’96, proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 31–42). ACM.Google Scholar
- 14.Gortler, S., Grzeszczuk, R., Szeliski, R., & Cohen, M. (1996). In SIGGRAPH ’96, proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 43–54). ACM.Google Scholar
- 15.McMillan, L., & Bishop, G. (1995). In SIGGRAPH ’95, proceedings of the 22rd annual conference on computer graphics and interactive techniques (pp. 39–46). ACM.Google Scholar
- 17.Raboud, D. (2009). The panoptic camera—plenoptic interpolation in an omnidirectional polydioptric camera. Master’s thesis, EPFL.Google Scholar
- 19.de Berg, M., Cheong, O., van Kreveld, M., & Overmars, M. (2008). Computational geometry (3rd Edn.). Springer-Verlag, ISBN: 978-3-540-77973-5.Google Scholar
- 20.Kilts, S. (2007). Advanced FPGA design: Architecture, implementation, and optimization. Wiley-IEEE Press.Google Scholar
- 21.Xilinx. Application note 136. Available online at http://www.xilinx.com/support/documentation/application_notes/xapp136.pdf. Accessed on 7 December 2011.
- 25.Meyer-Baese, U. (2007). Digital signal processing with field programmable gate arrays (3rd Edn.). Springer.Google Scholar
- 26.Bouguet, J. (2011). Camera calibration toolbox for matlab. Available online at http://www.vision.caltech.edu/bouguetj/calib_doc. Accessed on 7 December 2011.