Journal of Signal Processing Systems

, Volume 70, Issue 3, pp 305–328 | Cite as

The PANOPTIC Camera: A Plenoptic Sensor with Real-Time Omnidirectional Capability

  • Hossein AfshariEmail author
  • Laurent Jacques
  • Luigi Bagnato
  • Alexandre Schmid
  • Pierre Vandergheynst
  • Yusuf Leblebici


A new biologically-inspired vision sensor made of one hundred “eyes” is presented, which is suitable for real-time acquisition and processing of 3-D image sequences. This device, named the Panoptic camera, consists of a layered arrangement of approximately 100 classical CMOS imagers, distributed over a hemisphere of 13 cm in diameter. The Panoptic camera is a polydioptric system where all imagers have their own vision of the world, each with a distinct focal point, which is a specific feature of the Panoptic system. This enables 3-D information recording such as omnidirectional stereoscopy or depth estimation, applying specific signal processing. The algorithms dictating the image reconstruction of an omnidirectional observer located at any point inside the hemisphere are presented. A hardware architecture which has the capability of handling these algorithms, and the flexibility to support additional image processing in real time, has been developed as a two-layer system based on FPGAs. The detail of the hardware architecture, its internal blocks, the mapping of the algorithms onto the latter elements, and the device calibration procedure are presented, along with imaging results.


Omnidirectional Polydioptric Plenoptic Image rendering Computer vision Image processing hardware 3-D vision system FPGA 



The authors thank A. Cevrero, P. Athanasopoulos, S. Hauser, P. Bruehlmeier, D. Raboud, B. Devrim, A. Amini and the staff at the mechanical and electrical workshops of EPFL. The authors gratefully acknowledge the support of Xilinx, Inc., through the Xilinx University Program.


  1. 1.
    Richte, I. A. (1980). The notebooks of Leonardo Da Vinci. Oxford University Press.Google Scholar
  2. 2.
    Adelson, E. H., & Bergen, J. R. (1991). Computational models of visual processing (pp. 3–20). MIT Press.Google Scholar
  3. 3.
    Neumann, J., Fermuller, C., & Aloimonos, Y. (2002). In 3D data processing visualization and transmission, 2002. Proceedings. First international symposium on (pp. 2–11).Google Scholar
  4. 4.
    Zbikowski, R. (2005). Spectrum. IEEE, 42(11), 46.Google Scholar
  5. 5.
    Geyer, C., & Daniilidis, K. (2001). International Journal of Computer Vision, 45(3), 223.zbMATHCrossRefGoogle Scholar
  6. 6.
    Adelson, E., & Wang, J. (2002). Single lens stereo with a plenoptic camera. IEEE Transactions on Pattern Analysis and Machine Intelligence, 14(2), 99.CrossRefGoogle Scholar
  7. 7.
    Cianci, C., Raemy, X., Pugh, J., & Martinoli, A. (2006). In Proceedings of the 2nd international conference on swarm robotics (pp. 103–115). Springer-Verlag.Google Scholar
  8. 8.
    ViewPLUS. ASTRO Sensor Series (SOS:Stereo Omnidirectional System, BallCam). Available online at Accessed on 7 December 2011.
  9. 9.
    Lee, L. P., & Szema, R. (2005). Inspirations from biological optics for advanced photonic systems. Science, 310(5751), 1148.CrossRefGoogle Scholar
  10. 10.
    Jeong, K. H., Kim, J., & Lee, L. P. (2006). Biologically inspired artificial compound eyes. Science, 312(5773), 557.CrossRefGoogle Scholar
  11. 11.
    Spherical trigonometry. Available online at Accessed on 7 December 2011.
  12. 12.
    Hartley, R. I., & Zisserman, A. (2004). Multiple view geometry in computer vision (2nd Edn.). Cambridge University Press, ISBN: 0521540518.Google Scholar
  13. 13.
    Levoy, M., & Hanrahan, P. (1996). In SIGGRAPH ’96, proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 31–42). ACM.Google Scholar
  14. 14.
    Gortler, S., Grzeszczuk, R., Szeliski, R., & Cohen, M. (1996). In SIGGRAPH ’96, proceedings of the 23rd annual conference on computer graphics and interactive techniques (pp. 43–54). ACM.Google Scholar
  15. 15.
    McMillan, L., & Bishop, G. (1995). In SIGGRAPH ’95, proceedings of the 22rd annual conference on computer graphics and interactive techniques (pp. 39–46). ACM.Google Scholar
  16. 16.
    Górski, K. M., Hivon, E., Banday, A., Wandelt, B., Hansen, F., Reinecke, M., et al. (2005). HEALPix: A Framework for high-resolution discretization and fast analysis of data distributed on the sphere.. The Astrophysical Journal, 622, 759.CrossRefGoogle Scholar
  17. 17.
    Raboud, D. (2009). The panoptic camera—plenoptic interpolation in an omnidirectional polydioptric camera. Master’s thesis, EPFL.Google Scholar
  18. 18.
    Aurenhammer, F. (1991). Computational geometry: Algorithms and applications. ACM Computing Surveys, 23, 345.CrossRefGoogle Scholar
  19. 19.
    de Berg, M., Cheong, O., van Kreveld, M., & Overmars, M. (2008). Computational geometry (3rd Edn.). Springer-Verlag, ISBN: 978-3-540-77973-5.Google Scholar
  20. 20.
    Kilts, S. (2007). Advanced FPGA design: Architecture, implementation, and optimization. Wiley-IEEE Press.Google Scholar
  21. 21.
    Xilinx. Application note 136. Available online at Accessed on 7 December 2011.
  22. 22.
    Tierney, J., Rader, C., & Gold, B. (1971). A digital frequency synthesizer. IEEE Transactions on Audio and Electroacoustics, 19(1), 48–57.CrossRefGoogle Scholar
  23. 23.
    Volder, J. E. (1959). The CORDIC trigonometric computing technique. IRE Transactions on Electronic Computers, EC-8(3), 330.CrossRefGoogle Scholar
  24. 24.
    Walther, J. S. (1971). In Proceedings of the 18–20 May 1971, Spring joint computer conference, AFIPS ’71 (Spring) (pp. 379–385). New York, NY, U.S.A.: ACM.CrossRefGoogle Scholar
  25. 25.
    Meyer-Baese, U. (2007). Digital signal processing with field programmable gate arrays (3rd Edn.). Springer.Google Scholar
  26. 26.
    Bouguet, J. (2011). Camera calibration toolbox for matlab. Available online at Accessed on 7 December 2011.

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  • Hossein Afshari
    • 1
    Email author
  • Laurent Jacques
    • 1
  • Luigi Bagnato
    • 1
  • Alexandre Schmid
    • 1
  • Pierre Vandergheynst
    • 1
  • Yusuf Leblebici
    • 1
  1. 1.EPFL-STI-IEL-LSMLausanneSwitzerland

Personalised recommendations