International Journal of Computer Vision

, Volume 101, Issue 3, pp 519–532 | Cite as

Virtual Volumetric Graphics on Commodity Displays Using 3D Viewer Tracking

  • Charles Malleson
  • John CollomosseEmail author


Three dimensional (3D) displays typically rely on stereo disparity, requiring specialized hardware to be worn or embedded in the display. We present a novel 3D graphics display system for volumetric scene visualization using only standard 2D display hardware and a pair of calibrated web cameras. Our computer vision-based system requires no worn or other special hardware. Rather than producing the depth illusion through disparity, we deliver a full volumetric 3D visualization—enabling users to interactively explore 3D scenes by varying their viewing position and angle according to the tracked 3D position of their face and eyes. We incorporate a novel wand-based calibration that allows the cameras to be placed at arbitrary positions and orientations relative to the display. The resulting system operates at real-time speeds (∼25 fps) with low latency (120–225 ms) delivering a compelling natural user interface and immersive experience for 3D viewing. In addition to objective evaluation of display stability and responsiveness, we report on user trials comparing users’ timings on a spatial orientation task.


Volumetric display 3D graphics Tracking Kalman filter Camera Calibration 

Supplementary material

(AVI 6.2 MB)


  1. Alnowami, M., Alnwaimi, B., Copland, M., & Wells, K. (2011). A quantitative assessment of using the Kinect for Xbox 360 for respiratory surface motion tracking. In Proc. SPIE medical imaging. Google Scholar
  2. Brar, L., Sexton, I., Surman, P., Bates, R., Lee, W., Hopf, K., Neumann, F., Day, S., & Williman, E. (2010). Laser-based head-tracked 3D display research. Journal of Display Technology, 6(10), 531–543. CrossRefGoogle Scholar
  3. Chen, C., Huang, Y., Chuang, S., Wu, C., Shieh, H., Mphepo, W., Hsieh, C., & Hsu, S. (2009). Liquid crystal panel for high efficiency barrier type autostereoscopic 3D displays. Applied Optics, 48(18), 3446–3454. CrossRefGoogle Scholar
  4. Dang, T., Hoffmann, C., & Stiller, C. (2009). Continuous stereo self-calibration by camera parameter tracking. IEEE Transactions on Image Processing, 18(7), 1536–1549. MathSciNetCrossRefGoogle Scholar
  5. Dodgson, N. (2004). Variation and extrema of human interpupillary distance. Proceedings of SPIE, 5291, 36–46. CrossRefGoogle Scholar
  6. Ellis, S. R., Wolfram, A., & Adelstein, B. D. (2002). Three dimensional tracking in augmented environments: user performance trade-offs between system latency and update rate. Proceedings of the Human Factors and Ergonomics Society annual meeting, 46(26), 2149–2153. CrossRefGoogle Scholar
  7. Erden, E., Kishore, V., Urey, H., Baghsiahi, H., Willman, E., Day, S., Selviah, D., Fernandez, F., & Surman, P. (2009). Laser scanning based autostereoscopic 3D display with pupil tracking. In Proc. IEEE photonics (pp. 10–11). Google Scholar
  8. Ezra, D., Woodgate, G., Omar, B., Holliman, N., Harrold, J., & Shapiro, L. (1995). New autostereoscopic display system. In Proc. SPIE Intl. Society of Optical Engineering (pp. 31–40). Google Scholar
  9. Free2C (2010). The free2c desktop display (Technical report). Heinrich Hertz Institute. Google Scholar
  10. Freund, Y., & Schapire, R. (1999). A short introduction to boosting. Journal of Japanese Society for Artificial Intelligence, 14(7), 771–780. Google Scholar
  11. Lee, J. (2008). Head tracking for desktop VR displays using the WiiRemote (Technical report). Carnegie Mellon University. Google Scholar
  12. Malleson, C., & Collomosse, J. (2011). Volumetric 3D graphics on commodity displays using active gaze tracking. In Proc. ICCV workshop on human computer interaction. Google Scholar
  13. Nishimura, H., Abe, T., Yamamoto, H., Hayasaki, Y., Nagai, Y., Shimizu, Y., & Nishida, N. (2007). Development of a 140-inch autostereoscopic display by use of full-color LED panel. Proceedings of SPIE, the International Society for Optical Engineering, 6486, 64861B. CrossRefGoogle Scholar
  14. OpenCV. Open source computer vision library. Accessed July 2011. Google Scholar
  15. Perlin, K., Poultney, C., Kollin, J., Kristjansson, D., & Paxia, S. (2001). Recent advances in the NYU autostereoscopic display. Proceedings of SPIE, the International Society for Optical Engineering, 4297, 196–203. CrossRefGoogle Scholar
  16. Sandin, D., Margolis, T., Dawe, G., Leigh, J., & DeFanti, T. (2001). Varrier autostereographic display. Proceedings of SPIE, the International Society for Optical Engineering, 4297, 204–211. CrossRefGoogle Scholar
  17. Schwartz, A. (1985). Head tracking stereoscopic display. In Proc. IEEE intl. conf. on display research (pp. 141–144). Google Scholar
  18. Sorensen, S., Hansen, P., & Sorensen, N. (2004). Method for recording and viewing stereoscopic images in color using monochrome filters. U.S. Patent 6687003. Google Scholar
  19. Surman, P., Sexton, I., Hopf, K., Lee, W., Buckley, E., Jones, G., & Bates, R. (2008a). European research into head tracked autostereoscopic displays. In Proc. conf on 3DTV (pp. 161–164). Google Scholar
  20. Surman, P., Sexton, I., Hopf, K., Lee, W., Neumann, F., Buckley, E., Jones, G., Corbett, A., Bates, R., & Talukdar, S. (2008b). European research into head tracked autostereoscopic displays. Journal of the Society for Information Display, 16, 743–753. CrossRefGoogle Scholar
  21. Takaki, Y. (2006). High-density directional display for generating natural 3D images. Proceedings of the IEEE, 94(3), 654–663. CrossRefGoogle Scholar
  22. Tetsutani, N., Ichinose, S., & Ishibashi, M. (1989). 3D-TV projection display system with head-tracking. In Japan Display (pp. 56–59). Google Scholar
  23. Tetsutani, N., Omura, K., & Kishino, F. (1994). Study on a stereoscopic display system employing eye-position tracking for multi-viewers. Proceedings of SPIE, the International Society for Optical Engineering, 2177, 135. CrossRefGoogle Scholar
  24. Thacker, N. A. (1992). Online calibration of a 4 DOF stereo head. In Proc. British machine vision conference (BMVC) (pp. 528–537). Google Scholar
  25. Tsai, R., Tsai, C., Lee, K., Wu, C., Lin, L., Huang, K., Hsu, W., Wu, C., Lu, C., Yang, J., & Chen, Y. (2009). Challenge of 3D LCD displays. Proceedings of SPIE, the International Society for Optical Engineering, 7329, 732903. CrossRefGoogle Scholar
  26. Urey, H., & Erden, E. (2011). State of the art in stereoscopic and autostereoscopic displays. Proceedings of the IEEE, 99(4), 544–555. CrossRefGoogle Scholar
  27. Viola, P., & Jones, M. (2001). Rapid object detection using a boosted cascade of simple features. In Proc. computer vision and pattern recognition. Google Scholar
  28. Welch, B. L. (1947). The generalization of student’s problem when several different population variances are involved. Biometrika, 34(1–2), 28–35. doi: 10.1093/biomet/34.1-2.28. MathSciNetzbMATHGoogle Scholar
  29. Woodgate, G., Ezra, D., Harrold, J., Holliman, N., Jones, G., & Moseley, R. (1997). Observer-tracking autostereoscopic 3D display systems. Proceedings of SPIE, the International Society for Optical Engineering, 3012, 187–198. CrossRefGoogle Scholar
  30. Woodgate, G., Harrold, J., Jacobs, A., Mosely, R., & Ezra, D. (2000). Flat-panel autostereoscopic displays: characterization and enhancement. Proceedings of SPIE, the International Society for Optical Engineering, 3957, 153–164. CrossRefGoogle Scholar
  31. Woods, A. (2009). 3D displays in the home. Information Display, 7, 8–12. Google Scholar
  32. Yamamoto, H., Kouno, M., Muguruma, S., Hayasaki, Y., Nagai, Y., Shimizu, Y., & Nishida, N. (2002). Enlargement of viewing area of stereoscopic full-color LED display using parallax barrier combined with polarizer. Applied Optics, 41(32), 6907–6919. CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media, LLC 2012

Authors and Affiliations

  1. 1.Centre for Vision Speech and Signal ProcessingUniversity of SurreyGuildford, SurreyUK

Personalised recommendations