Journal of Real-Time Image Processing

, Volume 7, Issue 1, pp 21–29 | Cite as

Efficient FPGA implementation of homodyne-based time-of-flight range imaging

  • Adrian Peter Paul JongenelenEmail author
  • D. G. Bailey
  • A. D. Payne
  • D. A. Carnegie
  • A. A. Dorrington
Special Issue


Time-of-flight range imaging systems illuminate a scene with an amplitude-modulated light source, the light is reflected from objects in the scene, and measurement of the phase of the modulation envelope is performed to determine the object’s distance. As the image sensor is capable of performing this task for every pixel simultaneously, acquisition of the entire scene can be performed at rapid (video) update rates, making the technology ideal for real-time applications. In this paper we present an efficient real-time FPGA algorithm for determining phase, and hence distance, from the raw image sensor output. The algorithm has been implemented on a range imaging system based on the PMD19k-2 image sensor, with range processing performed in real time by a Stratix III FPGA. The scarcest resource in this implementation is RAM, and an analysis is presented to maximise the efficiency of this resource whilst maintaining acceptable processing accuracy. The algorithm can be extended for processing multiple simultaneous modulation frequencies. An efficient method for combining these results to determine unambiguous range, based on the Chinese remainder theorem, is also presented.


3D-Imaging FPGA Time-of-flight Range imaging Chinese remainder theorem 


  1. 1.
    Gulden, G., Becker, D., Vossiek, M.: Novel optical distance sensor based on MSM technology. IEEE Sens J 4(5), 612–618 (2004)CrossRefGoogle Scholar
  2. 2.
    Büttgen, B., Seitz, P.: Robust optical time-of-flight range imaging based on smart pixel structures. IEEE Trans Circuits Syst I Regul Pap 55, 1512–1525 (2008)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Kawahito, S., Halin, I.A., Ushinaga, T., Sawada, T., Homma, M., Maeda, Y.: A CMOS time-of-flight range image sensor with gates-on-field-oxide structure. IEEE Sens J 7(12), 1578–1586 (2007)CrossRefGoogle Scholar
  4. 4.
    Gokturk, S.B., Yalcin, H., Bamji, C.: A time-of-flight depth sensor—system description, issues, and solutions. In: Proceedings of Computer Vision and Pattern Recognition Workshop. (2004)
  5. 5.
    Dorrington, A.A., Cree, M.J., Carnegie, D.A., Payne, A.D., Conroy, R.M., Godbaz, J.P., Jongenelen, A.P.P.: Video-rate or high-precision: a flexible range imaging camera. Image processing: machine vision applications. In: Proceedings of the SPIE, 6813, 681307 (2008)Google Scholar
  6. 6.
    Hussmann, S., Schauer, D., Macdonald, B.: Integration of a 3D-TOF camera into an autonomous mobile robot system. In: 26th International Conference on Instrumentation and Measurement Technology, Singapore, 5–7 May, pp. 547–552 (2009)Google Scholar
  7. 7.
    Carnegie, D.A., Cree, M.J., Dorrington, A.A., Payne, A.D.: A high resolution full-field range imaging system for robotic devices. In: Proceedings of the Sixth IASTED International Conference on Robotics and Applications, Boston, USA, pp. 306–311, 31 October–2 November (2005)Google Scholar
  8. 8.
    Yahav, G., Iddan, G., Mandelbaum, D.: 3D imaging camera for gaming application. In: International Conference on Consumer Electronics, pp. 1–2. (2007)
  9. 9.
    Jongenelen, A.P.P., Carnegie, D.A., Payne A.D., Dorrington, A.A.: Maximising precision over extended unambiguous range for TOF range imaging systems. In: Proceedings of International Instrumentation and Measurement Technology Conference, Austin, Texas, pp. 1575–1580 (2010)Google Scholar
  10. 10.
    Payne, A.D., Jongenelen, A.P.P., Dorrington, A.A., Cree, M.J., Carnegie, D.A.: Multiple frequency range imaging to remove measurement ambiguity. In: Optical 3-D Measurement Techniques IX, vol. 2, pp. 139–148 (2009)Google Scholar
  11. 11.
    Bellis, S.J., Marnane, W.P.: A CORDIC arctangent FPGA implementation for a high-speed 3D-camera system. In: Proceedings of the Roadmap to Reconfigurable Computing, 10th International Workshop on Field-Programmable Logic and Applications, Lecture Notes in Computer Science, vol. 1896, pp. 485–494 (2000)Google Scholar
  12. 12.
    Jongenelen, A.P.P., Carnegie, D.A., Dorrington, A.A., Payne, A.D.: Heterodyne range imaging in real-time. In: Proceedings of International Conference on Sensing Technology, Tainan, pp. 57–62 (2008)Google Scholar
  13. 13.
    Bailey, D.G.: Space efficient division on FPGAs. In: Electronics New Zealand Conference (ENZCon’06), Christchurch, New Zealand, pp. 206–211, 13–14 November (2006)Google Scholar
  14. 14.
    Walther, J.S.: A unified algorithm for elementary functions. In: AFIPS Conference Proceedings, vol. 38, pp. 379–385 (1971)Google Scholar
  15. 15.
    Lange, R.: 3D time-of-flight distance measurement with custom solid-state image sensors in CMOS/CCD-technology. PhD dissertation, University of Siegen (2000)Google Scholar
  16. 16.
    Dorrington, A.A., Cree, M.J., Payne, A.D., Conroy, R.M., Carnegie, D.A.: Achieving sub-millimetre precision with a solid-state full-field heterodyning range imaging camera. Meas Sci Technol 18, 2809–2816 (2007)CrossRefGoogle Scholar
  17. 17.
    Payne, A.D., Dorrington, A.A., Cree, M.J., Carnegie, D.A.: Improved linearity using harmonic error rejection in a full-field range imaging system. In: Proceedings of SPIE 3D Image Capture and Applications VII, vol. 6805, pp. 68050D (2008)Google Scholar
  18. 18.
    Altera Corporation. Stratix III Device Handbook, v1.7 edition. (2008)
  19. 19.
    PMD Technologies GmbH. Photonics PMD 19K-2 high resolution 3D video sensor array 160 × 120 pixels datasheet, Revision 1.0. (2008)
  20. 20.
    Jongenelen, A.P.P., Carnegie, D.A., Payne, A.P., Dorrington, A.A.: Development and characterisation of an easily configurable range imaging system. In: Proceedings of Image and Vision Computing New Zealand, pp. 79–84 (2009)Google Scholar
  21. 21.
    Wang, Y.: New Chinese remainder theorems. In: Proceedings of 32nd Asilomar Conference on Signals, Systems and Computers, pp. 165–171 (1998)Google Scholar

Copyright information

© Springer-Verlag 2010

Authors and Affiliations

  • Adrian Peter Paul Jongenelen
    • 1
    Email author
  • D. G. Bailey
    • 2
  • A. D. Payne
    • 3
  • D. A. Carnegie
    • 1
  • A. A. Dorrington
    • 3
  1. 1.School of Engineering and Computer ScienceVictoria University of WellingtonWellingtonNew Zealand
  2. 2.School of Engineering and Advanced TechnologyMassey UniversityPalmerston NorthNew Zealand
  3. 3.School of EngineeringUniversity of WaikatoHamiltonNew Zealand

Personalised recommendations