Efficient Multi-frequency Phase Unwrapping Using Kernel Density Estimation

  • Felix Järemo LawinEmail author
  • Per-Erik Forssén
  • Hannes Ovrén
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9908)


In this paper we introduce an efficient method to unwrap multi-frequency phase estimates for time-of-flight ranging. The algorithm generates multiple depth hypotheses and uses a spatial kernel density estimate (KDE) to rank them. The confidence produced by the KDE is also an effective means to detect outliers. We also introduce a new closed-form expression for phase noise prediction, that better fits real data. The method is applied to depth decoding for the Kinect v2 sensor, and compared to the Microsoft Kinect SDK and to the open source driver libfreenect2. The intended Kinect v2 use case is scenes with less than 8 m range, and for such cases we observe consistent improvements, while maintaining real-time performance. When extending the depth range to the maximal value of 18.75 m, we get about \(52\,\%\) more valid measurements than libfreenect2. The effect is that the sensor can now be used in large depth scenes, where it was previously not a good choice.


Time-of-flight Kinect v2 Kernel-density-estimation 



This work has been supported by the Swedish Research Council in projects 2014-6227 (EMC2) and 2014-5928 (LCMM) and the EU’s Horizon 2020 Programme grant No 644839 (CENTAURO).

Supplementary material

Supplementary material 1 (mp4 12608 KB)

419976_1_En_11_MOESM2_ESM.pdf (15 mb)
Supplementary material 2 (pdf 15332 KB)


  1. 1.
    Trunk, G., Brockett, S.: Range and velocity ambiguity resolution. In: IEEE National Radar Conference, pp. 146–149 (1993)Google Scholar
  2. 2.
    Sell, J., O’Connor, P.: The Xbox one system on a chip and kinect sensor. IEEE Micro 34(2), 44–53 (2014)CrossRefGoogle Scholar
  3. 3.
    Newcombe, R.A., et al.: KinectFusion: real-time dense surface mapping and tracking. In: IEEE ISMAR 2011, Basel, Switzerland, October 2011Google Scholar
  4. 4.
    Jongenelen, A.P.P., Bailey, D.G., Payne, A.D., Dorrington, A.A., Carnegie, D.A.: Analysis of errors in tof range imaging with dual-frequency modulation. IEEE Trans. Instrum. Measur. 60(5), 1861–1868 (2011)CrossRefGoogle Scholar
  5. 5.
    Wang, Z., Du, H., Park, S., Xie, H.: Three-dimensional shape measurement with a fast and accurate approach. Appl. Opt. 48(6), 1052–1061 (2009)CrossRefGoogle Scholar
  6. 6.
    Gorthi, S.S., Pastogi, P.: Fringe projection techniques: wither we are? Opt. Lasers Eng. 48(2), 133–140 (2010)CrossRefGoogle Scholar
  7. 7.
    Hansard, M.: Disambiguation of time-of-flight data. In: Time-of-Flight Cameras, Chap. 2. Springer Briefs in Computer Science, pp. 29–43. Springer, London (2013)Google Scholar
  8. 8.
    Crabb, R., Manduchi, R.: Fast single-frequenct time-of-flight range imaging. In: IEEE International Conference on Computer Vision and Pattern Recognition (CVPR 2015) (2015)Google Scholar
  9. 9.
    Droeschel, D., Holz, D., Behnke, S.: Multi-frequency phase unwrapping for time-of-flight cameras. In: IEEE International Conference on Intelligent Robots and Systems (2010)Google Scholar
  10. 10.
    Kirmani, A., Benedetti, A., Chou, P.A.: SPUMIC: simultaneous phase unwrapping and multipath interference cancellation in time-of-flight cameras using spectral methods. In: IEEE International Conference on Multimedia& Expo (ICME 2013) (2013)Google Scholar
  11. 11.
    Feigin, M., Bhandari, A., Izadi, S., Rhemann, C., Schmidt, M., Raskar, R.: Resolving multipath interference in Kinect: an inverse problem approach. IEEE Sens. J. 16, 3419–3427 (2015)CrossRefGoogle Scholar
  12. 12.
    Mei, J., Kirmani, A., Colaco, A., Goyal, V.K.: Phase unwrapping and denoising for time-of-flight imaging using generalized approximate message passing. In: IEEE International Conference on Image Processing (ICIP 2013) (2013)Google Scholar
  13. 13.
    Frank, M., Plaue, M., Rapp, H., Köthe, U., Jähne, B.: Theoretical and experimental error analysis of continuous-wave time-of-flight range cameras. Opt. Eng. 48(1), 013602 (2009)CrossRefGoogle Scholar
  14. 14.
    Bamji, C., Charbon, E.: US 6,515,740 B2: methods for CMOS-compatible three-dimensional image sensing using quantum efficiency modulation (2004)Google Scholar
  15. 15.
    Bamji, C.S., et al.: A 0.13 \(\mu \)m CMOS system-on-chip for a \(512\times 424\) time-of-flight image sensor with multi-frequency photo-demodulation up to 130 MHz and 2 GS/s ADC. IEEE J. Solid-State Circuits 50(1), 303–319 (2015)CrossRefGoogle Scholar
  16. 16.
    Murphy, K.P.: Machine Learning: A Probabilistic Perspective. MIT Press, Cambridge (2012)zbMATHGoogle Scholar
  17. 17.
    Uhlmann, J.: Dynamic map building and localization: new theoretical foundations. Ph.D. thesis, University of Oxford (1995)Google Scholar
  18. 18.
    Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision, 2nd edn. Cambridge University Press, Cambridge (2003)zbMATHGoogle Scholar
  19. 19.
    Szeliski, R.: Computer Vision: Algorithms and Applications. Springer-Verlag New York Inc., New York (2010)zbMATHGoogle Scholar
  20. 20.
    Forssén, P.E.: Low and medium level vision using channel representations. Ph.D. thesis, Linköping University, Sweden, SE-581 83 Linköping, Sweden, March 2004. Dissertation No. 858, ISBN 91-7373-876-XGoogle Scholar
  21. 21.
    Felsberg, M., Forssén, P.E., Scharr, H.: Channel smoothing: efficient robust smoothing of low-level signal features. IEEE TPAMI 28(2), 209–222 (2006)CrossRefGoogle Scholar
  22. 22.
    Järemo Lawin, F., Forssén, P.E., Ovrén, H.: Efficient multi-frequency phase unwrapping using kernel density estimation, supplemental material. Technical report, Linköing University (2016)Google Scholar
  23. 23.
    Khronos Group: OpenCL language specification (2015).
  24. 24.
    Rusu, R.B., Cousins, S.: 3D is here: point cloud library (PCL). In: IEEE ICRA, 9–13 May 2011Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Felix Järemo Lawin
    • 1
    Email author
  • Per-Erik Forssén
    • 1
  • Hannes Ovrén
    • 1
  1. 1.Computer Vision LaboratoryLinköping UniversityLinköpingSweden

Personalised recommendations