Multimedia Tools and Applications

, Volume 74, Issue 17, pp 7331–7354 | Cite as

Comparative evaluation of methods for filtering Kinect depth data

  • Kyis Essmaeel
  • Luigi Gallo
  • Ernesto Damiani
  • Giuseppe De Pietro
  • Albert Dipanda
Article

Abstract

The release of the Kinect has fostered the design of novel methods and techniques in several application domains. It has been tested in different contexts, which span from home entertainment to surgical environments. Nonetheless, to promote its adoption to solve real-world problems, the Kinect should be evaluated in terms of precision and accuracy. Up to now, some filtering approaches have been proposed to enhance the precision and accuracy of the Kinect sensor, and preliminary studies have shown promising results. In this work, we discuss the results of a study in which we have compared the most commonly used filtering approaches for Kinect depth data, in both static and dynamic contexts, by using novel metrics. The experimental results show that each approach can be profitably used to enhance the precision and/or accuracy of Kinect depth data in a specific context, whereas the temporal filtering approach is able to reduce noise in different experimental conditions.

Keywords

Comparative evaluation Kinect Depth data Depth instability Temporal denoising Median filter Bilateral filter 

References

  1. 1.
    Andersen M, Jensen T, Lisouski P, Mortensen A, Hansen M, Gregersen T, Ahrendt P (2012) Kinect depth sensor evaluation for computer vision applicationsGoogle Scholar
  2. 2.
    Berger K, Meister S, Nair R, Kondermann D (2013) A state of the art report on kinect sensor setups in computer vision. In: Grzegorzek M, Theobalt C, Koch R, Kolb A (eds) Time-of-flight and depth imaging. Sensors, algorithms, and applications. Lecture notes in computer science, vol 8200. Springer, Berlin Heidelberg, pp 257–272Google Scholar
  3. 3.
    Buades A, Coll B, Morel JM (2005) A non-local algorithm for image denoising. In: IEEE computer society conference on computer vision and pattern recognition, 2005. CVPR 2005, vol 2, pp 60–65. doi:10.1109/CVPR.2005.38
  4. 4.
    Burrus: RGB Demo project (2011). http://labs.manctl.com/rgbdemo
  5. 5.
    Camplani M, Salgado L (2012) Efficient spatio-temporal hole filling strategy for kinect depth maps pp 82,900E–82,900E–10Google Scholar
  6. 6.
    Cerveri P, Pedotti A, Ferrigno G (2003) Robust recovery of human motion from video using kalman filters and virtual humans. Hum Mov Sci 22(3):377–404CrossRefGoogle Scholar
  7. 7.
    Chan D, Buisman H, Theobalt C, Thrun S (2008) A noise-aware filter for real-time depth upsampling. In: ECCV Workshop on multicamera and multimodal sensor fusion algorithms and applicationGoogle Scholar
  8. 8.
    Clark RA, Pua YH, Fortin K, Ritchie C, Webster KE, Denehy L, Bryant AL (2012) Validity of the microsoft kinect for assessment of postural control. Gait Posture 36(3):372–377CrossRefGoogle Scholar
  9. 9.
    Dutta T (2012) Evaluation of the kinect sensor for 3-d kinematic measurement in the workplace. Appl Ergon 43(4):645–649CrossRefGoogle Scholar
  10. 10.
    Essmaeel K, Gallo L, Damiani E, De Pietro G, Dipanda A (2012) Temporal denoising of kinect depth data. In: 2012 Eighth international conference on signal image technology and internet based systems (SITIS), pp 47–52Google Scholar
  11. 11.
    Frees S, Kessler GD, Kay E (2007) Prism interaction for enhancing control in immersive virtual environments. ACM Trans Comput-Hum Interact 14(1)Google Scholar
  12. 12.
    Fu J, Shiqi W, Lu Y, Li S, Zeng W (2012) Kinect-like depth denoising. In: 2012 IEEE international symposium on circuits and systems (ISCAS), pp 512–515Google Scholar
  13. 13.
    Gallo L, Ciampi M, Minutolo A (2010) Smoothed pointing: a user-friendly technique for precision enhanced remote pointing. In: International conference on complex, intelligent and software intensive systems (CISIS), pp 712–717. IEEEGoogle Scholar
  14. 14.
    Gallo L, Minutolo A (2012) Design and comparative evaluation of smoothed pointing: a velocity-oriented remote pointing enhancement technique. Int J Hum-Comput Stud 70(4):287–300CrossRefGoogle Scholar
  15. 15.
    Gallo L, Placitelli A, Ciampi M (2011) Controller-free exploration of medical image data: experiencing the kinect. In: 2011 24th international symposium on computer-based medical systems (CBMS), pp 1–6Google Scholar
  16. 16.
    Hartmann J, Forouher D, Litza M, Kluessendorff JH, Maehle E (2012) Real-time visual slam using fastslam and the microsoft kinect camera. In: 7th German conference on robotics; proceedings of ROBOTIK 2012, pp 1–6Google Scholar
  17. 17.
    Huhle B, Schairer T, Jenke P, Strasser W (2008) Robust non-local denoising of colored depth data. In: IEEE computer society conference on computer vision and pattern recognition workshops, 2008. CVPRW ’08, pp 1–7. doi:10.1109/CVPRW.2008.4563158
  18. 18.
    Khoshelham K, Elberink SO (2012) Accuracy and resolution of kinect depth data for indoor mapping applications. Sensors 12(2):1437–1454. doi:10.3390/s120201437 CrossRefGoogle Scholar
  19. 19.
    Kopf J, Cohen MF, Lischinski D, Uyttendaele M (2007) Joint bilateral upsampling. ACM Trans Graph 26(3). doi:10.1145/1276377.1276497
  20. 20.
    Maimone A, Bidwell J, Peng K, Fuchs H (2012) Augmented reality: enhanced personal autostereoscopic telepresence system using commodity depth cameras. Comput Graph 36(7):791–807CrossRefGoogle Scholar
  21. 21.
    Matyunin S, Vatolin D, Berdnikov Y, Smirnov M (2011) Temporal filtering for depth maps generated by kinect depth camera. In: 3DTV conference: the true vision—capture, transmission and display of 3D Video (3DTV-CON), 2011, pp 1–4Google Scholar
  22. 22.
    Merrell P, Akbarzadeh A, Wang L, Mordohai P, Frahm JM, Yang R, Nister D, Pollefeys M (2007) Real-time visibility-based fusion of depth maps. In: IEEE 11th international conference on computer vision, 2007, ICCV 2007, pp 1–8Google Scholar
  23. 23.
    Microsoft: Kinect for X-BOX 360 (2011). http://www.xbox.com/en-US/Kinect
  24. 24.
    Monnich H, Nicolai P, Beyl T, Raczkowsky J, Worn H (2011) A supervision system for the intuitive usage of a telemanipulated surgical robotic setup. In: 2011 IEEE international conference on robotics and biomimetics (ROBIO), pp 449–454Google Scholar
  25. 25.
    Nguyen C, Izadi S, Lovell D (2012) Modeling kinect sensor noise for improved 3d reconstruction and tracking. In: 2012 second international conference on 3D imaging, modeling, processing, visualization and transmission (3DIMPVT), pp 524–530. doi:10.1109/3DIMPVT.2012.84
  26. 26.
    Parra-Dominguez G, Taati B, Mihailidis A (2012) 3d human motion analysis to detect abnormal events on stairs. In: 2012 second international conference on 3D imaging, modeling, processing, visualization and transmission (3DIMPVT), pp 97–103Google Scholar
  27. 27.
    Petschnigg G, Szeliski R, Agrawala M, Cohen M, Hoppe H, Toyama K (2004) Digital photography with flash and no-flash image pairs. ACM Trans Graph 23(3):664–672. doi:10.1145/1015706.1015777 CrossRefGoogle Scholar
  28. 28.
    Shotton J, Fitzgibbon A, Cook M, Sharp T, Finocchio M, Moore R, Kipman A, Blake A (2011) Real-time human pose recognition in parts from single depth images. In: Proceedings of the 2011 IEEE conference on computer vision and pattern recognition, CVPR ’11. IEEE Computer Society, Washington, DC, pp 1297–1304Google Scholar
  29. 29.
    Smisek J, Jancosek M, Pajdla T (2011) 3d with kinect. In: 2011 IEEE international conference on computer vision workshops (ICCV Workshops), pp 1154–1160Google Scholar
  30. 30.
    Stoyanov T, Louloudi A, Andreasson H, Lilienthal AJ (2011) Comparative evaluation of range sensor accuracy in indoor environments. In: Proceedings of the European conference on mobile robots (ECMR)Google Scholar
  31. 31.
    Sun T, Neuvo Y (1994) Detail-preserving median based filters in image processing. Pattern Recogn Lett 15(4):341–347CrossRefGoogle Scholar
  32. 32.
    Tomasi C, Manduchi R (1998) Bilateral filtering for gray and color images. In: Sixth international conference on computer vision, 1998, pp 839–846Google Scholar
  33. 33.
    van Teijlingen W, van den Broek EL, Könemann R, Schavemaker JGM (2012) Towards sensing behavior using the kinect. In: Proceedings of measuring behavior 2012: 8th international conference on methods and techniques in behavioral research. Utrecht, The Netherlands, pp 372–375Google Scholar
  34. 34.
    Zhang B, Allebach J (2008) Adaptive bilateral filter for sharpness enhancement and noise removal. IEEE Trans Image Process 17(5):664–678MathSciNetCrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2014

Authors and Affiliations

  • Kyis Essmaeel
    • 1
    • 2
    • 3
  • Luigi Gallo
    • 2
  • Ernesto Damiani
    • 3
  • Giuseppe De Pietro
    • 2
  • Albert Dipanda
    • 1
  1. 1.Laboratoire LE2I, Aile des Sciences de lʼIngénieurUniversité de BourgogneDijon CedexFrance
  2. 2.ICAR-CNRNaplesItaly
  3. 3.Department of Computer TechnologyUniversity of MilanMilanItaly

Personalised recommendations