Real-Time Range Imaging in Health Care: A Survey

  • Sebastian Bauer
  • Alexander Seitel
  • Hannes Hofmann
  • Tobias Blum
  • Jakob Wasza
  • Michael Balda
  • Hans-Peter Meinzer
  • Nassir Navab
  • Joachim Hornegger
  • Lena Maier-Hein


The recent availability of dynamic, dense, and low-cost range imaging has gained widespread interest in health care. It opens up new opportunities and has an increasing impact on both research and commercial activities. This chapter presents a state-of-the-art survey on the integration of modern range imaging sensors into medical applications. The scope is to identify promising applications and methods, and to provide an overview of recent developments in this rapidly evolving domain. The survey covers a broad range of topics, including guidance in computer-assisted interventions, operation room monitoring and workflow analysis, touch-less interaction and on-patient visualization, as well as prevention and support in elderly care and rehabilitation. We put emphasis on dynamic and interactive tasks where real-time and dense 3-D imaging forms the key aspect. While considering different range imaging modalities that fulfill these requirements, we particularly investigate the impact of Time-of-Flight imaging in this domain. Eventually, we discuss practical demands and limitations, and open research issues and challenges that are of fundamental importance for the progression of the field.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Wolf, I., Vetter, M., Wegner, I., Böttger, T., Nolden, M., Schöbinger, M., Hastenteufel, M., Kunert, T., Meinzer, H.P.: The medical imaging interaction toolkit. Med. Image Anal. 9, 594–604 (2005)CrossRefGoogle Scholar
  2. 2.
    Salvi, J., Matabosch, C., Fofi, D., Forest, J.: A review of recent range image registration methods with accuracy evaluation. Image Vis. Comput. 25(5), 578–596 (2007)CrossRefGoogle Scholar
  3. 3.
    van Kaick, O., Zhang, H., Hamarneh, G., Cohen-Or, D.: A survey on shape correspondence. Computer Graphics Forum 30(6), 1681–1707 (2011)CrossRefGoogle Scholar
  4. 4.
    Heimann, T., Meinzer, H.P.: Statistical shape models for 3D medical image segmentation: A review. Med. Image Anal. 13(4), 543–563 (2009)CrossRefGoogle Scholar
  5. 5.
    Sotiras, A., Christos, D., Paragios, N.: Deformable medical image registration: A survey. Research Report RR-7919, INRIA (2012)Google Scholar
  6. 6.
    Schaller, C., Rohkohl, C., Penne, J., Stürmer, M., Hornegger, J.: Inverse C-arm positioning for interventional procedures using real-time body part detection. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009, Part I. LNCS, vol. 5761, pp. 549–556. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  7. 7.
    Grimm, R., Bauer, S., Sukkau, J., Hornegger, J., Greiner, G.: Markerless estimation of patient orientation, posture and pose using range and pressure imaging. Int. J. Comput. Assist. Radiol. Surg. 7(6), 921–929 (2012)CrossRefGoogle Scholar
  8. 8.
    Bauer, S., Wasza, J., Haase, S., Marosi, N., Hornegger, J.: Multi-modal surface registration for markerless initial patient setup in radiation therapy using Microsoft’s Kinect sensor. In: ICCV Workshop on Consumer Depth Cameras for Computer Vision, pp. 1175–1181. IEEE (2011)Google Scholar
  9. 9.
    Schöffel, P.J., Harms, W., Sroka-Perez, G., Schlegel, W., Karger, C.P.: Accuracy of a commercial optical 3D surface imaging system for realignment of patients for radiotherapy of the thorax. Phys. Med. Biol. 52(13), 3949–3963 (2007)CrossRefGoogle Scholar
  10. 10.
    Placht, S., Stancanello, J., Schaller, C., Balda, M., Angelopoulou, E.: Fast time-of-flight camera based surface registration for radiotherapy patient positioning. Med. Phys. 39(1), 4–17 (2012)CrossRefGoogle Scholar
  11. 11.
    Wasza, J., Bauer, S., Hornegger, J.: Real-time motion compensated patient positioning and non-rigid deformation estimation using 4-D shape priors. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part II. LNCS, vol. 7511, pp. 576–583. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  12. 12.
    Lindl, B.L., Müller, R.G., Lang, S., Lablanca, M.D.H., Klöck, S.: Topos: A new topometric patient positioning and tracking system for radiation therapy based on structured white light. Med. Phys. 40(4), 042701 (2013)Google Scholar
  13. 13.
    Brahme, A., Nyman, P., Skatt, B.: 4D laser camera for accurate patient positioning, collision avoidance, image fusion and adaptive approaches during diagnostic and therapeutic procedures. Med. Phys. 35(5), 1670–1681 (2008)CrossRefGoogle Scholar
  14. 14.
    Ettl, S., Fouladi-Movahed, S., Bauer, S., Arold, O., Willomitzer, F., Huber, F., Rampp, S., Stefan, H., Hornegger, J., Häusler, G.: Medical applications enabled by a motion-robust optical 3D sensor. In: DGaO Conference (2012)Google Scholar
  15. 15.
    Schaller, C., Adelt, A., Penne, J., Hornegger, J.: Time-of-flight sensor for patient positioning. In: Samei, E., Hsieh, J. (eds.) SPIE Medical Imaging, vol. 7258, p. 726110 (2009)Google Scholar
  16. 16.
    Besl, J., McKay, N.: A method for registration of 3-D shapes. IEEE Trans. Pattern Anal. Mach. Intell. 14(2), 239–256 (1992)CrossRefGoogle Scholar
  17. 17.
    Chen, Y., Medioni, G.: Object modelling by registration of multiple range images. Image Vis. Comput. 10(3), 145–155 (1992)CrossRefGoogle Scholar
  18. 18.
    Bauer, S., Berkels, B., Ettl, S., Arold, O., Hornegger, J., Rumpf, M.: Marker-less reconstruction of dense 4-D surface motion fields using active laser triangulation for respiratory motion management. In: Ayache, N., Delingette, H., Golland, P., Mori, K. (eds.) MICCAI 2012, Part I. LNCS, vol. 7510, pp. 414–421. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  19. 19.
    Keall, P.J., Mageras, G.S., Balter, J.M., Emery, R.S., Forster, K.M., Jiang, S.B., Kapatoes, J.M., Low, D.A., Murphy, M.J., Murray, B.R., Ramsey, C.R., Herk, M.B.V., Vedam, S.S., Wong, J.W., Yorke, E.: The management of respiratory motion in radiation oncology report of AAPM task group 76. Med. Phys. 33(10), 3874–3900 (2006)CrossRefGoogle Scholar
  20. 20.
    Verellen, D., Depuydt, T., Gevaert, T., Linthout, N., Tournel, K., Duchateau, M., Reynders, T., Storme, G., Ridder, M.D.: Gating and tracking, 4D in thoracic tumours. Cancer Radiother. 14(67), 446–454 (2010)CrossRefGoogle Scholar
  21. 21.
    Schaller, C., Penne, J., Hornegger, J.: Time-of-Flight Sensor for Respiratory Motion Gating. Med. Phys. 35(7), 3090–3093 (2008)CrossRefGoogle Scholar
  22. 22.
    Xia, J., Siochi, R.A.: A real-time respiratory motion monitoring system using kinect: Proof of concept. Med. Phys. 39(5), 2682–2685 (2012)CrossRefGoogle Scholar
  23. 23.
    Alnowami, M., Alnwaimi, B., Tahavori, F., Copland, M., Wells, K.: A quantitative assessment of using the kinect for Xbox360 for respiratory surface motion tracking. In: SPIE Medical Imaging, pp. 83161T–10 (2012)Google Scholar
  24. 24.
    Yan, H., Yin, F.F., Zhu, G.P., Ajlouni, M., Kim, J.H.: The correlation evaluation of a tumor tracking system using multiple external markers. Med. Phys. 33(11), 4073–4084 (2006)CrossRefGoogle Scholar
  25. 25.
    Fayad, H., Pan, T., Clement, J.F., Visvikis, D.: Correlation of respiratory motion between external patient surface and internal anatomical landmarks. Med. Phys. 38(6), 3157–3164 (2011)CrossRefGoogle Scholar
  26. 26.
    McClelland, J., Hawkes, D., Schaeffter, T., King, A.: Respiratory motion models: A review. Med. Image Anal. 17(1), 19–42 (2013)CrossRefGoogle Scholar
  27. 27.
    Bauer, S., Berkels, B., Hornegger, J., Rumpf, M.: Joint ToF image denoising and registration with a ct surface in radiation therapy. In: Bruckstein, A.M., ter Haar Romeny, B.M., Bronstein, A.M., Bronstein, M.M. (eds.) SSVM 2011. LNCS, vol. 6667, pp. 98–109. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  28. 28.
    Schaerer, J., Fassi, A., Riboldi, M., Cerveri, P., Baroni, G., Sarrut, D.: Multi-dimensional respiratory motion tracking from markerless optical surface imaging based on deformable mesh registration. Phys. Med. Biol. 57(2), 357–373 (2012)CrossRefGoogle Scholar
  29. 29.
    Bauer, S., Wasza, J., Hornegger, J.: Photometric estimation of 3D surface motion fields for respiration management. In: Tolxdorff, T., Deserno, T.M., Handels, H., Meinzer, H.P. (eds.) Bildverarbeitung für die Medizin, pp. 105–110. Springer (2012)Google Scholar
  30. 30.
    Wasza, J., Bauer, S., Haase, S., Hornegger, J.: Sparse principal axes statistical surface deformation models for respiration analysis and classification. In: Tolxdorff, T., Deserno, T.M., Handels, H., Meinzer, H.P. (eds.) Bildverarbeitung für die Medizin, pp. 316–321. Springer (2012)Google Scholar
  31. 31.
    Gianoli, C., Riboldi, M., Spadea, M.F., Travaini, L.L., Ferrari, M., Mei, R., Orecchia, R., Baroni, G.: A multiple points method for 4D CT image sorting. Med. Phys. 38(2), 656–667 (2011)CrossRefGoogle Scholar
  32. 32.
    Bettinardi, V., Bernardi, E.D., Presotto, L., Gilardi, M.: Motion-tracking hardware and advanced applications in PET and PET/CT. PET Clinics 8(1), 11–28 (2013)CrossRefGoogle Scholar
  33. 33.
    Alnowami, M.R., Lewis, E., Guy, M., Wells, K.: An observation model for motion correction in nuclear medicine. In: SPIE Medical Imaging, pp. 76232F–9 (2010)Google Scholar
  34. 34.
    Bruyant, P., Gennert, M.A., Speckert, G., Beach, R., Morgenstern, J., Kumar, N., Nadella, S., King, M.: A robust visual tracking system for patient motion detection in SPECT: Hardware solutions. IEEE Trans. Nucl. Sci. 52(5), 1288–1294Google Scholar
  35. 35.
    McNamara, J.E., Pretorius, P.H., Johnson, K., Mukherjee, J.M., Dey, J., Gennert, M.A., King, M.A.: A flexible multicamera visual-tracking system for detecting and correcting motion-induced artifacts in cardiac SPECT slices. Med. Phys. 36(5), 1913–1923 (2009)CrossRefGoogle Scholar
  36. 36.
    Olesen, O.V., Jorgensen, M.R., Paulsen, R.R., Hojgaard, L., Roed, B., Larsen, R.: Structured light 3D tracking system for measuring motions in PET brain imaging. In: SPIE Medical Imaging, pp. 76250X–11 (2010)Google Scholar
  37. 37.
    Noonan, P., Howard, J., Tout, D., Armstrong, I., Williams, H., Cootes, T., Hallett, W., Hinz, R.: Accurate markerless respiratory tracking for gated whole body PET using the Microsoft Kinect. In: IEEE NSS-MIC (2012)Google Scholar
  38. 38.
    Cash, D.M., Sinha, T.K., Chapman, W.C., Terawaki, H., Dawant, B.M., Galloway, R.L., Miga, M.I.: Incorporation of a laser range scanner into image-guided liver surgery: Surface acquisition, registration, and tracking. Med. Phys. 30(7), 1671–1682 (2003)CrossRefGoogle Scholar
  39. 39.
    Cash, D.M., Miga, M.I., Glasgow, S.C., Dawant, B.M., Clements, L.W., Cao, Z., Galloway, R.L., Chapman, W.C.: Concepts and preliminary data toward the realization of image-guided liver surgery. J. Gastrointest. Surg. 11, 844–859 (2007)CrossRefGoogle Scholar
  40. 40.
    Cash, D.M., Miga, M.I., Sinha, T.K., Galloway, R.L., Chapman, W.C.: Compensating for intraoperative soft-tissue deformations using incomplete surface data and finite elements. IEEE Trans. Med. Imaging 24(11), 1479–1491 (2005)CrossRefGoogle Scholar
  41. 41.
    Seitel, A.: Markerless Navigation For Percutaneus Needle Insertions. PhD thesis, Universität Heidelberg (2012)Google Scholar
  42. 42.
    Mersmann, S., Müller, M., Seitel, A., Arnegger, F., Tetzlaff, R., Dinkel, J., Baumhauer, M., Schmied, B., Meinzer, H.P., Maier-Hein, L.: Time-of-flight camera technology for augmented reality in computer-assisted interventions. In: Wong, K.H., Holmes, D.R. (eds.) SPIE Medical Imaging, p. 79642C (2011)Google Scholar
  43. 43.
    Baumhauer, M., Simpfendörfer, T., Stich, B.M., Teber, D., Gutt, C., Rassweiler, J., Meinzer, H.P., Wolf, I.: Soft tissue navigation for laparoscopic partial nephrectomy. Int. J. Comput. Assist. Radiol. Surg. 3, 307–314 (2008)CrossRefGoogle Scholar
  44. 44.
    dos Santos, T.R.: Muti-Modal Partial Surface Matching For Intraoperative Registration. PhD thesis, Universität Heidelberg (2012)Google Scholar
  45. 45.
    Wang, X.L., Stolka, P.J., Boctor, E., Hager, G., Choti, M.: The Kinect as an interventional tracking system. In: SPIE Medical Imaging, pp. 83160U–6 (2012)Google Scholar
  46. 46.
    Nicolau, S., Brenot, J., Goffin, L., Graebling, P., Soler, L., Marescaux, J.: A structured light system to guide percutaneous punctures in interventional radiology. In: SPIE Medical Imaging, p. 700016 (2008)Google Scholar
  47. 47.
    Mirota, D.J., Ishii, M., Hager, G.D.: Vision-based navigation in image-guided interventions. Annu. Rev. Biomed. Eng. 13(13), 297–319 (2011)CrossRefGoogle Scholar
  48. 48.
    Stoyanov, D., Mylonas, G.P., Deligianni, F., Darzi, A., Yang, G.Z.: Soft-tissue motion tracking and structure estimation for robotic assisted MIS procedures. In: Duncan, J.S., Gerig, G. (eds.) MICCAI 2005. LNCS, vol. 3750, pp. 139–146. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  49. 49.
    Collins, T., Bartoli, A.: Towards live monocular 3D laparoscopy using shading and specularity information. In: Abolmaesumi, P., Joskowicz, L., Navab, N., Jannin, P. (eds.) IPCAI 2012. LNCS, vol. 7330, pp. 11–21. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  50. 50.
    Malti, A., Bartoli, A., Collins, T.: Template-based conformal shape-from-motion-and-shading for laparoscopy. In: Abolmaesumi, P., Joskowicz, L., Navab, N., Jannin, P. (eds.) IPCAI 2012. LNCS, vol. 7330, pp. 1–10. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  51. 51.
    Mountney, P., Stoyanov, D., Yang, G.Z.: Three-dimensional tissue deformation recovery and tracking. IEEE Signal Proc. Mag. 27, 14–24 (2010)CrossRefGoogle Scholar
  52. 52.
    Clancy, N.T., Stoyanov, D., Yang, G.Z., Elson, D.S.: An endoscopic structured lighting probe using spectral encoding. In: SPIE Novel Biophotonic Techniques and Applications, vol. 8090 (2011)Google Scholar
  53. 53.
    Schmalz, C., Forster, F., Schick, A., Angelopoulou, E.: An endoscopic 3D scanner based on structured light. Med. Image Anal. 16(5), 1063–1072 (2012)CrossRefGoogle Scholar
  54. 54.
    Maier-Hein, L., Mountney, P., Bartoli, A., Elhawary, H., Elson, D., Groch, A., Kolb, A., Rodrigues, M., Sorger, J., Speidel, S., Stoyanov, D.: Optical techniques for 3D surface reconstruction in computer-assisted laparoscopic surgery. Med. Image Anal. (in press, 2013)Google Scholar
  55. 55.
    Penne, J., Höller, K., Stürmer, M., Schrauder, T., Schneider, A., Engelbrecht, R., Feußner, H., Schmauss, B., Hornegger, J.: Time-of-flight 3-D endoscopy. In: Yang, G.-Z., Hawkes, D., Rueckert, D., Noble, A., Taylor, C. (eds.) MICCAI 2009, Part I. LNCS, vol. 5761, pp. 467–474. Springer, Heidelberg (2009)CrossRefGoogle Scholar
  56. 56.
    Groch, A., Seitel, A., Hempel, S., Speidel, S., Engelbrecht, R., Penne, J., Höller, K., Röhl, S., Yung, K., Bodenstedt, S., Pflaum, F., dos Santos, T., Mersmann, S., Meinzer, H.P., Hornegger, J., Maier-Hein, L.: 3D surface reconstruction for laparoscopic computer-assisted interventions: Comparison of state-of-the-art methods. In: SPIE Medical Imaging, vol. 796415 (2011)Google Scholar
  57. 57.
    Haase, S., Wasza, J., Kilgus, T., Hornegger, J.: Laparoscopic instrument localization using a 3-D Time-of-Flight/RGB endoscope. In: Workshop on the Applications of Computer Vision, pp. 449–454. IEEE (2013)Google Scholar
  58. 58.
    Groch, A., Haase, S., Wagner, M., Kilgus, T., Kenngott, H., Schlemmer, H.P., Hornegger, J., Meinzer, H.P., Maier-Hein, L.: A probabilistic approach to fusion of Time-of-Flight and multiple view based 3D surface reconstruction for laparoscopic interventions. Int. J. Comput. Assist. Radiol. Surg. 7, S397–S398 (2012)Google Scholar
  59. 59.
    Kolb, C., Groch, A., Seitel, A., Kilgus, T., Haase, S., Bendl, R., Meinzer, H.P., Hornegger, J., Maier-Hein, L.: Simultaneous localization and soft-tissue shape recovery with a time of flight endoscope for computer-assisted surgery. Int. J. Comput. Assist. Radiol. Surg. (in press, 2013)Google Scholar
  60. 60.
    Ladikos, A., Benhimane, S., Navab, N.: Real-time 3D reconstruction for collision avoidance in interventional environments. In: Metaxas, D., Axel, L., Fichtinger, G., Székely, G. (eds.) MICCAI 2008, Part II. LNCS, vol. 5242, pp. 526–534. Springer, Heidelberg (2008)CrossRefGoogle Scholar
  61. 61.
    Navab, N., Holzer, S.: Real-time 3D reconstruction: Applications to collision detection and surgical workflow monitoring. In: IROS Workshop on Methods for Safer Surgical Robotics Procedures (2011)Google Scholar
  62. 62.
    SAFROS project,
  63. 63.
    ACTIVE project,
  64. 64.
    Mönnich, H., Nicolai, P., Raczkowsky, J., Wörn, H.: A semi-autonomous robotic teleoperation surgery setup with multi 3D camera supervision. Int. J. Comput. Assist. Radiol. Surg., 132–133 (2011)Google Scholar
  65. 65.
    Nicolai, P., Raczkowsky, J.: Operation room supervision for safe robotic surgery with a multi 3D-camera setup. In: IROS Workshop on Methods for Safer Surgical Robotics Procedures (2011)Google Scholar
  66. 66.
    Katic, D., Wekerle, A.L., Gärtner, F., Kenngott, H., Müller-Stich, B.P., Dillmann, R., Speidel, S.: Ontology-based prediction of surgical events in laparoscopic surgery. In: SPIE Medical Imaging, pp. 86711A–7 (2013)Google Scholar
  67. 67.
    Padoy, N., Mateus, D., Weinland, D., Berger, M.O., Navab, N.: Workflow monitoring based on 3D motion features. In: ICCV Workshop on Video-oriented Object and Event Classification, pp. 585–592. IEEE (2009)Google Scholar
  68. 68.
    Lea, C.S., Fackler, J.C., Hager, G.D., Taylor, R.H.: Towards automated activity recognition in an intensive care unit. In: MICCAI Workshop on Modeling and Monitoring of Computer Assisted Interventions, pp. 19–28 (2012)Google Scholar
  69. 69.
    Ladikos, A., Cagniart, C., Ghotbi, R., Reiser, M., Navab, N.: Estimating radiation exposure in interventional environments. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part III. LNCS, vol. 6363, pp. 237–244. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  70. 70.
    Gratzel, C., Fong, T., Grange, S., Baur, C.: A non-contact mouse for surgeon-computer interaction. Technology and Health Care – European Society for Engineering and Medicine 12(3), 245–258 (2004)Google Scholar
  71. 71.
    Soutschek, S., Penne, J., Hornegger, J., Kornhuber, J.: 3-D gesture-based scene navigation in medical imaging applications using Time-Of-Flight cameras. In: CVPR Workshop on Time of Flight Camera based Computer Vision, pp. 1–6. IEEE (2008)Google Scholar
  72. 72.
    Ruppert, G., Reis, L., Amorim, P., de Moraes, T., da Silva, J.: Touchless gesture user interface for interactive image visualization in urological surgery. World J. Urol. 30, 1–5 (2012)CrossRefGoogle Scholar
  73. 73.
    Gallo, L., Placitelli, A.P., Ciampi, M.: Controller-free exploration of medical image data: Experiencing the Kinect. In: International Symposium on Computer-Based Medical Systems, pp. 1–6. IEEE (2011)Google Scholar
  74. 74.
    Jacob, M., Cange, C., Packer, R., Wachs, J.P.: Intention, context and gesture recognition for sterile MRI navigation in the operating room. In: Alvarez, L., Mejail, M., Gomez, L., Jacobo, J. (eds.) CIARP 2012. LNCS, vol. 7441, pp. 220–227. Springer, Heidelberg (2012)CrossRefGoogle Scholar
  75. 75.
    Kirmizibayrak, C., Radeva, N., Wakid, M., Philbeck, J., Sibert, J., Hahn, J.: Evaluation of gesture based interfaces for medical volume visualization tasks. In: International Conference on Virtual Reality Continuum and Its Applications in Industry, pp. 69–74. ACM (2011)Google Scholar
  76. 76.
    Ebert, L., Hatch, G., Ampanozi, G., Thali, M., Ross, S.: You can’t touch this: Touch-free navigation through radiological images. Surg. Innov. 19(3), 301–307 (2012)CrossRefGoogle Scholar
  77. 77.
    Bigdelou, A., Stauder, R., Benz, T., Okur, A., Blum, T., Ghotbi, R., Navab, N.: HCI design in the OR: A gesturing case-study. In: MICCAI Workshop on Modeling and Monitoring of Computer Assisted Interventions, Springer, pp. 10–18. Springer (2012)Google Scholar
  78. 78.
    Karl Storz GmbH, Tuttlingen, Germany,
  79. 79.
    Bigdelou, A., Benz, T., Schwarz, L., Navab, N.: Simultaneous categorical and spatio-temporal 3D gestures using Kinect. In: Symposium on 3D User Interfaces, pp. 53–60. IEEE (2012)Google Scholar
  80. 80.
    Dressler, C., Neumuth, T., Fischer, M., Abri, O., Strauss, G.: Intraoperative Bedienung einer elektronischen Patientenakte durch den Operateur. HNO 59(9), 900–907 (2011)CrossRefGoogle Scholar
  81. 81.
    Navab, N., Traub, J., Sielhorst, T., Feuerstein, M., Bichlmeier, C.: Action- and workflow-driven augmented reality for computer-aided medical procedures. IEEE Comput. Graph. Appl. 27(5), 10–14 (2007)CrossRefGoogle Scholar
  82. 82.
    Sugimoto, M., Yasuda, H., Koda, K., Suzuki, M., Yamazaki, M., Tezuka, T., Kosugi, C., Higuchi, R., Watayo, Y., Yagawa, Y., Uemura, S., Tsuchiya, H., Azuma, T.: Image overlay navigation by markerless surface registration in gastrointestinal, hepatobiliary and pancreatic surgery. J. Hepatobiliary Pancreat Sci. 17(5), 629–636 (2010)CrossRefGoogle Scholar
  83. 83.
    Maier-Hein, L., Franz, A.M., Fangerau, M., Schmidt, M., Seitel, A., Mersmann, S., Kilgus, T., Groch, A., Yung, K., dos Santos, T.R., Meinzer, H.P.: Towards mobile augmented reality for on-patient visualization of medical images. In: Bildverarbeitung für die Medizin, pp. 389–393. Springer (2011)Google Scholar
  84. 84.
    Blum, T., Kleeberger, V., Bichlmeier, C., Navab, N.: mirracle: An augmented reality magic mirror system for anatomy education. In: Virtual Reality, pp. 115–116. IEEE (2012)Google Scholar
  85. 85.
    Maier-Hein, L., Schmidt, M., Franz, A., dos Santos, T., Seitel, A., Jähne, B., Fitzpatrick, J., Meinzer, H.: Accounting for anisotropic noise in fine registration of time-of-flight range data with high-resolution surface data. In: Jiang, T., Navab, N., Pluim, J.P.W., Viergever, M.A. (eds.) MICCAI 2010, Part I. LNCS, vol. 6361, pp. 251–258. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  86. 86.
    Maier-Hein, L., Franz, A., dos Santos, T., Schmidt, M., Fangerau, M., Meinzer, H.P., Fitzpatrick, J.M.: Convergent iterative closest-point algorithm to accomodate anisotropic and inhomogenous localization error. IEEE Trans. Pattern Anal. Mach. Intell. 34(8), 1520–1532 (2012)CrossRefGoogle Scholar
  87. 87.
    Kilgus, T., Franz, A.M., Seitel, A., März, K., Bartha, L., Fangerau, M., Mersmann, S., Groch, A., Meinzer, H.P., Maier-Hein, L.: Registration of partially overlapping surfaces for range image based augmented reality on mobile devices. In: SPIE Medical Imaging, p. 83160T (2012)Google Scholar
  88. 88.
    Gabel, M., Gilad-Bachrach, R., Renshaw, E., Schuster, A.: Full body gait analysis with Kinect. In: International Conference of Engineering in Medicine and Biology Society, pp. 1964–1967. IEEE (2012)Google Scholar
  89. 89.
    Parra-Dominguez, G., Taati, B., Mihailidis, A.: 3D human motion analysis to detect abnormal events on stairs. In: International Conference on 3D Imaging, Modeling, Processing, Visualization and Transmission, pp. 97–103 (2012)Google Scholar
  90. 90.
    Garcia, J.A., Navarro, K.F., Schoene, D., Smith, S.T., Pisan, Y.: Exergames for the elderly: towards an embedded Kinect-based clinical test of falls risk. Studies in Health Technology and Informatics. In: Health Informatics: Building a Healthcare Future Through Trusted Information, pp. 51–57. IOS (2012)Google Scholar
  91. 91.
    Parajuli, M., Tran, D., Ma, W., Sharma, D.: Senior health monitoring using Kinect. In: International Conference on Communications and Electronics, pp. 309–312 (2012)Google Scholar
  92. 92.
    Stone, E., Skubic, M.: Evaluation of an inexpensive depth camera for in-home gait assessment. J. Ambient Intell. Smart Environ. 3(4), 349–361 (2011)Google Scholar
  93. 93.
    Stone, E., Skubic, M.: Passive in-home measurement of stride-to-stride gait variability comparing vision and Kinect sensing. In: International Conference of Engineering in Medicine and Biology Society, pp. 6491–6494 (2011)Google Scholar
  94. 94.
    Gross, H., Schroeter, C., Mueller, S., Volkhardt, M., Einhorn, E., Bley, A., Martin, C., Langner, T., Merten, M.: Progress in developing a socially assistive mobile home robot companion for the elderly with mild cognitive impairment. In: IEEE/RSJ IROS, pp. 2430–2437 (2011)Google Scholar
  95. 95.
    Lowet, D., Isken, M., Lee, W., van Heesch, F., Eertink, E.: Robotic telepresence for 24/07 remote assistance to elderly at home, workshop on social robotic telepresence. In: International Symposium on Robot and Human Interactive Communication. IEEE (2012)Google Scholar
  96. 96.
    Woo, J., Wada, K., Kubota, N.: Robot partner system for elderly people care by using sensor network. In: International Conference on Biomedical Robotics and Biomechatronics, IEEE, RAS, EMBS, pp. 1329–1334 (2012)Google Scholar
  97. 97.
    Shotton, J., Girshick, R., Fitzgibbon, A., Sharp, T., Cook, M., Finocchio, M., Moore, R., Kohli, P., Criminisi, A., Kipman, A., Blake, A.: Efficient human pose estimation from single depth images. IEEE Trans. Pattern Anal. Mach. Intell. 99 (2012) (PrePrints)Google Scholar
  98. 98.
    Ganapathi, V., Plagemann, C., Koller, D., Thrun, S.: Real time motion capture using a single Time-of-Flight camera. In: CVPR, pp. 755–762. IEEE (2010)Google Scholar
  99. 99.
    Schwarz, L., Mkhitaryan, A., Mateus, D., Navab, N.: Estimating human 3D pose from Time-of-Flight images based on geodesic distances and optical flow. In: International Conference on Automatic Face Gesture Recognition and Workshops, pp. 700–706. IEEE (2011)Google Scholar
  100. 100.
    Coronato, A., Gallo, L.: Towards abnormal behavior detection of cognitive impaired people. In: International Conference on Pervasive Computing and Communications Workshops, pp. 859–864. IEEE (2012)Google Scholar
  101. 101.
    Sivalingam, R., Cherian, A., Fasching, J., Walczak, N., Bird, N.D., Morellas, V., Murphy, B., Cullen, K., Lim, K., Sapiro, G., Papanikolopoulos, N.: A multi-sensor visual tracking system for behavior monitoring of at-risk children. In: ICRA, pp. 1345–1350 (2012)Google Scholar
  102. 102.
    Walczak, N., Fasching, J., Toczyski, W.D., Sivalingam, R., Bird, N.D., Cullen, K., Morellas, V., Murphy, B., Sapiro, G., Papanikolopoulos, N.: A nonintrusive system for behavioral analysis of children using multiple RGB+depth sensors. In: Workshop on the Applications of Computer Vision, pp. 217–222 (2012)Google Scholar
  103. 103.
    Falie, D., Ichim, M., David, L.: Respiratory motion visualization and the sleep apnea diagnosis with the time of flight (ToF) camera. In: International Conference on Visualization, Imaging and Simulation, WSEAS, pp. 179–184 (2008)Google Scholar
  104. 104.
    Yu, M.C., Wu, H., Liou, J.L., Lee, M.S., Hung, Y.P.: Breath and position monitoring during sleeping with a depth camera. In: HEALTHINF, pp. 12–22 (2012)Google Scholar
  105. 105.
    Smith, S.T., Schoene, D.: The use of exercise-based videogames for training and rehabilitation of physical function in older adults: current practice and guidelines for future research. Aging Health 8(3), 243–252 (2012)CrossRefGoogle Scholar
  106. 106.
    Virtualware Group, Basauri, Spain,
  107. 107.
    Jintronix, Inc., Montreal, QC, Canada,
  108. 108.
    Chang, Y.J., Chen, S.F., Huang, J.D.: A Kinect-based system for physical rehabilitation: A pilot study for young adults with motor disabilities. Research in Developmental Disabilities 32(6), 2566–2570 (2011)CrossRefGoogle Scholar
  109. 109.
    da Gama, A., Chaves, T., Figueiredo, L., Teichrieb, V.: Improving motor rehabilitation process through a natural interaction based system using Kinect sensor. In: IEEE Symposium on 3D User Interfaces, pp. 145–146 (2012)Google Scholar
  110. 110.
    Huang, J.D.: Kinerehab: A Kinect-based system for physical rehabilitation: a pilot study for young adults with motor disabilities. In: International ACM SIGACCESS Conference on Computers and Accessibility, pp. 319–320. ASSETS (2011)Google Scholar
  111. 111.
    Soutschek, S., Maier, A., Bauer, S., Kugler, P., Bebenek, M., Steckmann, S., von Stengel, S., Kemmler, W., Hornegger, J., Kornhuber, J.: Measurement of angles in Time-of-Flight data for the automatic supervision of training exercises. In: IEEE Conference on Pervasive Computing Technologies for Healthcare, pp. 1–4 (2010)Google Scholar
  112. 112.
    Schoenauer, C., Pintaric, T., Kaufmann, H., Jansen Kosterink, S., Vollenbroek-Hutten, M.: Chronic pain rehabilitation with a serious game using multimodal input. In: International Conference on Virtual Rehabilitation, pp. 1–8 (2011)Google Scholar
  113. 113.
    Chang, C.Y., Lange, B., Zhang, M., Koenig, S., Requejo, P., Somboon, N., Sawchuk, A.A., Rizzo, A.A.: Towards pervasive physical rehabilitation using Microsoft Kinect. In: International Conference on Pervasive Computing Technologies for Healthcare, pp. 159–162 (2012)Google Scholar
  114. 114.
    Lange, B., Chang, C.Y., Suma, E., Newman, B., Rizzo, A., Bolas, M.: Development and evaluation of low cost game-based balance rehabilitation tool using the Microsoft Kinect sensor. In: International Conference of Engineering in Medicine and Biology Society, pp. 1831–1834. IEEE (2011)Google Scholar
  115. 115.
    Hersh, M., Johnson, M., Keating, D.: Assistive Technology for Visually Impaired and Blind People. Springer (2007)Google Scholar
  116. 116.
    Gallo, S., Chapuis, D., Santos-Carreras, L., Kim, Y., Retornaz, P., Bleuler, H., Gassert, R.: Augmented white cane with multimodal haptic feedback. In: International Conference on Biomedical Robotics and Biomechatronics, IEEE, RAS, EMBS, pp. 149–155 (2010)Google Scholar
  117. 117.
    Gassert, R., Kim, Y., Oggier, T., Riesch, M., Deschler, M., Prott, C., Schneller, S., Hayward, V.: White cane with integrated electronic travel aid using 3D TOF sensor, Patent WO 2012/040703 (2012)Google Scholar
  118. 118.
    Katz, B., Kammoun, S., Parseihian, G., Gutierrez, O., Brilhault, A., Auvray, M., Truillet, P., Denis, M., Thorpe, S., Jouffrais, C.: Navig: Augmented reality guidance system for the visually impaired. Virtual Reality 16, 253–269 (2012)CrossRefGoogle Scholar
  119. 119.
    Ong, S.K., Zhang, J., Nee, A.Y.C.: Assistive obstacle detection and navigation devices for vision-impaired users. Disability and Rehabilitation: Assistive Technology (2013) (Epub ahead of print)Google Scholar
  120. 120.
    IS2you, Santa Maria, Portugal,
  121. 121.
    Buttgen, B.: Extending Time-of-Flight optical 3D-imaging to extreme operating conditions. PhD thesis, Universite de Neuchatel (2007)Google Scholar
  122. 122.
    Maimone, A., Fuchs, H.: Reducing interference between multiple structured light depth sensors using motion. In: IEEE Virtual Reality, pp. 51–54 (2012)Google Scholar
  123. 123.
    Roggan, A., Friebel, M., Dörschel, K., Hahn, A., Müller, G.: Optical properties of circulating human blood in the wavelength range 400-2500 nm. J. Biomed. Opt. 4(1), 36–46 (1999)CrossRefGoogle Scholar
  124. 124.
    Fuchs, S.: Multipath interference compensation in time-of-flight camera images. In: ICPR, pp. 3583–3586 (2010)Google Scholar
  125. 125.
    Dorrington, A.A., Godbaz, J.P., Cree, M.J., Payne, A.D., Streeter, L.V.: Separating true range measurements from multi-path and scattering interference in commercial range cameras. In: SPIE Electronic Imaging, pp. 786404–786410 (2011)Google Scholar
  126. 126.
    Wu, D., O’Toole, M., Velten, A., Agrawal, A., Raskar, R.: Decomposing global light transport using time of flight imaging. In: CVPR, pp. 366–373. IEEE (2012)Google Scholar
  127. 127.
    Bert, C., Metheany, K.G., Doppke, K., Chen, G.T.Y.: A phantom evaluation of a stereo-vision surface imaging system for radiotherapy patient setup. Med. Phys. 32(9), 2753–2762 (2005)CrossRefGoogle Scholar
  128. 128.
    Lange, R.: 3D Time-of-Flight Distance Measurement with Custom Solid-State Image Sensors in CMOS/CCD-Technology. PhD thesis, University of Siegen (2000)Google Scholar
  129. 129.
    Seitel, A., Yung, K., Mersmann, S., Kilgus, T., Groch, A., Santos, T., Franz, A., Nolden, M., Meinzer, H.P., Maier-Hein, L.: MITK-ToF - range data within MITK. Int. J. Comput. Assist. Radiol. Surg. 7, 87–96 (2012)CrossRefGoogle Scholar
  130. 130.
    Wolf, I., Vetter, M., Wegner, I., Böttger, T., Nolden, M., Schöbinger, M., Hastenteufel, M., Kunert, T., Meinzer, H.P.: The medical imaging interaction toolkit. Med. Image Anal. 9, 594–604 (2005)CrossRefGoogle Scholar
  131. 131.
    Wasza, J., Bauer, S., Haase, S., Schmid, M., Reichert, S., Hornegger, J.: RITK: The range imaging toolkit - a framework for 3-D range image stream processing. In: Eisert, P., Hornegger, J., Polthier, K. (eds.) International Workshop on Vision, Modeling and Visualization, pp. 57–64 (2011)Google Scholar
  132. 132.
    Ibanez, L., Schroeder, W., Ng, L., Cates, J.: The ITK Software Guide, 2nd edn. Kitware, Inc. (2005)Google Scholar
  133. 133.
    Rusu, R.B., Cousins, S.: 3D is here: Point cloud library (PCL). In: ICRA, pp. 1–4 (2011)Google Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2013

Authors and Affiliations

  • Sebastian Bauer
    • 1
  • Alexander Seitel
    • 2
    • 3
  • Hannes Hofmann
    • 4
  • Tobias Blum
    • 5
  • Jakob Wasza
    • 1
  • Michael Balda
    • 4
  • Hans-Peter Meinzer
    • 3
  • Nassir Navab
    • 5
  • Joachim Hornegger
    • 1
  • Lena Maier-Hein
    • 2
    • 3
  1. 1.Pattern Recognition LabFriedrich-Alexander-Universität Erlangen-NürnbergErlangenGermany
  2. 2.Junior group: Computer-assisted InterventionsGermany
  3. 3.Division of Medical and Biological InformaticsGerman Cancer Research Center (DKFZ)HeidelbergGermany
  4. 4.Metrilus GmbHErlangenGermany
  5. 5.Computer Aided Medical Procedures & Augmented Reality (CAMP)Technische Universität MünchenMunichGermany

Personalised recommendations