Skip to main content
SpringerLink
Log in

Enhancing 3D Capture with Multiple Depth Camera Systems: A State-of-the-Art Report

  • Chapter
  • First Online:
RGB-D Image Analysis and Processing

Part of the book series: Advances in Computer Vision and Pattern Recognition ((ACVPR))

Abstract

Over the past decade, depth-sensing cameras rapidly found their way into consumer products and became a staple in computer vision, robotics, and 3D reconstruction systems. Under some circumstances, the use of multiple depth sensors brings unique advantages in facilitating model acquisition, such as capture from complementary points of view and higher sampling density, with the potential to reduce the effects of sensor noise. Typically, multiple camera systems allow users to obtain visual information that might be unavailable from a particular point of view in a single-camera setup. As a result of this characteristic, the use of multiple depth cameras has great potential for a number of applications. However, there are some challenges that arise when implementing multi-depth camera systems, including calibration, synchronization and registration. In this chapter, we survey how some of these challenges have been addressed and present the most comprehensive review to date of the techniques used to implement multiple depth-sensing camera systems. In addition, we present a wide array of applications supported by multiple depth camera systems (MDCs).

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 189.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 249.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 249.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  1. Ahmed N, Junejo I (2014) Using multiple RGB-D cameras for 3D video acquisition and spatio-temporally coherent 3D animation reconstruction. Int J Comput Theory Eng 6. https://doi.org/10.7763/IJCTE.2014.V6.907, http://www.ijcte.org/papers/907-AC0002.pdf

    Article  Google Scholar 

  2. Alexiadis DS, Zarpalas D, Daras P (2013) Real-time, full 3-d reconstruction of moving foreground objects from multiple consumer depth cameras. IEEE Trans Multimed 15(2):339–358. https://doi.org/10.1109/TMM.2012.2229264

    Article  Google Scholar 

  3. Alexiadis S, Kordelas G, Apostolakis KC, Agapito JD, Vegas J, Izquierdo E, Daras P (2012) Reconstruction for 3D immersive virtual environments. In: 2012 13th international workshop on image analysis for multimedia interactive services (WIAMIS), pp 1–4. IEEE. https://doi.org/10.1109/WIAMIS.2012.6226760

  4. Anand A, Koppula HS, Joachims T, Saxena A (2013) Contextually guided semantic labeling and search for three-dimensional point clouds. Int J Robot Res 32(1):19–34. https://doi.org/10.1177/0278364912461538

    Article  Google Scholar 

  5. Asteriadis S, Chatzitofis A, Zarpalas D, Alexiadis DS, Daras P (2013) Estimating human motion from multiple Kinect sensors. In: Proceedings of the 6th international conference on computer vision/computer graphics collaboration techniques and applications, MIRAGE ’13, pp 3:1–3:6. ACM, New York, NY, USA. https://doi.org/10.1145/2466715.2466727

  6. Auvinet E, Meunier J, Multon F (2012) Multiple depth cameras calibration and body volume reconstruction for gait analysis. In: 2012 11th international conference on information science, signal processing and their applications (ISSPA), pp 478–483. https://doi.org/10.1109/ISSPA.2012.6310598

  7. Baek S, Kim M (2015) Dance experience system using multiple Kinects. Int J Future Comput Commun 4(1):45–49. https://doi.org/10.7763/IJFCC.2015.V4.353, http://www.ijfcc.org/vol4/353-N039.pdf

    Article  Google Scholar 

  8. Baek S, Kim M (2017) User pose estimation based on multiple depth sensors. In: SIGGRAPH Asia 2017 Posters, SA ’17, pp. 1:1–1:2. ACM, New York, NY, USA. https://doi.org/10.1145/3145690.3145709

  9. Berger K (2013) A state of the art report on research in multiple RGB-D sensor setups. arXiv:1310.2050

  10. Berger K (2014) A state of the art report on multiple RGB-D sensor research and on publicly available RGB-D datasets, pp 27–44. https://doi.org/10.1007/978-3-319-08651-4_2

    Google Scholar 

  11. Berger K, Meister S, Nair R, Kondermann D (2013) A state of the art report on kinect sensor setups in computer vision, pp 257–272. Springer, Berlin. https://doi.org/10.1007/978-3-642-44964-2_12, http://www.grk1564.uni-siegen.de/sites/www.grk1564.uni-siegen.de/files/inm2013/kinect-star.pdf

    Google Scholar 

  12. Berger K, Ruhl K, Schroeder Y, Bruemmer C, Scholz A, Magnor M (2011) Markerless Motion Capture using multiple Color-Depth Sensors. In: Eisert P, Hornegger J, Polthier K (eds) Vision, Modeling, and Visualization. The Eurographics Association. https://doi.org/10.2312/PE/VMV/VMV11/317-324, https://graphics.tu-bs.de/upload/publications/multikinectsMocap.pdf

  13. Besl PJ, McKay ND (1992) A method for registration of 3-d shapes. IEEE Trans Pattern Anal Mach Intell 14(2):239–256. https://doi.org/10.1109/34.12179110.1109/34.121791

  14. Butler DA, Izadi S, Hilliges O, Molyneaux D, Hodges S, Kim D (2012) Shake’n’sense: reducing interference for overlapping structured light depth cameras. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’12, pp 1933–1936. ACM, New York, NY, USA. https://doi.org/10.1145/2207676.2208335

  15. Cai Z, Han J, Liu L, Shao L (2017) RGB-D datasets using Microsoft Kinect or similar sensors: a survey. Multimed Tools Appl 76(3):4313–4355. https://doi.org/10.1007/s11042-016-3374-6

    Article  Google Scholar 

  16. Calderita L, Bandera J, Bustos P, Skiadopoulos A (2013) Model-based reinforcement of Kinect depth data for human motion capture applications. Sensors 13(7):8835–8855. https://doi.org/10.3390/s130708835

    Article  Google Scholar 

  17. Chatzitofis A, Zarpalas D, Kollias S, Daras P (2019) DeepMoCap: Deep optical motion capture using multiple depth sensors and retro-reflectors. Sensors 19:282. https://doi.org/10.3390/s19020282

    Article  Google Scholar 

  18. Chen Y, Medioni G (1991) Object modeling by registration of multiple range images. In: Proceedings. 1991 IEEE international conference on robotics and automation, vol 3, pp 2724–2729. https://doi.org/10.1109/ROBOT.1991.132043

  19. Cippitelli E, Gasparrini S, Gambi E, Spinsante S, Wåhslény J, Orhany I, Lindhy T (2015) Time synchronization and data fusion for RGB-Depth cameras and inertial sensors in aal applications. In: 2015 IEEE international conference on communication workshop (ICCW), pp 265–270. https://doi.org/10.1109/ICCW.2015.7247189

  20. Collet A, Chuang M, Sweeney P, Gillett D, Evseev D, Calabrese D, Hoppe H, Kirk A, Sullivan S (2015) High-quality streamable free-viewpoint video. ACM Trans Graph 34(4):69:1–69:13. https://doi.org/10.1145/2766945

    Article  Google Scholar 

  21. Creative Commons: Creative commons attribution license (cc by 4.0). https://creativecommons.org/licenses/by/4.0/ (2019). Accessed: 2019-06-25

  22. Creative, Corp.: Creative senz3d. https://us.creative.com/p/peripherals/blasterx-senz3d (2013). Accessed 14 June 2019

  23. Crispim-Junior CF, Gomez Uria A, Strumia C, Koperski M, Koenig A, Negin F, Cosar S, Nghiem AT, Chau DP, Charpiat G, Bremond F (2017) Online recognition of daily activities by color-depth sensing and knowledge models. Sens J, MDPI 17(7):2118. https://www.ncbi.nlm.nih.gov/pubmed/28661440

    Article  Google Scholar 

  24. Czarnuch S, Ploughman M (2014) Automated gait analysis in people with multiple sclerosis using two unreferenced depth imaging sensors: preliminary steps. In: NECEC 2014, newfoundland electrical and computer engineering conference. https://doi.org/10.13140/2.1.2187.6481

  25. Deng T, Bazin JC, Martin T, Kuster C, Cai J, Popa T, Gross M (2014) Registration of multiple RGBD cameras via local rigid transformations. https://doi.org/10.1109/ICME.2014.6890122, http://www.cs.utah.edu/~martin/calibration.pdf

  26. Desai K, Prabhakaran B, Raghuraman S (2018) Combining skeletal poses for 3D human model generation using multiple Kinects. In: Proceedings of the 9th ACM multimedia systems conference, MMSys ’18. ACM, New York, NY, USA, pp 40–51. https://doi.org/10.1145/3204949.3204958

  27. Dou M, Davidson P, Fanello SR, Khamis S, Kowdle A, Rhemann C, Tankovich V, Izadi S (2017) Motion2Fusion: real-time volumetric performance capture. ACM Trans Graph 36(6):246:1–246:16. https://doi.org/10.1145/3130800.3130801

    Article  Google Scholar 

  28. Dou M, Fuchs H, Frahm J (2013) Scanning and tracking dynamic objects with commodity depth cameras. In: 2013 IEEE international symposium on mixed and augmented Reality (ISMAR), pp 99–106. https://doi.org/10.1109/ISMAR.2013.6671769

  29. Dou M, Khamis S, Degtyarev Y, Davidson PL, Fanello SR, Kowdle A, Orts S, Rhemann C, Kim D, Taylor J, Kohli P, Tankovich V, Izadi S (2016) Fusion4d: real-time performance capture of challenging scenes. ACM Trans Graph 35:114:1–114:13. https://www.samehkhamis.com/dou-siggraph2016.pdf

    Article  Google Scholar 

  30. Esser SK, Merolla PA, Arthur JV, Cassidy AS, Appuswamy R, Andreopoulos A, Berg DJ, McKinstry JL, Melano T, Barch DR, di Nolfo C, Datta P, Amir A, Taba B, Flickner MD, Modha DS (2016) Convolutional networks for fast, energy-efficient neuromorphic computing. Proc Natl Acad Sci. https://doi.org/10.1073/pnas.1604850113

    Article  Google Scholar 

  31. Factory 42: “Hold the World” with David Attenborough (2019). https://www.factory42.uk/. Accessed 28 June 2019

  32. Faion F, Friedberger S, Zea A, Hanebeck UD (2012) Intelligent sensor-scheduling for multi-kinect-tracking. In: 2012 IEEE/RSJ international conference on intelligent robots and systems, pp. 3993–3999. https://doi.org/10.1109/IROS.2012.6386007

  33. Fehrman B, McGough J (2014) Depth mapping using a low-cost camera array. In: 2014 Southwest symposium on image analysis and interpretation, pp 101–104. https://doi.org/10.1109/SSIAI.2014.6806039

  34. Fischler MA, Bolles RC (1981) Random sample consensus: a paradigm for model fitting with applications to image analysis and automated cartography. Commun ACM 24(6):381–395. https://doi.org/10.1145/358669.358692

    Article  MathSciNet  Google Scholar 

  35. Fuhrmann A, Kretz J, Burwik P (2013) Multi sensor tracking for live sound transformation. In: Proceedings of the international conference on new interfaces for musical expression, pp 358–362. Graduate School of Culture Technology, KAIST, Daejeon, Republic of Korea. http://nime.org/proceedings/2013/nime2013_44.pdf

  36. Gavrila D, Davis LS (1996) 3-d model-based tracking of humans in action: a multi-view approach. In: CVPR. https://doi.org/10.1109/CVPR.1996.517056

  37. Ge S, Fan G (2015) Articulated non-rigid point set registration for human pose estimation from 3D sensors. pp. 15,218–15,245. MDPI AG. https://doi.org/10.3390/s150715218

    Article  Google Scholar 

  38. Geerse DJ, Coolen B, Roerdink M (2015) Kinematic validation of a multi-Kinect v2 instrumented 10-meter walkway for quantitative gait assessments. PLoS One 10:e0139,913. https://doi.org/10.1371/journal.pone.0139913, https://www.ncbi.nlm.nih.gov/pmc/articles/PMC4603795/. Accessed 01 Feb 2019

    Article  Google Scholar 

  39. Geiselhart F, Otto M, Rukzio E (2016) On the use of multi-depth-camera based motion tracking systems in production planning environments. Procedia CIRP 41:759–764. https://doi.org/10.1016/j.procir.2015.12.088, http://www.sciencedirect.com/science/article/pii/S2212827115011671. Research and Innovation in Manufacturing: Key Enabling Technologies for the Factories of the Future - Proceedings of the 48th CIRP Conference on Manufacturing Systems

    Article  Google Scholar 

  40. Ghose A, Chakravarty K, Agrawal AK, Ahmed N (2013) Unobtrusive indoor surveillance of patients at home using multiple Kinect sensors. In: Proceedings of the 11th ACM conference on embedded networked sensor systems, SenSys ’13. ACM, New York, NY, USA, pp 40:1–40:2. https://doi.org/10.1145/2517351.2517412

  41. Ghose A, Sinha P, Bhaumik C, Sinha A, Agrawal A, Dutta Choudhury A (2013) Ubiheld: ubiquitous healthcare monitoring system for elderly and chronic patients. In: Proceedings of the 2013 ACM conference on pervasive and ubiquitous computing adjunct publication, UbiComp ’13 Adjunct. ACM, New York, NY, USA, pp 1255–1264. https://doi.org/10.1145/2494091.2497331

  42. Gonzalez-Ortega D, Diaz-Pernas F, Martinez-Zarzuela M, Anton-Rodriguez M (2014) A Kinect-based system for cognitive rehabilitation exercises monitoring. Comput Methods Prog Biomed 113(2):620–631. https://doi.org/10.1016/j.cmpb.2013.10.014, http://www.sciencedirect.com/science/article/pii/S0169260713003568

    Article  Google Scholar 

  43. Gotsch D, Zhang X, Merritt T, Vertegaal R (2018) Telehuman2: a cylindrical light field teleconferencing system for life-size 3D human telepresence. In: Proceedings of the 2018 CHI conference on human factors in computing systems, CHI ’18. ACM, New York, NY, USA, pp 522:1–522:10. https://doi.org/10.1145/3173574.3174096

  44. Grunnet-Jepsen A, Winer P, Takagi A, Sweetser J, Zhao K, Khuong T, Nie D, Woodfill J (2019) Using the realsense d4xx depth sensors in multi-camera configurations. White Paper. https://www.intel.ca/content/www/ca/en/support/articles/000028140/emerging-technologies/intel-realsense-technology.html/. Accessed 01 July 2019

  45. Hong S, Kim Y (2018) Dynamic pose estimation using multiple RGB-D cameras. Sensors 18(11). https://doi.org/10.3390/s18113865, http://www.mdpi.com/1424-8220/18/11/3865

    Article  Google Scholar 

  46. Horaud R, Hansard M, Evangelidis G, Ménier C (2016) An overview of depth cameras and range scanners based on time-of-flight technologies. Mach Vis Appl 27(7):1005–1020. https://doi.org/10.1007/s00138-016-0784-4, https://hal.inria.fr/hal-01325045

    Article  Google Scholar 

  47. HTC Corp.: HTC Vive Wireless Adapter (2019) https://www.vive.com/us/wireless-adapter//. Accessed 14 June 2019

  48. Intel Corp.: Intel realsense (2017). https://realsense.intel.com/. Accessed 21 Jan 2019

  49. Intel Corp.: Intel volumetric content studio large (2019) https://newsroom.intel.com/wp-content/uploads/sites/11/2018/01/intel-studios-fact-sheet.pdf. Accessed 21 Jan 2019

  50. Intel Corp.: Intel volumetric content studio small (2019). https://realsense.intel.com/intel-realsense-volumetric-capture/ (2019). Accessed 21 Jan 2019

  51. Izadi S, Kim D, Hilliges O, Molyneaux D, Newcombe R, Kohli P, Shotton J, Hodges S, Freeman D, Davison A, Fitzgibbon A (2011) KinectFusion: real-time 3D reconstruction and interaction using a moving depth camera. In: Proceedings of the 24th annual ACM symposium on user interface software and technology, UIST ’11. ACM, New York, NY, USA, pp 559–568. https://doi.org/10.1145/2047196.2047270

  52. Joachimczak M, Liu J, Ando H (2017) Real-time mixed-reality telepresence via 3D reconstruction with hololens and commodity depth sensors. In: Proceedings of the 19th ACM international conference on multimodal interaction, ICMI 2017. ACM, New York, NY, USA, pp 514–515. https://doi.org/10.1145/3136755.3143031

  53. Jones B, Sodhi R, Murdock M, Mehra R, Benko H, Wilson A, Ofek E, MacIntyre B, Raghuvanshi N, Shapira L (2014) Roomalive: magical experiences enabled by scalable, adaptive projector-camera units. In: Proceedings of the 27th annual ACM symposium on user interface software and technology, UIST ’14. ACM, New York, NY, USA, pp 637–644. https://doi.org/10.1145/2642918.2647383

  54. Joo H, Simon T, Li X, Liu H, Tan L, Gui L, Banerjee S, Godisart T, Nabbe B, Matthews I, Kanade T, Nobuhara S, Sheikh Y (2017) Panoptic studio: a massively multiview system for social interaction capture. IEEE Trans Pattern Anal Mach Intell 41(1):190–204. https://doi.org/10.1109/TPAMI.2017.2782743

    Article  Google Scholar 

  55. Kaenchan S, Mongkolnam P, Watanapa B, Sathienpong S (2013) Automatic multiple Kinect cameras setting for simple walking posture analysis. In: 2013 international computer science and engineering conference (ICSEC), pp 245–249. https://doi.org/10.1109/ICSEC.2013.6694787

  56. Kainz B, Hauswiesner S, Reitmayr G, Steinberger M, Grasset R, Gruber L, Veas E, Kalkofen D, Seichter H, Schmalstieg D (2012) OmniKinect: real-time dense volumetric data acquisition and applications. In: Proceedings of the 18th ACM symposium on virtual reality software and technology, VRST ’12. ACM, New York, NY, USA, pp 25–32. https://doi.org/10.1145/2407336.2407342

  57. Kilner J, Neophytou A, Hilton A (2012) 3D scanning with multiple depth sensors. In: Proceedings of 3rd international conference on 3D body scanning technologies, pp 295–301 (2012). https://doi.org/10.15221/12.295

  58. Kim K, Bolton J, Girouard A, Cooperstock J, Vertegaal R (2012) TeleHuman: Effects of 3D perspective on gaze and pose estimation with a life-size cylindrical telepresence pod. In: Proceedings of the SIGCHI conference on human factors in computing systems, CHI ’12. ACM, New York, NY, USA, pp 2531–2540. https://doi.org/10.1145/2207676.2208640

  59. Kim Y, Baek S, Bae BC (2017) Motion capture of the human body using multiple depth sensors. ETRI J 39(2):181–190. https://doi.org/10.4218/etrij.17.2816.0045

    Article  Google Scholar 

  60. Kim YM, Theobalt C, Diebel J, Kosecka J, Miscusik B, Thrun S (2009) Multi-view image and tof sensor fusion for dense 3D reconstruction. In: 2009 IEEE 12th international conference on computer vision workshops, ICCV workshops, pp 1542–1549. https://doi.org/10.1109/ICCVW.2009.5457430

  61. Kitsikidis A, Dimitropoulos K, Douka S, Grammalidis N (2014) Dance analysis using multiple Kinect sensors. In: 2014 international conference on computer vision theory and applications (VISAPP) vol 2, pp 789–795 (2014). https://ieeexplore.ieee.org/document/7295020

  62. Kolkmeier J, Harmsen E, Giesselink S, Reidsma D, Theune M, Heylen D (2018) With a little help from a holographic friend: The OpenIMPRESS mixed reality telepresence toolkit for remote collaboration systems. In: Proceedings of the 24th ACM symposium on virtual reality software and technology, VRST ’18. ACM, New York, NY, USA, pp 26:1–26:11. https://doi.org/10.1145/3281505.3281542

  63. Kowalski M, Naruniec J, Daniluk M (2015) Livescan3d: a fast and inexpensive 3D data acquisition system for multiple Kinect v2 sensors. In: 2015 international conference on 3D vision, pp 318–325. https://doi.org/10.1109/3DV.2015.43

  64. Kramer J, Burrus N, Echtler F, Daniel HC, Parker M (2012) Object modeling and detection. Apress, Berkeley, CA, pp 173–206. https://doi.org/10.1007/978-1-4302-3868-3_9

    Chapter  Google Scholar 

  65. Kreylos O (2010) Movies - 2 Kinects 1 box (2010). http://idav.ucdavis.edu/~okreylos/ResDev/Kinect/Movies.html. Accessed 22 June 2019

  66. Kurillo G, Bajcsy R (2008) Wide-area external multi-camera calibration using vision graphs and virtual calibration object. In: 2008 Second ACM/IEEE international conference on distributed smart cameras, pp 1–9 (2008). https://doi.org/10.1109/ICDSC.2008.4635695

  67. Leap Motion Inc. (2019) Leap motion technology. https://www.leapmotion.com/technology/. Accessed 14 June 2019

  68. Li H, Liu H, Cao N, Peng Y, Xie S, Luo J, Sun Y (2017) Real-time RGB-D image stitching using multiple Kinects for improved field of view. Int J Adv Robot Syst 14(2):1729881417695,560. https://doi.org/10.1177/1729881417695560

    Article  Google Scholar 

  69. Li S, Pathirana PN, Caelli T (2014) Multi-kinect skeleton fusion for physical rehabilitation monitoring. In: 2014 36th Annual international conference of the IEEE engineering in medicine and biology society, pp 5060–5063. https://doi.org/10.1109/EMBC.2014.6944762

  70. Lin S, Chen Y, Lai YK, Martin RR, Cheng ZQ (2016) Fast capture of textured full-body avatar with RGB-D cameras. Vis Comput 32(6):681–691. https://doi.org/10.1007/s00371-016-1245-9

    Article  Google Scholar 

  71. Liu Y, Ye G, Wang Y, Dai Q, Theobalt C (2014) Human performance capture using multiple Handheld Kinects, pp. 91–108. Springer International Publishing, Cham. https://doi.org/10.1007/978-3-319-08651-4_5

    Google Scholar 

  72. Magic Leap Inc (2019) Introducing Spatiate to Magic Leap One. https://www.magicleap.com/news/product-updates/spatiate-on-magic-leap-one/, https://youtu.be/ePQ5w8oQxWM. Accessed 14 June 2019

  73. Maimone A, Fuchs H (2012) Real-time volumetric 3D capture of room-sized scenes for telepresence. In: 2012 3DTV-conference: the true vision - capture, transmission and display of 3D video (3DTV-CON), pp 1–4. https://doi.org/10.1109/3DTV.2012.6365430

  74. Meng X, Gao W, Hu Z (2018) Dense RGB-D SLAM with multiple cameras. Sensors 18(7) (2018). https://doi.org/10.3390/s18072118, https://www.mdpi.com/1424-8220/18/7/2118

    Article  Google Scholar 

  75. Microsoft Corp (2019) Microsoft hololens - mixed reality technology for business. https://www.microsoft.com/en-us/hololens. Accessed 14 June 2019

  76. Microsoft Corp (2019) Mixed reality capture studios. https://www.microsoft.com/en-us/mixed-reality/capture-studios. Accessed 27 June 2019

  77. Morell-Gimenez V, Saval-Calvo M, Villena Martinez V, Azorin-Lopez J, Rodriguez J, Cazorla M, Orts S, Guilló A (2018) A survey of 3D rigid registration methods for RGB-D cameras, pp 74–98 (2018). https://www.researchgate.net/publication/325194952_A_survey_of_3d_rigid_registration_methods_for_RGB-D_cameras

    Chapter  Google Scholar 

  78. Muybridge E, Wikipedia E (1878) Sallie gardner at a gallop. https://en.wikipedia.org/wiki/Sallie_Gardner_at_a_Gallop. Accessed 21 Jan 2019

  79. Newcombe RA, Davison AJ, Izadi S, Kohli P, Hilliges O, Shotton J, Molyneaux D, Hodges S, Kim D, Fitzgibbon A (2011) KinectFusion: real-time dense surface mapping and tracking. In: 2011 10th IEEE international symposium on mixed and augmented reality (ISMAR). IEEE, pp 127–136. https://doi.org/10.1109/ISMAR.2011.6092378

  80. Ortiz L, Cabrera E, Gonçalves L (2018) Depth data error modeling of the ZED 3D vision sensor from stereolabs. Electron Lett Comput Vis Image Anal 17. https://doi.org/10.5565/rev/elcvia.1084

    Article  Google Scholar 

  81. Orts-Escolano S, Rhemann C, Fanello S, Chang W, Kowdle A, Degtyarev Y, Kim D, Davidson PL, Khamis S, Dou M, Tankovich V, Loop C, Cai Q, Chou PA, Mennicken S, Valentin J, Pradeep V, Wang S, Kang SB, Kohli P, Lutchyn Y, Keskin C, Izadi S (2016) Holoportation: virtual 3D teleportation in real-time. In: Proceedings of the 29th annual symposium on user interface software and technology, UIST ’16. ACM, New York, NY, USA, pp 741–754. https://doi.org/10.1145/2984511.2984517

  82. Palasek P, Yang H, Xu Z, Hajimirza N, Izquierdo E, Patras I (2015) A flexible calibration method of multiple Kinects for 3D human reconstruction. In: 2015 IEEE international conference on multimedia expo workshops (ICMEW), pp 1–4. https://doi.org/10.1109/ICMEW.2015.7169829

  83. Rafighi A, Seifi S, Meruvia-Pastor O (2015) Automatic and adaptable registration of live RGBD video streams. In: Proceedings of the 8th international conference on motion in games. ACM. https://doi.org/10.1145/2984511.2984517

  84. Rander P, Narayanan PJ, Kanade T (1997) Virtualized reality: constructing time-varying virtual worlds from real world events. In: Proceedings. Visualization ’97 (Cat. No. 97CB36155), pp 277–283. https://doi.org/10.1109/VISUAL.1997.663893

  85. Sarbolandi H, Lefloch D, Kolb A (2015) Kinect range sensing: structured-light versus time-of-flight Kinect. Comput Vis Image Underst 139:1–20. https://doi.org/10.1016/j.cviu.2015.05.006, http://www.sciencedirect.com/science/article/pii/S1077314215001071

    Article  Google Scholar 

  86. Satnik, A., Izquierdo, E.: Real-time multi-view volumetric reconstruction of dynamic scenes using Kinect v2. In: 2018 - 3DTV-conference: the true vision - capture, transmission and display of 3D video (3DTV-CON), pp 1–4 (2018). https://doi.org/10.1109/3DTV.2018.8478536

  87. Schröder Y, Scholz A, Berger K, Ruhl K, Guthe S, Magnor M (2011) Multiple Kinect studies. Technical Report - Computer Graphics Lab, TU Braunschweig 2011-09-15. http://www.digibib.tu-bs.de/?docid=00041359

  88. Seer S, Brändle N, Ratti C (2012) Kinects and human kinetics: a new approach for studying crowd behavior. arXiv:1210.28388

  89. Shi Z, Sun Y, Xiong L, Hu Y, Yin B (2015) A multisource heterogeneous data fusion method for pedestrian tracking. Math Prob Eng 150541:1–10. https://doi.org/10.1155/2015/150541

    Google Scholar 

  90. Si L, Wang Q, Xiao Z (2014) Matching cost fusion in dense depth recovery for camera-array via global optimization. In: 2014 international conference on virtual reality and visualization, pp 180–185. https://doi.org/10.1109/ICVRV.2014.67

  91. Silberman S (2003) Matrix2. https://www.wired.com/2003/05/matrix2/. Accessed 25 June 2019

  92. Singh A, Sha J, Narayan KS, Achim T, Abbeel P (2014) BigBIRD: a large-scale 3D database of object instances. In: 2014 IEEE international conference on robotics and automation (ICRA), pp 509–516. https://doi.org/10.1109/ICRA.2014.6906903

  93. Song W, Yun S, Jung SW, Won CS (2016) Rotated top-bottom dual-Kinect for improved field of view. Multimed Tools Appl 75(14):8569–8593. https://doi.org/10.1007/s11042-015-2772-5

    Article  Google Scholar 

  94. Steinbruecker F, Sturm J, Cremers D (2011) Real-time visual odometry from dense RGB-D images. In: Workshop on live dense reconstruction with moving cameras at the international conference on computer vision (ICCV). https://vision.in.tum.de/data/software/dvo

  95. Stereolabs Inc (2019) ZED camera and SDK overview. https://www.stereolabs.com/zed/docs/ZED_Datasheet_2016.pdf. Accessed 21 Jan 2019

  96. Sterzentsenko V, Karakottas A, Papachristou A, Zioulis N, Doumanoglou A, Zarpalas D, Daras P (2018) A low-cost, flexible and portable volumetric capturing system. In: 2018 14th international conference on signal-image technology internet-based systems (SITIS), pp 200–207 (2018). https://doi.org/10.1109/SITIS.2018.00038

  97. Svoboda T, Martinec D, Pajdla T (2005) A convenient multicamera self-calibration for virtual environments. Presence 14(4):407–422. https://doi.org/10.1162/105474605774785325, http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.83.9884&rep=rep1&type=pdf

    Article  Google Scholar 

  98. Tam GKL, Cheng ZQ, Lai YK, Langbein FC, Liu Y, Marshall AD, Martin RR, Sun X, Rosin PL (2013) Registration of 3D point clouds and meshes: a survey from rigid to nonrigid. IEEE Trans Vis Comput Graph 19:1199–1217. https://doi.org/10.1109/TVCG.2012.310

    Article  Google Scholar 

  99. Taylor D (1996) Virtual camera movement: the way of the future? Am Cinematogr 77(9):93–100 (1996). https://www.digitalair.com/pdfs/Virtual_Camera_Movement_1996.pdf

  100. Toldo R, Beinat A, Crosilla F (2010) Global registration of multiple point clouds embedding the generalized procrustes analysis into an ICP framework (2010). https://www.researchgate.net/publication/228959196_Global_registration_of_multiple_point_clouds_embedding_the_Generalized_Procrustes_Analysis_into_an_ICP_framework

  101. Tong J, Zhou J, Liu L, Pan Z, Yan H (2012) Scanning 3D full human bodies using Kinects. IEEE Trans Vis Comput Graph 18(4):643–650. https://doi.org/10.1109/TVCG.2012.56

    Article  Google Scholar 

  102. Walas K, Nowicki M, Ferstl D, Skrzypczynski P (2016) Depth data fusion for simultaneous localization and mapping – RGB-DD SLAM. In: 2016 IEEE international conference on multisensor fusion and integration for intelligent systems (MFI), pp 9–14. https://doi.org/10.1109/MFI.2016.7849459

  103. Wang D, Pan Q, Zhao C, Hu J, Xu Z, Yang F, Zhou Y (2017) A study on camera array and its applications. IFAC-PapersOnLine 50(1), 10,323–10,328. https://doi.org/10.1016/j.ifacol.2017.08.1662. 20th IFAC World Congress

    Article  Google Scholar 

  104. Wen C, Qin L, Zhu Q, Wang C, Li J (2014) Three-dimensional indoor mobile mapping with fusion of two-dimensional laser scanner and rgb-d camera data. IEEE Geosci Remote Sens Lett 11(4):843–847. https://doi.org/10.1109/LGRS.2013.2279872

    Article  Google Scholar 

  105. Wikipedia (2019) Multiple-camera setup. https://en.wikipedia.org/wiki/Multiple-camera_setup. Accessed 25 June 2019

  106. Wilburn B, Joshi N, Vaish V, Talvala EV, Antunez E, Barth A, Adams A, Horowitz M, Levoy M (2005) High performance imaging using large camera arrays. ACM Trans Graph 24(3):765–776. https://doi.org/10.1145/1073204.1073259

    Article  Google Scholar 

  107. Wilson AD, Benko H (2010) Combining multiple depth cameras and projectors for interactions on, above and between surfaces. In: Proceedings of the 23rd annual ACM symposium on user interface software and technology, UIST ’10. ACM, New York, NY, USA, pp 273–282. https://doi.org/10.1145/1866029.1866073

  108. Wu CJ, Quigley A, Harris-Birtill D (2017) Out of sight: A toolkit for tracking occluded human joint positions. Pers Ubiquitous Comput 21(1):125–135. https://doi.org/10.1007/s00779-016-0997-6

    Article  Google Scholar 

  109. Wu, X., Yu, C., Shi, Y.: Multi-depth-camera sensing and interaction in smart space. In: 2018 IEEE smartworld, ubiquitous intelligence computing, Advanced trusted computing, scalable computing communications, Cloud big data computing, Internet of people and smart city innovation (SmartWorld/SCALCOM/UIC/ATC/CBDCom/IOP/SCI), pp 718–725. https://doi.org/10.1109/SmartWorld.2018.00139

  110. Xiang S, Yu L, Yang Y, Liu Q, Zhou J (2015) Interfered depth map recovery with texture guidance for multiple structured light depth cameras. Image Commun 31(C):34–46. https://doi.org/10.1016/j.image.2014.11.004

    Google Scholar 

  111. Yang S, Yi X, Wang Z, Wang Y, Yang X (2015) Visual SLAM using multiple RGB-D cameras. In: 2015 IEEE International conference on robotics and biomimetics (ROBIO), pp 1389–1395. https://doi.org/10.1109/ROBIO.2015.7418965

  112. Ye G, Liu Y, Deng Y, Hasler N, Ji X, Dai Q, Theobalt C (2013) Free-viewpoint video of human actors using multiple handheld Kinects. IEEE Trans Cybern 43(5):1370–1382. https://doi.org/10.1109/TCYB.2013.2272321

    Article  Google Scholar 

  113. Zhang L, Sturm J, Cremers D, Lee D (2012) Real-time human motion tracking using multiple depth cameras. 2012 IEEE/RSJ international conference on intelligent robots and systems, pp 2389–2395. https://doi.org/10.1109/IROS.2012.6385968

  114. Zhou QY, Koltun V (2014) Color map optimization for 3D reconstruction with consumer depth cameras. ACM Trans Graph 33(4):155:1–155:10. https://doi.org/10.1145/2601097.2601134, http://vladlen.info/papers/color-mapping.pdf

    Google Scholar 

  115. Zollhöfer M, Stotko P, Görlitz A, Theobalt C, Nießner M, Klein R, Kolb A (2018) State of the art on 3D reconstruction with RGB-D cameras. Comput Graph Forum (Eurographics State of the Art Reports (2018) 37(2). https://doi.org/10.1111/cgf.13386, https://web.stanford.edu/~zollhoef/papers/EG18_RecoSTAR/paper.pdf

    Article  Google Scholar 

Download references

Acknowledgements

The author would like to thank the anonymous reviewers for their kind revisions to improve the manuscript, Dr. Lourdes Peña-Castillo (Computer Science, Memorial University of Newfoundland) for her thorough proof reading of the manuscript, and the many authors who approved of the use of their images, including Dr. Kai Berger (Magic Leap Inc.) for contributing Fig. 7.5 and for his insightful comments on the draft.

Author information

Authors and Affiliations

  1. Department of Computer Science, Memorial University of Newfoundland, St. John’s, NL, A1B 3X7, Canada

    Oscar Meruvia-Pastor

Authors
  1. Oscar Meruvia-Pastor
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Oscar Meruvia-Pastor .

Editor information

Editors and Affiliations

  1. School of Computer Science and Informatics, Cardiff University, Cardiff, UK

    Paul L. Rosin

  2. School of Computer Science and Informatics, Cardiff University, Cardiff, UK

    Yu-Kun Lai

  3. IEEE, University of East Anglia, Norwich, UK

    Ling Shao

  4. Department of Computer Science, Edge Hill University, Ormskirk, UK

    Yonghuai Liu

Rights and permissions

Reprints and permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this chapter

Check for updates. Verify currency and authenticity via CrossMark

Cite this chapter

Meruvia-Pastor, O. (2019). Enhancing 3D Capture with Multiple Depth Camera Systems: A State-of-the-Art Report. In: Rosin, P., Lai, YK., Shao, L., Liu, Y. (eds) RGB-D Image Analysis and Processing. Advances in Computer Vision and Pattern Recognition. Springer, Cham. https://doi.org/10.1007/978-3-030-28603-3_7

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-28603-3_7

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-28602-6

  • Online ISBN: 978-3-030-28603-3

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation