Mobile Mapping and Visualization of Indoor Structures to Simplify Scene Understanding and Location Awareness

  • Giovanni Pintore
  • Fabio Ganovelli
  • Enrico Gobbetti
  • Roberto Scopigno
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9914)

Abstract

We present a technology to capture, reconstruct and explore multi-room indoor structures, starting from panorama images generated with the aid of commodity mobile devices. Our approach is motivated by the need for fast and effective systems to simplify indoor data acquisition, as required in many real-world cases where mapping the structure is more important than capturing 3D details, such as the design of smart houses or in the security domain. We combine and extend state-of-the-art results to obtain indoor models scaled to their real-world metric dimension, making them available for online exploration. Moreover, since our target is to assist end-users not necessarily skilled in virtual reality and 3D objects interaction, we introduce a client-server image-based navigation system, exploiting this simplified indoor structure to support a low-degree-of-freedom user interface. We tested our approach in several indoor environments and carried out a preliminary user study to assess the usability of the system by people without a specific technical background.

Keywords

Mobile systems Scene understanding Scene reconstruction Smart environments Safety and security 

Notes

Acknowledgements

This work has received funding from the European Union’s Seventh Framework Programme for research, technological development and demonstration under grant agreement no 607737 (VASCO). We also acknowledge the contribution of Sardinian Regional Authorities under projects VIGEC and Vis&VideoLab.

References

  1. 1.
    Ariani, A., Redmond, S.J., Chang, D., Lovell, N.H.: Simulation of a smart home environment. In: 2013 3rd International Conference on Instrumentation, Communications, Information Technology, and Biomedical Engineering (ICICI-BME), pp. 27–32, November 2013Google Scholar
  2. 2.
    Bermudez-Cameo, J., Puig, L., Guerrero, J.: Hypercatadioptric line images for 3d orientation and image rectification. Robot. Auton. Syst. 60(6), 755–768 (2012)CrossRefGoogle Scholar
  3. 3.
    Brivio, P., Benedetti, L., Tarini, M., Ponchio, F., Cignoni, P., Scopigno, R.: Photocloud: interactive remote exploration of large 2d–3d datasets. IEEE Comput. Graphics Appl. 33(2), 86–96 (2013)CrossRefGoogle Scholar
  4. 4.
    Cabral, R., Furukawa, Y.: Piecewise planar and compact floorplan reconstruction from images. In: Proceedings of the CVPR, pp. 628–635 (2014)Google Scholar
  5. 5.
    Clarke, D., Duimering, P.R.: How computer gamers experience the game situation: A behavioral study. Comput. Entertain. 4(3) (2006)Google Scholar
  6. 6.
    Coughlan, J.M., Yuille, A.L.: Manhattan world: Compass direction from a single image by bayesian inference. In: Proceedings of the ICCV, vol. 2, pp. 941–947 (1999)Google Scholar
  7. 7.
    Crisis Plan, B.V.: Crisis management experts. http://www.crisisplan.nl/
  8. 8.
    Dev, K., Lau, M.: Democratizing digital content creation using mobile devices with inbuilt sensors. Comput. Graph. Appl. 35(1), 84–94 (2015)CrossRefGoogle Scholar
  9. 9.
    Di Benedetto, M., Ganovelli, F., Balsa Rodriguez, M., Jaspe Villanueva, A., Scopigno, R., Gobbetti, E.: Exploremaps: Efficient construction and ubiquitous exploration of panoramic view graphs of complex 3d environments. Comput. Graph. Forum 33(2) (2014). proc. Eurographics 2014Google Scholar
  10. 10.
    Flint, A., Mei, C., Murray, D., Reid, I.: A dynamic programming approach to reconstructing building interiors. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part V. LNCS, vol. 6315, pp. 394–407. Springer, Heidelberg (2010)CrossRefGoogle Scholar
  11. 11.
    Furukawa, Y., Curless, B., Seitz, S.M., Szeliski, R.: Reconstructing building interiors from images. In: Proceedings of the ICCV (2009)Google Scholar
  12. 12.
    Geyer, C., Daniilidis, K.: A unifying theory for central panoramic systems and practical implications. In: Vernon, D. (ed.) ECCV 2000. LNCS, vol. 1843, pp. 445–461. Springer, Heidelberg (2000)CrossRefGoogle Scholar
  13. 13.
    Goesele, M., Ackermann, J., Fuhrmann, S., Haubold, C., Klowsky, R., Steedly, D., Szeliski, R.: Ambient point clouds for view interpolation. ACM Trans. Graph. 29(4), 95:1–95:6 (2010)CrossRefGoogle Scholar
  14. 14.
    Goetz, M., Zipf, A.: Open issues in bringing 3d to location based services (lbs): a review focusing on 3d data streaming and 3d indoor navigation. Remote Sensing and Spatial Information Sciences, pp. 38–4 (2010)Google Scholar
  15. 15.
    Guest, J., Eaglin, T., Subramanian, K., Ribarsky, W.: Interactive analysis and visualization of situationally aware building evacuations. Information Visualization (2014)Google Scholar
  16. 16.
    Helal, S., Mann, W., El-Zabadani, H., King, J., Kaddoura, Y., Jansen, E.: The gator tech smart house: a programmable pervasive space. Computer 38(3), 50–60 (2005)CrossRefGoogle Scholar
  17. 17.
    Hoiem, D., Efros, A.A., Hebert, M.: Geometric context from a single image. In: Proceedings of the ICCV, vol. 1, pp. 654–661. IEEE (2005)Google Scholar
  18. 18.
    Jahromi, Z.F., Rajabzadeh, A., Manashty, A.R.: A multi-purpose scenario-based simulator for smart house environments. (IJCSIS) Int. J. Comput. Sci. Inf. Secur. 9(1), 13–19 (2011). http://dx.doi.org/10.1007/s00779-014-0813-0
  19. 19.
    Mura, C., Mattausch, O., Gobbetti, A.J.E., Pajarola, R.: Automatic room detection and reconstruction in cluttered indoor environments with complex room layouts. Comput. Graph. 44, 20–32 (2014)Google Scholar
  20. 20.
    Nguyen, T.V., Kim, J.G., Choi, D.: Iss: The interactive smart home simulator. In: 11th International Conference on Advanced Communication Technology, 2009. ICACT 2009, vol. 03, pp. 1828–1833, February 2009Google Scholar
  21. 21.
    O’Neill, E., Klepal, M., Lewis, D., O’Donnell, T., O’Sullivan, D., Pesch, D.: A testbed for evaluating human interaction with ubiquitous computing environments. In: First International Conference on Testbeds and Research Infrastructures for the Development of Networks and COmmunities. pp. 60–69, February 2005Google Scholar
  22. 22.
    Pintore, G., Garro, V., Ganovelli, F., Agus, M., Gobbetti, E.: Omnidirectional image capture on mobile devices for fast automatic generation of 2.5D indoor maps. In: Proceedings of the IEEE Winter Conference on Applications of Computer Vision (WACV), February 2016Google Scholar
  23. 23.
    Pintore, G., Gobbetti, E.: Effective mobile mapping of multi-room indoor structures. Vis. Comput. 30, 707–716 (2014). proc. CGI 2014CrossRefGoogle Scholar
  24. 24.
    Pintore, G., Gobbetti, E., Ganovelli, F., Brivio, P.: 3dnsite: A networked interactive 3d visualization system to simplify location recognition in crisis management. In: Proceedings of the ACM Web3D International Symposium, pp. 59–67. ACM, New York, August 2012Google Scholar
  25. 25.
    Sankar, A., Seitz, S.: Capturing indoor scenes with smartphones. In: Proceedings of the UIST, pp. 403–412. ACM, New York (2012)Google Scholar
  26. 26.
    Seitz, S.M., Curless, B., Diebel, J., Scharstein, D., Szeliski, R.: A comparison and evaluation of multi-view stereo reconstruction algorithms. In: Proceedings of the CVPR, vol. 1, pp. 519–528 (2006)Google Scholar
  27. 27.
    Snavely, N., Seitz, S.M., Szeliski, R.: Photo tourism: exploring photo collections in \(3\)D. ACM Trans. Graph. 25(3), 835–846 (2006)CrossRefGoogle Scholar
  28. 28.
    Wilson, C., Hargreaves, T., Hauxwell-Baldwin, R.: Smart homes and their users: A systematic analysis and key challenges. Pers. Ubiquit. Comput. 19(2), 463–476 (2015). http://dx.doi.org/10.1007/s00779-014-0813-0
  29. 29.
    Xiong, X., Adan, A., Akinci, B., Huber, D.: Automatic creation of semantically rich 3D building models from laser scanner data. Autom. Constr. 31, 325–337 (2013)CrossRefGoogle Scholar

Copyright information

© Springer International Publishing Switzerland 2016

Authors and Affiliations

  • Giovanni Pintore
    • 1
  • Fabio Ganovelli
    • 2
  • Enrico Gobbetti
    • 1
  • Roberto Scopigno
    • 2
  1. 1.Visual Computing, CRS4PulaItaly
  2. 2.Visual Computing Group, ISTI-CNRPisaItaly

Personalised recommendations