Skip to main content
SpringerLink
Log in
Book cover

Asian Conference on Computer Vision

ACCV 2014: Computer Vision - ACCV 2014 Workshops pp 486–501Cite as

Collaborative Mobile 3D Reconstruction of Urban Scenes

  • Conference paper
  • First Online:

Part of the book series: Lecture Notes in Computer Science ((LNIP,volume 9010))

Abstract

Reconstruction of the surrounding 3D world is of particular interest either for mapping, civil applications or for entertainment. The wide availability of smartphones with cameras and wireless networking capabilities makes collecting 2D images of a particular scene easy. In contrast to the client-server architecture adopted by most mobile services, we propose an architecture where data, computations and results can be shared in a collaborative manner among the participating devices without centralization. Camera calibration and pose estimation parameters are determined using classical image-based methods. The reconstruction is based on interactively selected arbitrary planar regions which is especially suitable for objects having large (near) planar surfaces often found in urban scenes (e.g. building facades, windows, etc). The perspective distortion of a planar region in two views makes it possible to compute the normal and distance of the region w.r.t the world coordinate system. Thus a fairly precise 3D model can be built by reconstructing a set of planar regions with different orientation. We also show how visualization, data sharing and communication can be solved. The applicability of the method is demonstrated on reconstructing real urban scenes.

This research was supported by the European Union and the European Social Fund through project FuturICT.hu (grant no.: TÁMOP-4.2.2.C-11/1/KONV-2012-0013). The authors thank the work of Zoltán Molnár, Péter Rácz and Sándor Laczik in the Android implementation.

This is a preview of subscription content, log in via an institution.

Chapter
USD   29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD   39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD   54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Learn about institutional subscriptions

Notes

  1. 1.

    The free scale parameter of the separate reconstructions can be computed as the ratio of the distance reconstruction parameter of the same patch.

References

  1. Agarwal, S., Snavely, N., Simon, I., Seitz, S.M., Szeliski, R.: Building Rome in a day. In: Proceedings of IEEE International Conference on Computer Vision, pp. 72–79 (2009)

    Google Scholar 

  2. Frahm, J.-M., Fite-Georgel, P., Gallup, D., Johnson, T., Raguram, R., Wu, C., Jen, Y.-H., Dunn, E., Clipp, B., Lazebnik, S., Pollefeys, M.: Building rome on a cloudless day. In: Daniilidis, K., Maragos, P., Paragios, N. (eds.) ECCV 2010, Part IV. LNCS, vol. 6314, pp. 368–381. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  3. Shum, H.Y., Han, M., Szeliski, R.: Interactive construction of 3D models from panoramic mosaics. In: Proceedings of IEEE Computer Vision and Pattern Recognition, pp. 427–433 (1998)

    Google Scholar 

  4. Wei, C.-H., Chiang, C.-K., Sun, Y.-W., Lin, M.-H., Lai, S.-H.: Novel multi-view synthesis from a stereo image pair for 3d display on mobile phone. In: Park, J.-I., Kim, J. (eds.) ACCV Workshops 2012, Part II. LNCS, vol. 7729, pp. 568–579. Springer, Heidelberg (2013)

    Chapter  Google Scholar 

  5. Chen, S.C., Hsu, C.W., Huang, D.Y., Lin, S.Y., Hung, Y.P.: TelePort: Virtual touring of Dun-Huang with a mobile device. In: Proceedings of IEEE International Conference on Multimedia and Expo Workshops, pp. 1–6 (2013)

    Google Scholar 

  6. Google: Project Tango (2014). https://www.google.com/atap/projecttango/#project. Accessed 10 June 2014

  7. Werner, T., Zisserman, A.: New techniques for automated architecture reconstruction from photographs. In: Proceedings of European Conference on Computer Vision, vol. 2, pp. 541–555 (2002)

    Google Scholar 

  8. Furukawa, Y., Curless, B., Seitz, S., Szeliski, R.: Manhattan-world stereo. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition, pp. 1422–1429 (2009)

    Google Scholar 

  9. He, Q., Chu, C.H.: Planar surface detection in image pairs using homographic constraints. In: Bebis, G., Boyle, R., Parvin, B., Koracin, D., Remagnino, P., Nefian, A., Meenakshisundaram, G., Pascucci, V., Zara, J., Molineros, J., Theisel, H., Malzbender, T. (eds.) ISVC 2006. LNCS, vol. 4291, pp. 19–27. Springer, Heidelberg (2006)

    Chapter  Google Scholar 

  10. Molnár, J., Huang, R., Kato, Z.: 3D reconstruction of planar surface patches: a direct solution. In: Jawahar, C.V., Shan, S. (eds.) ACCV 2014 Workshops, Part I. LNCS, vol. 9008, pp. 286–300. Springer, Heidelberg (2014)

    Google Scholar 

  11. Lowe, D.: Distinctive image features from scale-invariant keypoints. Int. J. Comput. Vis. 60, 91–110 (2004)

    Article  Google Scholar 

  12. Bay, H., Ess, A., Tuytelaars, T., Gool, L.: SURF: Speeded up robust features. Comput. Vis. Image Underst. 110, 346–359 (2008)

    Article  Google Scholar 

  13. Mikolajczyk, K., Schmid, C.: A performance evaluation of local descriptors. IEEE Trans. Pattern Anal. Mach. Intell. 27, 1615–1630 (2004)

    Article  Google Scholar 

  14. Miksik, O., Mikolajczyk, K.: Evaluation of local detectors and descriptors for fast feature matching. In: Proceedings of the International Conference on Pattern Recognition, pp. 2681–2684 (2012)

    Google Scholar 

  15. Juhász, E., Tanács, A., Kato, Z.: Evaluation of point matching methods for wide-baseline stereo correspondence on mobile platforms. In: Proceedings of the International Symposium on Image and Signal Processing and Analysis, pp. 806–811. IEEE, Trieste (2013)

    Google Scholar 

  16. Torii, A., Sivic, J., Pajdla, T., Okutomi, M.: Visual place recognition with repetitive structures. In: Proceedings of IEEE Conference on Computer Vision and Pattern Recognition, pp. 883–890. IEEE (2013)

    Google Scholar 

  17. Tanács, A., Majdik, A., Molnár, J., Rai, A., Kato, Z.: Establishing correspondences between planar image patches. In: Proceedings of the International Conference on Digital Image Computing: Techniques and Applications, Wollongong, Australia (2014)

    Google Scholar 

  18. Kovács, L.: Processing geotagged image sets for collaborative compositing and view reconstruction. In: Proceedings of ICCV Workshop on Computer Vision for Converging Perspectives, Sydney, Australia, pp. 460–467 (2013)

    Google Scholar 

  19. Bouguet, J.: Camera Calibration Toolbox for Matlab (2013). http://www.vision.caltech.edu/bouguetj/calib_doc/. Accessed 10 June 2014

  20. Morel, J., Yu, G.: ASIFT: A new framework for fully affine invariant image comparison. SIAM J. Imaging Sci. 2, 438–469 (2009)

    Article  MATH  MathSciNet  Google Scholar 

  21. Hartley, R., Zisserman, A.: Multiple View Geometry in Computer Vision. University Press, Cambridge (2003)

    Google Scholar 

  22. Lourakis, M.A., Argyros, A.: SBA: A software package for generic sparse bundle adjustment. ACM Trans. Math. Softw. 36, 1–30 (2009)

    Article  MathSciNet  Google Scholar 

  23. Shi, J., Malik, J.: Normalized cuts and image segmentation. IEEE Trans. Pattern Anal. Mach. Intell. 22, 888–905 (2000)

    Article  Google Scholar 

  24. Matas, J., Chum, O., Urban, M., Pajdla, T.: Robust wide baseline stereo from maximally stable extremal regions. In: Proceedings of British Machine Vision Conference, pp. 384–396 (2002)

    Google Scholar 

  25. Comaniciu, D., Meer, P., Member, S.: Mean shift: A robust approach toward feature space analysis. IEEE Trans. Pattern Anal. Mach. Intell. 24, 603–619 (2002)

    Article  Google Scholar 

  26. Hartley, R.I.: Theory and practice of projective rectification. Int. J. Comput. Vis. 35, 115–127 (1999)

    Article  Google Scholar 

  27. Cahill, N.D.: Normalized measures of mutual information with general definitions of entropy for multimodal image registration. In: Fischer, B., Dawant, B.M., Lorenz, C. (eds.) WBIR 2010. LNCS, vol. 6204, pp. 258–268. Springer, Heidelberg (2010)

    Chapter  Google Scholar 

  28. Press, W., Teukolsky, S., Vetterling, W., Flannery, B.: Numerical Recipes in C: The Art of Scientific Computing, 2nd edn. Cambridge University Press, New York (1992)

    Google Scholar 

  29. Burt, P., Adelson, E.: The Laplacian pyramid as a compact image code. IEEE Trans. Commun. 31, 532–540 (1983)

    Article  Google Scholar 

  30. Faugeras, O., Lustman, F.: Motion and structure from motion in a piecewise planar environment. Technical Report RR-0856, INRIA (1988)

    Google Scholar 

  31. Zhang, Z., Hanson, A.R.: 3D reconstruction based on homography mapping. In: Proceedings of ARPA Image Understanding Workshop, pp. 0249–6399 (1996)

    Google Scholar 

  32. de Berg, M., van Kreveld, M., Overmars, M., Schwarzkopf, O.: Computational Geometry: Algorithms and Applications, 2nd edn. Springer, Heidelberg (2000)

    Book  MATH  Google Scholar 

  33. Reinhard, E., Ashikhmin, M., Gooch, B., Shirley, P.: Color transfer between images. IEEE Comput. Graph. Appl. 21, 34–41 (2001)

    Article  Google Scholar 

  34. Kang, H., Efros, A.A., Hebert, M., Kanade, T.: Image matching in large scale indoor environment. In: Proceedings of IEEE Computer Society Conference on Computer Vision and Pattern Recognition Workshops, pp. 33–40 (2009)

    Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Szeged, Árpád tér 2, Szeged, 6720, Hungary

    Attila Tanács, András Majdik, Levente Hajder, József Molnár, Zsolt Sánta & Zoltan Kato

  2. Institute for Computer Science and Control (MTA SZTAKI), Kende u. 13–17, Budapest, 1111, Hungary

    Levente Hajder

Authors
  1. Attila Tanács
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. András Majdik
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Levente Hajder
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. József Molnár
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Zsolt Sánta
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Zoltan Kato
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Attila Tanács .

Editor information

Editors and Affiliations

  1. Center for Visual Information Technology, International Institute of Information Technology, Hyderabad, India

    C. V. Jawahar

  2. Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China

    Shiguang Shan

Rights and permissions

Reprints and permissions

Copyright information

© 2015 Springer International Publishing Switzerland

About this paper

Cite this paper

Tanács, A., Majdik, A., Hajder, L., Molnár, J., Sánta, Z., Kato, Z. (2015). Collaborative Mobile 3D Reconstruction of Urban Scenes. In: Jawahar, C., Shan, S. (eds) Computer Vision - ACCV 2014 Workshops. ACCV 2014. Lecture Notes in Computer Science(), vol 9010. Springer, Cham. https://doi.org/10.1007/978-3-319-16634-6_36

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-16634-6_36

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-16633-9

  • Online ISBN: 978-3-319-16634-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation