Skip to main content
SpringerLink
Log in

An Optimized Pipeline for Image-Based Localization in Museums from Egocentric Images

  • Conference paper
  • First Online:
Image Analysis and Processing – ICIAP 2023 (ICIAP 2023)

Part of the book series: Lecture Notes in Computer Science ((LNCS,volume 14233))

Included in the following conference series:

  • 510 Accesses

Abstract

With the increasing interest in augmented and virtual reality, visual localization is acquiring a key role in many downstream applications requiring a real-time estimate of the user location only from visual streams. In this paper, we propose an optimized hierarchical localization pipeline by specifically tackling cultural heritage sites with specific applications in museums. Specifically, we propose to enhance the Structure from Motion (SfM) pipeline for constructing the sparse 3D point cloud by a-priori filtering blurred and near-duplicated images. We also study an improved inference pipeline that merges similarity-based localization with geometric pose estimation to effectively mitigate the effect of strong outliers. We show that the proposed optimized pipeline obtains the lowest localization error on the challenging Bellomo dataset [11]. Our proposed approach keeps both build and inference times bounded, in turn enabling the deployment of this pipeline in real-world scenarios.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 69.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 89.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://github.com/colmap/pycolmap/blob/master/estimators/absolute_pose.cc.

References

  1. Arandjelovic, R., Gronat, P., Torii, A., Pajdla, T., Sivic, J.: NetVLAD: CNN architecture for weakly supervised place recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5297–5307 (2016)

    Google Scholar 

  2. Arrighi, G., See, Z.S., Jones, D.: Victoria theatre virtual reality: a digital heritage case study and user experience design. Digit. Appl. Archaeol. Cult. Herit. 21, e00176 (2021)

    Google Scholar 

  3. Balntas, V., Li, S., Prisacariu, V.: RelocNet: continuous metric learning relocalisation using neural nets. In: Ferrari, V., Hebert, M., Sminchisescu, C., Weiss, Y. (eds.) Computer Vision – ECCV 2018. LNCS, vol. 11218, pp. 782–799. Springer, Cham (2018). https://doi.org/10.1007/978-3-030-01264-9_46

    Chapter  Google Scholar 

  4. Cao, S., Snavely, N.: Graph-based discriminative learning for location recognition. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 700–707 (2013)

    Google Scholar 

  5. DeTone, D., Malisiewicz, T., Rabinovich, A.: Superpoint: self-supervised interest point detection and description. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition Workshops, pp. 224–236 (2018)

    Google Scholar 

  6. Durrant-Whyte, H., Bailey, T.: Simultaneous localization and mapping: part i. IEEE Robot. Autom. Mag. 13(2), 99–110 (2006)

    Article  Google Scholar 

  7. Furnari, A., Farinella, G.M., Battiato, S.: Recognizing personal locations from egocentric videos. IEEE Trans. Hum.-Mach. Syst. 47(1), 6–18 (2016)

    Google Scholar 

  8. Ishihara, T., Vongkulbhisal, J., Kitani, K.M., Asakawa, C.: Beacon-guided structure from motion for smartphone-based navigation. In: 2017 IEEE Winter Conference on Applications of Computer Vision (WACV), pp. 769–777. IEEE (2017)

    Google Scholar 

  9. Kendall, A., Grimes, M., Cipolla, R.: PoseNet: a convolutional network for real-time 6-DoF camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2938–2946 (2015)

    Google Scholar 

  10. Laskar, Z., Melekhov, I., Kalia, S., Kannala, J.: Camera relocalization by computing pairwise relative poses using convolutional neural network. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 929–938 (2017)

    Google Scholar 

  11. Mauro, D.D., Furnari, A., Signorello, G., Farinella, G.M.: Unsupervised domain adaptation for 6DoF indoor localization. In: International Conference on Computer Vision Theory and Applications - VISAPP (2021). https://iplab.dmi.unict.it/EGO-CH-LOC-UDA/

  12. Milosz, M., Skulimowski, S., Kȩsik, J., Montusiewicz, J.: Virtual and interactive museum of archaeological artefacts from Afrasiyab - an ancient city on the silk road. Digit. Appl. Archaeol. Cult. Herit. 18, e00155 (2020)

    Google Scholar 

  13. Ragusa, F., Furnari, A., Battiato, S., Signorello, G., Farinella, G.M.: Egocentric visitors localization in cultural sites. J. Comput. Cult. Herit. (JOCCH) 12(2), 11 (2019)

    Google Scholar 

  14. Ragusa, F., Furnari, A., Battiato, S., Signorello, G., Farinella, G.M.: Ego-CH: dataset and fundamental tasks for visitors behavioral understanding using egocentric vision. Pattern Recogn. Lett. 131, 150–157 (2020)

    Article  Google Scholar 

  15. Sarlin, P.E., Cadena, C., Siegwart, R., Dymczyk, M.: From coarse to fine: robust hierarchical localization at large scale. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 12716–12725 (2019)

    Google Scholar 

  16. Sarlin, P.E., Debraine, F., Dymczyk, M., Siegwart, R., Cadena, C.: Leveraging deep visual descriptors for hierarchical efficient localization. In: Conference on Robot Learning, pp. 456–465. PMLR (2018)

    Google Scholar 

  17. Sarlin, P.E., DeTone, D., Malisiewicz, T., Rabinovich, A.: Superglue: learning feature matching with graph neural networks. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 4938–4947 (2020)

    Google Scholar 

  18. Sarlin, P.E., et al.: Back to the feature: learning robust camera localization from pixels to pose. In: Proceedings of the IEEE/CVF Conference on Computer Vision and Pattern Recognition, pp. 3247–3257 (2021)

    Google Scholar 

  19. Schönberger, J.L., Price, T., Sattler, T., Frahm, J.-M., Pollefeys, M.: A vote-and-verify strategy for fast spatial verification in image retrieval. In: Lai, S.-H., Lepetit, V., Nishino, K., Sato, Y. (eds.) ACCV 2016. LNCS, vol. 10111, pp. 321–337. Springer, Cham (2017). https://doi.org/10.1007/978-3-319-54181-5_21

    Chapter  Google Scholar 

  20. Schönberger, J.L., Zheng, E., Frahm, J.-M., Pollefeys, M.: Pixelwise view selection for unstructured multi-view stereo. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9907, pp. 501–518. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46487-9_31

    Chapter  Google Scholar 

  21. Starner, T., Schiele, B., Pentland, A.: Visual contextual awareness in wearable computing. In: Digest of Papers, Second International Symposium on Wearable Computers (Cat. No. 98EX215), pp. 50–57. IEEE (1998)

    Google Scholar 

  22. Walch, F., Hazirbas, C., Leal-Taixe, L., Sattler, T., Hilsenbeck, S., Cremers, D.: Image-based localization using LSTMs for structured feature correlation. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 627–637 (2017)

    Google Scholar 

  23. Weyand, T., Kostrikov, I., Philbin, J.: PlaNet - photo geolocation with convolutional neural networks. In: Leibe, B., Matas, J., Sebe, N., Welling, M. (eds.) ECCV 2016. LNCS, vol. 9912, pp. 37–55. Springer, Cham (2016). https://doi.org/10.1007/978-3-319-46484-8_3

    Chapter  Google Scholar 

Download references

Acknowledgements

This research was supported by AI4Media – A European Excellence Centre for Media, Society, and Democracy (EC, H2020 No. 951911), by SUN – Social and hUman ceNtered XR (EC, Horizon Europe No. 101092612), and by VALUE – Visual Analysis For Location And Understanding Of Environments (PO FESR 2014/2020 program, action 1.1.5).

Author information

Authors and Affiliations

  1. ISTI-CNR, via G. Moruzzi 1, 56017, Pisa, Italy

    Nicola Messina, Fabrizio Falchi & Claudio Gennaro

  2. University of Catania, Viale A. Doria 6, 95125, Catania, Italy

    Antonino Furnari & Giovanni Maria Farinella

Authors
  1. Nicola Messina
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Fabrizio Falchi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Antonino Furnari
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Claudio Gennaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Giovanni Maria Farinella
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicola Messina .

Editor information

Editors and Affiliations

  1. University of Udine, Udine, Italy

    Gian Luca Foresti

  2. University of Udine, Udine, Italy

    Andrea Fusiello

  3. University of York, York, UK

    Edwin Hancock

Rights and permissions

Reprints and permissions

Copyright information

© 2023 The Author(s), under exclusive license to Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Messina, N., Falchi, F., Furnari, A., Gennaro, C., Farinella, G.M. (2023). An Optimized Pipeline for Image-Based Localization in Museums from Egocentric Images. In: Foresti, G.L., Fusiello, A., Hancock, E. (eds) Image Analysis and Processing – ICIAP 2023. ICIAP 2023. Lecture Notes in Computer Science, vol 14233. Springer, Cham. https://doi.org/10.1007/978-3-031-43148-7_43

Download citation

  • DOI: https://doi.org/10.1007/978-3-031-43148-7_43

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-031-43147-0

  • Online ISBN: 978-3-031-43148-7

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation