Skip to main content
SpringerLink
Log in

APS: A Large-Scale Multi-modal Indoor Camera Positioning System

  • Conference paper
  • First Online:
Pattern Recognition and Artificial Intelligence (MedPRAI 2020)

Part of the book series: Communications in Computer and Information Science ((CCIS,volume 1322))

  • 566 Accesses

Abstract

Navigation inside a closed area with no GPS-signal accessibility is a highly challenging task. In order to tackle this problem, recently the imaging-based methods have grabbed the attention of many researchers. These methods either extract the features (e.g. using SIFT, or SOSNet) and map the descriptive ones to the camera position and rotation information, or deploy an end-to-end system that directly estimates this information out of RGB images, similar to PoseNet. While the former methods suffer from heavy computational burden during the test process, the latter suffers from lack of accuracy and robustness against environmental changes and object movements. However, end-to-end systems are quite fast during the test and inference and are pretty qualified for real-world applications, even though their training phase could be longer than the former ones. In this paper, a novel multi-modal end-to-end system for large-scale indoor positioning has been proposed, namely APS (Alpha Positioning System), which integrates a Pix2Pix GAN network to reconstruct the point cloud pair of the input query image, with a deep CNN network in order to robustly estimate the position and rotation information of the camera. For this integration, the existing datasets have the shortcoming of paired RGB/point cloud images for indoor environments. Therefore, we created a new dataset to handle this situation. By implementing the proposed APS system, we could achieve a highly accurate camera positioning with a precision level of less than a centimeter.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    https://sketchfab.com/TheHallwylMuseum.

  2. 2.

    http://opensource.alphareality.io.

References

  1. Brahmbhatt, S., Gu, J., Kim, K., Hays, J., Kautz, J.: Geometry-aware learning of maps for camera localization. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 2616–2625 (2018)

    Google Scholar 

  2. Caso, G., De Nardis, L., Lemic, F., Handziski, V., Wolisz, A., Di Benedetto, M.G.: Vifi: virtual fingerprinting wifi-based indoor positioning via multi-wall multi-floor propagation model. IEEE Trans. Mobile Comput. (2019)

    Google Scholar 

  3. Duque Domingo, J., Cerrada, C., Valero, E., Cerrada, J.: An improved indoor positioning system using RGB-D cameras and wireless networks for use in complex environments. Sensors 17(10), 2391 (2017)

    Article  Google Scholar 

  4. Duque Domingo, J., Cerrada, C., Valero, E., Cerrada, J.A.: Indoor positioning system using depth maps and wireless networks. J. Sens. 2016 (2016)

    Google Scholar 

  5. Faragher, R., Harle, R.: An analysis of the accuracy of bluetooth low energy for indoor positioning applications. In: Proceedings of the 27th International Technical Meeting of The Satellite Division of the Institute of Navigation (ION GNSS + 2014), vol. 812, pp. 201–210 (2014)

    Google Scholar 

  6. Ghofrani, A., Mahdian, R., Tabatabaie, S.M., Tabasi, S.M.: L-icpsnet: Lidar indoor camera positioning system for RGB to point cloud translation using end2end generative network. In: 2020 8th Iranian Joint Congress on Fuzzy and intelligent Systems (CFIS), pp. 110–115. IEEE (2020)

    Google Scholar 

  7. Ghofrani, A., Toroghi, R.M., Tabatabaie, S.M.: Icps-net: An end-to-end RGB-based indoor camera positioning system using deep convolutional neural networks. arXiv preprint arXiv:1910.06219 (2019)

  8. Guan, K., Ma, L., Tan, X., Guo, S.: Vision-based indoor localization approach based on surf and landmark. In: 2016 International Wireless Communications and Mobile Computing Conference (IWCMC), pp. 655–659. IEEE (2016)

    Google Scholar 

  9. Henriques, J.F., Vedaldi, A.: Mapnet: An allocentric spatial memory for mapping environments. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 8476–8484 (2018)

    Google Scholar 

  10. Henry, P., Krainin, M., Herbst, E., Ren, X., Fox, D.: RGB-D mapping: using depth cameras for dense 3D modeling of indoor environments. In: Khatib, O., Kumar, V., Sukhatme, G. (eds.) Experimental Robotics. STAR, vol. 79, pp. 477–491. Springer, Heidelberg (2014). https://doi.org/10.1007/978-3-642-28572-1_33

    Chapter  Google Scholar 

  11. Huang, K., He, K., Du, X.: A hybrid method to improve the BLE-based indoor positioning in a dense bluetooth environment. Sensors 19(2), 424 (2019)

    Article  Google Scholar 

  12. Isola, P., Zhu, J.Y., Zhou, T., Efros, A.A.: Image-to-image translation with conditional adversarial networks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 1125–1134 (2017)

    Google Scholar 

  13. Jianyong, Z., Haiyong, L., Zili, C., Zhaohui, L.: RSSI based bluetooth low energy indoor positioning. In: 2014 International Conference on Indoor Positioning and Indoor Navigation (IPIN), pp. 526–533. IEEE (2014)

    Google Scholar 

  14. Kendall, A., Cipolla, R.: Geometric loss functions for camera pose regression with deep learning. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 5974–5983 (2017)

    Google Scholar 

  15. Kendall, A., Grimes, M., Cipolla, R.: Posenet: a convolutional network for real-time 6-dof camera relocalization. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 2938–2946 (2015)

    Google Scholar 

  16. Kingma, D.P., Ba, J.: Adam: a method for stochastic optimization. arXiv preprint arXiv:1412.6980 (2014)

  17. Kumar, S.K.: On weight initialization in deep neural networks. arXiv preprint arXiv:1704.08863 (2017)

  18. Lai, C.C., Su, K.L.: Development of an intelligent mobile robot localization system using kinect RGB-D mapping and neural network. Comput. Electr. Eng. 67, 620–628 (2018)

    Article  Google Scholar 

  19. Liang, J.Z., Corso, N., Turner, E., Zakhor, A.: Image based localization in indoor environments. In: 2013 Fourth International Conference on Computing for Geospatial Research and Application, pp. 70–75. IEEE (2013)

    Google Scholar 

  20. Liang, J.Z., Corso, N., Turner, E., Zakhor, A.: Image-based positioning of mobile devices in indoor environments. In: Choi, J., Friedland, G. (eds.) Multimodal Location Estimation of Videos and Images, pp. 85–99. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-09861-6_5

    Chapter  Google Scholar 

  21. Lin, X.Y., Ho, T.W., Fang, C.C., Yen, Z.S., Yang, B.J., Lai, F.: A mobile indoor positioning system based on ibeacon technology. In: 2015 37th Annual International Conference of the IEEE Engineering in Medicine and Biology Society (EMBC), pp. 4970–4973. IEEE (2015)

    Google Scholar 

  22. Liu, T., Zhang, X., Li, Q., Fang, Z., Tahir, N.: An accurate visual-inertial integrated geo-tagging method for crowdsourcing-based indoor localization. Remote Sens. 11(16), 1912 (2019)

    Article  Google Scholar 

  23. Ma, Z., Wu, B., Poslad, S.: A wifi RSSI ranking fingerprint positioning system and its application to indoor activities of daily living recognition. Int. J. Distrib. Sensor Networks 15(4), 1550147719837916 (2019)

    Article  Google Scholar 

  24. Mekki, K., Bajic, E., Meyer, F.: Indoor positioning system for IoT device based on BLE technology and MQTT protocol. In: 2019 IEEE 5th World Forum on Internet of Things (WF-IoT), pp. 787–792. IEEE (2019)

    Google Scholar 

  25. Mendoza-Silva, G.M., Torres-Sospedra, J., Huerta, J.: A meta-review of indoor positioning systems. Sensors 19(20), 4507 (2019)

    Article  Google Scholar 

  26. Morgado, F., Martins, P., Caldeira, F.: Beacons positioning detection, a novel approach. In: The 10th International Conference on Ambient Systems, Networks and Technologies (ANT 2019), vol. 151, pp. 23–30 (2019)

    Google Scholar 

  27. Ramachandran, P., Zoph, B., Le, Q.V.: Swish: a self-gated activation function, vol. 7. arXiv preprint arXiv:1710.05941 (2017)

  28. Ronneberger, O., Fischer, P., Brox, T.: U-Net: convolutional networks for biomedical image segmentation. In: Navab, N., Hornegger, J., Wells, W.M., Frangi, A.F. (eds.) MICCAI 2015. LNCS, vol. 9351, pp. 234–241. Springer, Cham (2015). https://doi.org/10.1007/978-3-319-24574-4_28

    Chapter  Google Scholar 

  29. Russ, M., et al.: Augmented reality systems and methods for providing player action recommendations in real time, 14 February 2019. US Patent App. 15/852, 088

    Google Scholar 

  30. Sandler, M., Howard, A., Zhu, M., Zhmoginov, A., Chen, L.C.: Mobilenetv 2: inverted residuals and linear bottlenecks. In: Proceedings of the IEEE Conference on Computer Vision and Pattern Recognition, pp. 4510–4520 (2018)

    Google Scholar 

  31. Sattler, T., Leibe, B., Kobbelt, L.: Improving image-based localization by active correspondence search. In: Fitzgibbon, A., Lazebnik, S., Perona, P., Sato, Y., Schmid, C. (eds.) ECCV 2012. LNCS, vol. 7572, pp. 752–765. Springer, Heidelberg (2012). https://doi.org/10.1007/978-3-642-33718-5_54

    Chapter  Google Scholar 

  32. Sattler, T., Leibe, B., Kobbelt, L.: Efficient and effective prioritized matching for large-scale image-based localization. IEEE Trans. Pattern Anal. Mach. Intell. 39(9), 1744–1756 (2016)

    Article  Google Scholar 

  33. Shao, S., Shuo, N., Kubota, N.: An ibeacon indoor positioning system based on multi-sensor fusion. In: 2018 Joint 10th International Conference on Soft Computing and Intelligent Systems (SCIS) and 19th International Symposium on Advanced Intelligent Systems (ISIS), pp. 1115–1120. IEEE (2018)

    Google Scholar 

  34. Sharp, I., Yu, K.: Indoor WiFi positioning. Wireless Positioning: Principles and Practice. NST, pp. 219–240. Springer, Singapore (2019). https://doi.org/10.1007/978-981-10-8791-2_8

    Chapter  Google Scholar 

  35. Shen, Z., Liu, J., Zheng, Y., Cao, L.: A low-cost mobile VR walkthrough system for displaying multimedia works based on unity3d. In: 2019 14th International Conference on Computer Science and Education (ICCSE), pp. 415–419. IEEE (2019)

    Google Scholar 

  36. Sieberth, T., Dobay, A., Affolter, R., Ebert, L.C.: Applying virtual reality in forensics-a virtual scene walkthrough. Forensic Sci. Med. Pathol. 15(1), 41–47 (2019)

    Article  Google Scholar 

  37. Tan, M., Le, Q.V.: Efficientnet: rethinking model scaling for convolutional neural networks. arXiv preprint arXiv:1905.11946 (2019)

  38. Valgren, C., Lilienthal, A.J.: Sift, surf and seasons: appearance-based long-term localization in outdoor environments. Robot. Auton. Syst. 58(2), 149–156 (2010)

    Article  Google Scholar 

  39. Wan, L., Zeiler, M., Zhang, S., Le Cun, Y., Fergus, R.: Regularization of neural networks using dropconnect. In: International Conference on Machine Learning, pp. 1058–1066 (2013)

    Google Scholar 

  40. Wang, R., Wan, W., Di, K., Chen, R., Feng, X.: A high-accuracy indoor-positioning method with automated RGB-D image database construction. Remote Sens. 11(21), 2572 (2019)

    Article  Google Scholar 

  41. Yang, C., Shao, H.R.: Wifi-based indoor positioning. IEEE Commun. Mag. 53(3), 150–157 (2015)

    Article  Google Scholar 

  42. Yuan, W., Li, Z., Su, C.Y.: RGB-D sensor-based visual slam for localization and navigation of indoor mobile robot. In: 2016 International Conference on Advanced Robotics and Mechatronics (ICARM), pp. 82–87. IEEE (2016)

    Google Scholar 

  43. Zhang, F., et al.: Real-time calibration and registration method for indoor scene with joint depth and color camera. Int. J. Pattern Recogn. Artif. Intell. 32(07), 1854021 (2018)

    Article  Google Scholar 

  44. Zuo, Z., Liu, L., Zhang, L., Fang, Y.: Indoor positioning based on bluetooth low-energy beacons adopting graph optimization. Sensors 18(11), 3736 (2018)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. Iran Broadcasting University (IRIBU), Tehran, Iran

    Ali Ghofrani & Rahil Mahdian Toroghi

  2. CEO/CTO at Alpha Reality, AR/VR Solution Company, Tehran, Iran

    Seyed Mojtaba Tabatabaie

  3. VP of Artificial Intelligence at Alpha Reality, Tehran, Iran

    Ali Ghofrani

Authors
  1. Ali Ghofrani
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Rahil Mahdian Toroghi
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Seyed Mojtaba Tabatabaie
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Rahil Mahdian Toroghi .

Editor information

Editors and Affiliations

  1. Larbi Tebessi University, Tebessa, Algeria

    Chawki Djeddi

  2. Digital Research Center of Sfax, Sfax, Tunisia

    Yousri Kessentini

  3. Bahria University, Islamabad, Pakistan

    Imran Siddiqi

  4. Digital Research Centre of Sfax, Sfax, Tunisia

    Mohamed Jmaiel

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Ghofrani, A., Toroghi, R.M., Tabatabaie, S.M. (2021). APS: A Large-Scale Multi-modal Indoor Camera Positioning System. In: Djeddi, C., Kessentini, Y., Siddiqi, I., Jmaiel, M. (eds) Pattern Recognition and Artificial Intelligence. MedPRAI 2020. Communications in Computer and Information Science, vol 1322. Springer, Cham. https://doi.org/10.1007/978-3-030-71804-6_3

Download citation

  • DOI: https://doi.org/10.1007/978-3-030-71804-6_3

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-71803-9

  • Online ISBN: 978-3-030-71804-6

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics

Search

Navigation