Skip to main content
Log in

How to add new knowledge to already trained deep learning models applied to semantic localization

  • Published:
Applied Intelligence Aims and scope Submit manuscript

Abstract

The capacity of a robot to automatically adapt to new environments is crucial, especially in social robotics. Often, when these robots are deployed in home or office environments, they tend to fail because they lack the ability to adapt to new and continuously changing scenarios. In order to accomplish this task, robots must obtain new information from the environment, and then add it to their already learned knowledge. Deep learning techniques are often used to tackle this problem successfully. However, these approaches, complete retraining of the models, which is highly time-consuming. In this work, several strategies are tested to find the best way to include new knowledge in an already learned model in a deep learning pipeline, putting the spotlight on the time spent for this training. We tackle the localization problem in the long term with a deep learning approach and testing several retraining strategies. The results of the experiments are discussed and, finally, the best approach is deployed on a Pepper robot.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

Similar content being viewed by others

Notes

  1. https://github.com/spotify/annoy

References

  1. Bui H M, Lech M, Cheng E, Neville K, Burnett I S (2016) Object recognition using deep convolutional features transformed by a recursive network structure. IEEE Access 4:10059–10066. https://doi.org/10.1109/ACCESS.2016.2639543

    Article  Google Scholar 

  2. Dillmann R (2004) Teaching and learning of robot tasks via observation of human performance. Robot Auton Syst 47(2):109–116. https://doi.org/10.1016/j.robot.2004.03.005. http://www.sciencedirect.com/science/article/pii/S0921889004000417, robot Learning from Demonstration

    Article  MathSciNet  Google Scholar 

  3. Dillmann R, Rogalla O, Ehrenmann M, Zöliner R, Bordegoni M (2000) Learning robot behaviour and skills based on human demonstration and advice: The machine learning paradigm. In: Hollerbach JM, Koditschek DE (eds) Robotics research. Springer, London, pp 229–238

    Chapter  Google Scholar 

  4. Eitel A, Springenberg J T, Spinello L, Riedmiller M A, Burgard W (2015) Multimodal deep learning for robust RGB-D object recognition. arXiv:1507.06821

  5. Galindo C, Saffiotti A, Coradeschi S, Buschka P, Fernandez-Madrigal JA, Gonzalez J (2005) Multi-hierarchical semantic maps for mobile robotics. In: 2005 IEEE/RSJ international conference on intelligent robots and systems, pp 2278–2283. https://doi.org/10.1109/IROS.2005.1545511

  6. Goeddel R, Olson E (2016) Learning semantic place labels from occupancy grids using cnns. In: 2016 IEEE/RSJ international conference on intelligent robots and systems (IROS), pp 3999–4004. https://doi.org/10.1109/IROS.2016.7759589

  7. He K, Zhang X, Ren S, Sun J (2015) Deep residual learning for image recognition. arXiv:1512.03385

  8. Krizhevsky A, Sutskever I, Hinton GE (2012) Imagenet classification with deep convolutional neural networks. In: Pereira F, Burges CJC, Bottou L, Weinberger KQ (eds) Advances in neural information processing systems 25, Curran Associates, Inc., pp 1097–1105. http://papers.nips.cc/paper/4824-imagenet-classification-with-deep-convolutional-neural-networks.pdf

  9. Li Z, Hoiem D (2016) Learning without forgetting. arXiv:1606.09282

    Chapter  Google Scholar 

  10. Lowry S, Sünderhauf N, Newman P, Leonard J J, Cox D, Corke P, Milford M J (2016) Visual place recognition: A survey. IEEE Trans Robot 32(1):1–19. https://doi.org/10.1109/TRO.2015.2496823

    Article  Google Scholar 

  11. Martinez-Gomez J, Garcia-Varea I, Cazorla M, Morell V (2015) Vidrilo: The visual and depth robot indoor localization with objects information dataset. Int J Robot Res 34(14):1681–1687. https://doi.org/10.1177/0278364915596058

    Article  Google Scholar 

  12. Martínez-Gómez J, Morell V, Cazorla M, García-Varea I (2016) Semantic localization in the {PCL} library. Robot Auton Syst 75(Part B):641–648. https://doi.org/10.1016/j.robot.2015.09.006 https://doi.org/10.1016/j.robot.2015.09.006. http://www.sciencedirect.com/science/article/pii/S0921889015001943

    Article  Google Scholar 

  13. Pronobis A, Mozos OM, Caputo B, Jensfelt P (2010) Multi-modal semantic place classification. Int J Robot Res 29(2-3):298–320. https://doi.org/10.1177/0278364909356483

    Article  Google Scholar 

  14. Rangel JC, Cazorla M, García-Varea I, Martínez-Gómez J, Fromont É, Sebban M (2016) Scene classification based on semantic labeling. Adv Robot 30(11-12):758–769. https://doi.org/10.1080/01691864.2016.1164621

    Article  Google Scholar 

  15. Romero-González C, Martínez-Gómez J, García-Varea I, Rodríguez-Ruiz L (2017) On robot indoor scene classification based on descriptor quality and efficiency. Expert Syst Appl 79:181–193. https://doi.org/10.1016/j.eswa.2017.02.040. http://www.sciencedirect.com/science/article/pii/S0957417417301318

    Article  Google Scholar 

  16. Rubio F, Martínez-Gómez J, Flores MJ, Puerta JM (2016) Comparison between bayesian network classifiers and {SVMs} for semantic localization. Expert Syst Appl 64:434–443. https://doi.org/10.1016/j.eswa.2016.08.029. http://www.sciencedirect.com/science/article/pii/S0957417416304237

    Article  Google Scholar 

  17. Ruiz-Sarmiento JR, Galindo C, Gonzalez-Jimenez J (2017) Building multiversal semantic maps for mobile robot operation. Knowl-Based Syst 119:257–272. http://www.sciencedirect.com/science/article/pii/S0950705116305184

    Article  Google Scholar 

  18. Schwarz M, Schulz H, Behnke S (2015) Rgb-d object recognition and pose estimation based on pre-trained convolutional neural network features. In: 2015 IEEE international conference on robotics and automation (ICRA), pp 1329–1335. https://doi.org/10.1109/ICRA.2015.7139363

  19. van der Maaten L, Hinton G (2008) Visualizing high-dimensional data using t-sne. J Mach Learn Res 9:2579–2605

    MATH  Google Scholar 

  20. Yosinski J, Clune J, Bengio Y, Lipson H (2014) How transferable are features in deep neural networks?. arXiv:1411.1792

  21. Zhou B, Lapedriza A, Khosla A, Oliva A, Torralba A (2017) Places: A 10 million image database for scene recognition. IEEE Transactions on Pattern Analysis and Machine Intelligence

Download references

Acknowledgments

This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with Feder funds. Edmanuel Cruz is funded by a Panamenian grant for PhD studies IFARHU & SENACYT 270-2016-207. Jose Carlos Rangel was supported by the National System of Research (SNI) of the SENACYT of Panama. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243. Thanks also to Nvidia for the generous donation of a Titan Xp and a Quadro P6000.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to Francisco Gomez-Donoso.

Additional information

Publisher’s note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

This work has been supported by the Spanish Government TIN2016-76515R Grant, supported with Feder funds. Edmanuel Cruz is funded by a Panamenian grant for PhD studies IFARHU & SENACYT 270-2016-207. This work has also been supported by a Spanish grant for PhD studies ACIF/2017/243. Thanks to Nvidia also for the generous donation of a Titan Xp and a Quadro P6000.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cruz, E., Rangel, J.C., Gomez-Donoso, F. et al. How to add new knowledge to already trained deep learning models applied to semantic localization. Appl Intell 50, 14–28 (2020). https://doi.org/10.1007/s10489-019-01517-1

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10489-019-01517-1

Keywords

Navigation