Skip to main content

Entropy and PCA Analysis for Environments Associated to Q-Learning for Path Finding

  • 172 Accesses

Part of the Communications in Computer and Information Science book series (CCIS,volume 1096)


This work is based on the simulation of the reinforcement learning method for path search for mobile robotic agents in unstructured environments. The choice of the learning and reward coefficients of the Q-learning method affect the number of average actions that the algorithm requires for reach the goal from the start position. In addition, another important factor is the randomness degree and environment size over which the path must be calculated, since they affect the time of convergence in the learning. Likewise, a performance metric of the Q-learning algorithm is proposed, based on the Entropy and Principal Component Analysis of the environment representative images. The analysis by Entropy only allows to determine, in a scalar form, the environment randomness degree, but it does not provide information about the dispersion location. In contrast, the analysis by PCA allows to quantify not only the randomness, but also helps to estimate the direction of greater randomness of the environment. The advantage of this analysis by PCA and Entropy is that one could estimate the actions number or movements required for path search algorithms based on the randomness of unstructured environments.


  • Entropy
  • PCA
  • Unstructured environment
  • Q-Learning
  • Path planning

This is a preview of subscription content, access via your institution.

Buying options

USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-030-36211-9_17
  • Chapter length: 14 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
USD   64.99
Price excludes VAT (USA)
  • ISBN: 978-3-030-36211-9
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   84.99
Price excludes VAT (USA)
Fig. 1.
Fig. 2.
Fig. 3.
Fig. 4.
Fig. 5.
Fig. 6.
Fig. 7.
Fig. 8.
Fig. 9.
Fig. 10.
Fig. 11.


  1. Nurmaini, S., et al.: Intelligent navigation in unstructured environment by using memory-based reasoning in embedded mobile robot. Eur. J. Sci. Res. 2(72), 228–244 (2012)

    Google Scholar 

  2. Ingrand, F., Ghallab, M.: Deliberation for autonomous robots: a survey. Artif. Intell. 247, 10–44 (2017)

    MathSciNet  CrossRef  Google Scholar 

  3. Sutton, R.S., Barto, A.: Reinforcement Learning: An Introduction, 2nd edn. MIT Press, MA (2012)

    MATH  Google Scholar 

  4. Tamilselvi, D., et al.: Q learning for mobile robot navigation in indoor environment. In: IEEE-International Conference on Recent Trends in Information Technology, pp. 324–329. MIT Anna University, Chennai (2011)

    Google Scholar 

  5. Xiang, Y., et al.: A PCA-based model to predict adversarial examples on Q-learning of path finding. In: Third International Conference on Data Science in Cyberspace, pp. 773–780. IEEE, USA (2018)

    Google Scholar 

  6. Xing, J., et al.: Application of deep reinforcement learning in mobile robot path planning. In: Chinese Automatic Congress (CAC), pp. 7112–711. IEEE (2017)

    Google Scholar 

  7. Patle, B.K., et al.: Real time navigation approach for mobile robot. J. Comput. 2(12), 135–142 (2017)

    CrossRef  Google Scholar 

  8. Poncela, A., et al.: Efficient integration of metric and topological maps for directed exploration of unknown environments. Robot. Auton. Syst. 41, 21–39 (2002)

    CrossRef  Google Scholar 

  9. Yang, S., Li, C.: Behavior control algorithm for mobile robot based on Q-Learning. In: 2017 International Conference on Computer Network, Electronic and Automation, pp. 45–47. IEEE (2017)

    Google Scholar 

  10. Shannon, C.E.: A mathematical theory of communication. Bell Syst. Tech. J. 2(27), 379–423 (1948)

    MathSciNet  CrossRef  Google Scholar 

  11. Gonzalez, R.C., et al.: Digital Image Processing Using MATLAB. Prentice Hall, New Jersey (2003)

    Google Scholar 

  12. Hong, D., et al.: The entropy and PCA based anomaly prediction in data streams. In: 20th International Conference on Knowledge Based and Intelligent Information and Engineering Systems, pp. 139–146. Procedia Computer Science 96, USA (2016)

    Google Scholar 

  13. Ng, S.C.: Principal component analysis to reduce dimension on digital image. In: 8th International Conference on Advances in Information Technology, Procedia Computer Science, vol. 1(111), pp. 113–119, Macau China (2017)

    Google Scholar 

  14. Gao, C.: Theory of fractional covariance matrix and its applications in PCA and 2D-PCA. Expert Syst. Appl. 1(40), 5395–5401 (2013)

    CrossRef  Google Scholar 

  15. Zhang, L., et al.: Two-stage image denoising by principal component analysis with local pixel grouping. Pattern Recognit. 1(43), 1531–1549 (2010)

    CrossRef  Google Scholar 

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to Manuel Garcia-Quijada .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2019 Springer Nature Switzerland AG

About this paper

Verify currency and authenticity via CrossMark

Cite this paper

Garcia-Quijada, M., Gorrostieta-Hurtado, E., Vargas-Soto, J.E., Toledano-Ayala, M. (2019). Entropy and PCA Analysis for Environments Associated to Q-Learning for Path Finding. In: Orjuela-Cañón, A., Figueroa-García, J., Arias-Londoño, J. (eds) Applications of Computational Intelligence. ColCACI 2019. Communications in Computer and Information Science, vol 1096. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-36210-2

  • Online ISBN: 978-3-030-36211-9

  • eBook Packages: Computer ScienceComputer Science (R0)