Skip to main content

A Supervisor Αgent-Based on the Markovian Decision Process Framework to Optimize the Behavior of a Highly Automated System

  • Conference paper
  • First Online:
Augmented Cognition (HCII 2021)

Part of the book series: Lecture Notes in Computer Science ((LNAI,volume 12776))

Included in the following conference series:


In this paper, we explore how MDP can be used as the framework to design and develop an Intelligent Decision Support System/Recommender System, in order to extend human perception and overcome human senses limitations (because covered by the ADS), by augmenting human cognition, emphasizing human judgement and intuition, as well as supporting him/her to take the proper decision in the right terms and time.

Moreover, we develop Human-Machine Interaction (HMI) strategies able to make “transparent” the decision-making/recommendation process. This is strongly needed, since the adoption of partial automated systems is not only connected to the effectiveness of the decision and control processes, but also relies on how these processes are communicated and “explained” to the human driver, in order to achieve his/her trust.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Subscribe and save

Springer+ Basic
EUR 32.99 /Month
  • Get 10 units per month
  • Download Article/Chapter or Ebook
  • 1 Unit = 1 Article or 1 Chapter
  • Cancel anytime
Subscribe now

Buy Now

USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
USD 39.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 54.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Similar content being viewed by others


  1. 1.

    For more information, see the website:


  1. Rasmussen, J.: Human errors. A taxonomy for describing human malfunction in industrial installations. J. Occup. Accid. 4(2–4), 311–333 (1982)

    Article  Google Scholar 

  2. Siau, K., Wang, W.: Building trust in artificial intelligence, machine learning, and robotics. Cutter Bus. Technol. J. 31(2), 47–53 (2018)

    Google Scholar 

  3. Puterman, M.L.: Markov Decision Processes. Discrete Stochastic Dynamic Programming. Wiley, Chichester (2005)

    MATH  Google Scholar 

  4. Jean Piaget’s Theory and Stages of Cognitive Development, by Saul McLeod, Simply Psychology. Accessed 2018

    Google Scholar 

  5. Michon, J.A.: A critical view of driver behavior models: what do we know, what should we do? In: Evans, L., Schwing, R.C. (eds.) Human Behavior and Traffic Safety, pp. 485–524. Springer US, Boston, MA (1986).

    Chapter  Google Scholar 

  6. Tango, F., Aras, R., Pietquin, O.: Learning Optimal Control Strategies from Interactions with a PADAS. In: Cacciabue, P.C., Hjälmdahl, Magnus, Luedtke, Andreas, Riccioli, Costanza (eds.) Human Modelling in Assisted Transportation, pp. 119–127. Springer Milan, Milano (2011).

    Chapter  Google Scholar 

  7. Ricci, F., Rokach, L., Shapira, B.: Introduction to recommender systems handbook. In: Ricci, F., Rokach, L., Shapira, B., Kantor, P.B. (eds.) Recommender Systems Handbook, pp. 1–35. Springer, Boston, MA (2011).

    Chapter  MATH  Google Scholar 

  8. Fayyaz, Z., Ebrahimian, M., Nawara, D., Ibrahim, A., Kashef, R.: Recommendation systems: algorithms, challenges, metrics, and business opportunities. Appl. Sci. 10(21), 7748 (2020).

    Article  Google Scholar 

  9. Beel, J., Langer, S., Genzmehr, M., Gipp, B., Breitinger, C., Nürnberger, A.: Research paper recommender system evaluation: a quantitative literature survey. In: Proceedings of the International Workshop on Reproducibility and Replication in Recommender Systems Evaluation, Hong Kong, China, 12 October (2013), pp. 15–22 (2013)

    Google Scholar 

  10. Carsten, O., Martens, M.H.: How can humans understand their automated cars? HMI principles, problems and solutions. Cogn. Technol. Work 21(1), 3–20 (2018).

    Article  Google Scholar 

  11. Sharma, A., et al.: Is an informed driver a better decision maker? a grouped random parameter with heterogeneity-in-means approach to investigate the impact of the connected environment on driving behavior in safety-critical situations. Anal. Meth. Accid. Res. 27, 100127 (2020)

    Google Scholar 

  12. Castellano, A., et al.: Is your request just this? New automation paradigm to reduce the requests of transition without increasing the effort of the driver. In: 25th ITS World Congress. Copenhagen, Denmark, vol. 17 (2018)

    Google Scholar 

  13. Gowda, N., Ju, W., Kohler, K.: Dashboard design for an autonomous car. In: Adjunct Proceedings of the 6th International Conference on Automotive user Interfaces and Interactive Vehicular Applications (2014)

    Google Scholar 

  14. Ju, W.: The design of implicit interactions. Synth. Lect. Hum.-Centered Inf. 8(2), 1–93 (2015)

    Article  Google Scholar 

  15. Findeisen, R., Allgower, F., Biegel, L.: Assessment and future directions of nonlinear model predictive control. In: Lecture Notes in Control and Information Sciences. Springer (2007).

  16. Grune, L., Pannek, J.: Nonlinear model predictive control - theory and algorithms. In: Communications and control engineering. Springer (2011)

Download references

Author information

Authors and Affiliations


Corresponding author

Correspondence to C. Novara .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and permissions

Copyright information

© 2021 Springer Nature Switzerland AG

About this paper

Check for updates. Verify currency and authenticity via CrossMark

Cite this paper

Castellano, A., Karimshoushtari, M., Novara, C., Tango, F. (2021). A Supervisor Αgent-Based on the Markovian Decision Process Framework to Optimize the Behavior of a Highly Automated System. In: Schmorrow, D.D., Fidopiastis, C.M. (eds) Augmented Cognition. HCII 2021. Lecture Notes in Computer Science(), vol 12776. Springer, Cham.

Download citation

  • DOI:

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-030-78113-2

  • Online ISBN: 978-3-030-78114-9

  • eBook Packages: Computer ScienceComputer Science (R0)

Publish with us

Policies and ethics