Fuzzy Optimization and Decision Making

, Volume 1, Issue 2, pp 143–159 | Cite as

Fuzzy Markov Chains and Decision-Making

  • Konstantin E. Avrachenkov
  • Elie Sanchez


Decision-making in an environment of uncertainty and imprecision for real-world problems is a complex task. In this paper it is introduced general finite state fuzzy Markov chains that have a finite convergence to a stationary (may be periodic) solution. The Cesaro average and the α-potential for fuzzy Markov chains are defined, then it is shown that the relationship between them corresponds to the Blackwell formula in the classical theory of Markov decision processes. Furthermore, it is pointed out that recurrency does not necessarily imply ergodicity. However, if a fuzzy Markov chain is ergodic, then the rows of its ergodic projection equal the greatest eigen fuzzy set of the transition matrix. Then, the fuzzy Markov chain is shown to be a robust system with respect to small perturbations of the transition matrix, which is not the case for the classical probabilistic Markov chains. Fuzzy Markov decision processes are finally introduced and discussed.

fuzzy Markov chains fuzzy Markov decision processes eigen fuzzy sets ergodicity robustness 


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. Abbad, M. and J.A. Filar. (1992). “Perturbation and Stability Theory for Markov Control Problems,” IEEE Trans. Auto. Contr. 37, 1415–1420.Google Scholar
  2. Avrachenkov, K.E. and E. Sanchez. (2000). “Fuzzy Markov Chains: Specificities and Properties,” Proceedings of the 8-th IPMU'2000 Conference, Madrid,Spain, 1851–1856.Google Scholar
  3. Bellman, R.E. (1957). Dynamic Programming, Princeton University Press.Google Scholar
  4. Bellman, R.E. and L.A. Zadeh. (1970). “Decision-Making in a Fuzzy Environment,” Management Science Series B 17, 141–164.Google Scholar
  5. Chung, K.L. (1960). Markov Chains with Stationary Transition Probabilities, New York: Springer-Verlag.Google Scholar
  6. Delebecque, F. and J.P. Quadrat. (1981). “Optimal Control of Markov Chains Admitting Strong and Weak Interactions,” Automatica 17, 281–296.Google Scholar
  7. Doob, J.L. (1953). Stochastic Processes, New York: John Wiley.Google Scholar
  8. Esogbue, A.O. and R.E. Bellman. (1984). “Fuzzy Dynamic Programming and its Extensions,” TIMS/Studies in the Management Sciences 20, 147–167.Google Scholar
  9. Give'on, Y. (1964). “Lattice Matrices,” Information and Control 7, 477–484. Kruse, R., R. Buck-Emden, and R. Cordes. (1987). “Processor Power Considerations-An Application of Fuzzy Markov Chains,” Fuzzy Sets and Systems 21, 289-299.Google Scholar
  10. Kurano, M., M. Yasuda, J. Nakagami, and Y. Yoshida. (1992). “A Limit Theorem in Some Fuzzy Dynamic Systems,” Fuzzy Sets and Systems 51, 83–88.Google Scholar
  11. Kurano, M., M. Yasuda, J. Nakagami, and Y. Yoshida. (1996). “Markov-Type Fuzzy Decision Processes with a Discounted Reward on a Closed Interval,” European Journal of Operational Research 92, 649–662.Google Scholar
  12. Pervozvanskii, A.A. and V.G. Gaitsgori. (1988). Theory of Suboptimal Decisions, Dordrecht: Kluwer Academic Publishers.Google Scholar
  13. Puterman, M.L. (1994). Markov Decision Processes, New York: John Wiley.Google Scholar
  14. Sanchez, E. (1976). “Resolution of Composite Fuzzy Relation Equations,” Information and Control 30, 38–48.Google Scholar
  15. Sanchez, E. (1978). “Resolution of Eigen Fuzzy Sets Equations,” Fuzzy Sets and Systems 1, 69–74.Google Scholar
  16. Sanchez, E. (1981). “Eigen Fuzzy Sets and Fuzzy Relations,” J. Math. Anal. Appl. 81, 399–421.Google Scholar
  17. Schweizer, B. and A. Sklar. (1963). “Associative Functions and Abstract Semigroups,” Publ. Math. Debrecen. 10, 69–81.Google Scholar
  18. Schweitzer, P.J. (1968). “Perturbation Theory and Finite Markov Chains,” J. App. Prob. 5, 401–413.Google Scholar
  19. Thomason, M.G. (1977). “Convergence of Powers of a Fuzzy Matrix,” J. Math. Anal. Appl. 57, 476–480.Google Scholar
  20. Yoshida, Y., M. Yasuda, J. Nakagami, and M. Kurano. (1993). “A Potential of Fuzzy Relations with a Linear Structure: The Contractive Case,” Fuzzy Sets and Systems 60, 283–294.Google Scholar
  21. Yoshida, Y., M. Yasuda, J. Nakagami, and M. Kurano. (1994). “A Potential of Fuzzy Relations with a Linear Structure: The Unbounded Case,” Fuzzy Sets and Systems 66, 83–95.Google Scholar
  22. Yoshida, Y. (1994). “Markov Chains with a Transition Possibility Measure and Fuzzy Dynamic Programming,” Fuzzy Sets and Systems 66, 39–57.Google Scholar
  23. Zadeh, L.A. (1965). “Fuzzy Sets,” Information and Control 8, 338–353.Google Scholar
  24. Zadeh, L.A. (1998). “Maximizing Sets and Fuzzy Markoff Algorithms,” IEEE Trans. on Systems, Man, and Cybernetics-Part C: Applications and Reviews 28, 9–15.Google Scholar

Copyright information

© Kluwer Academic Publishers 2002

Authors and Affiliations

  • Konstantin E. Avrachenkov
    • 1
  • Elie Sanchez
    • 2
  1. 1.INRIA Sophia AntipolisFrance
  2. 2.Laboratoire d'Informatique MédicaleFaculté de MédecineMarseille Cedex 5France

Personalised recommendations