Abstract
Two frameworks of hidden Markov modeling for multi-agent systems and its learning procedure are proposed. Although a couple of variations of HMMs have been proposed to model agents and their interactions, these models have not handled changes of environments, so that it is hard to simulate behaviors of agents that act in dynamic environments like soccer. The proposed frameworks enables HMMs to represent environments directly inside of state transitions. I first propose a model that handles the dynamics of the environments in the same state transition of the agent itself. In this model, the derived learning procedure can segment the environments according to the tasks and behaviors the agent is performing. I also investigate a more structured model in which the dynamics of the environments and agents are treated as separated state transitions and coupled each other. For this model, in order to reduce the number of parameters, I introduce “symmetricity” among agents. Furthermore, I discuss relation between reducing dependency in transitions and assumption of cooperative behaviors among multiple agents.
Chapter PDF
Similar content being viewed by others
References
Bengio, Y., Frasconi, P.: An input output hmm architecuture. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, pp. 427–434. The MIT Press, Cambridge (1995)
Brand, M.: Coupled hideen markov models for modeling interacting processes. Perceptual Computing/Learning and Common Sense Technical Report 405, MIT Lab (June 1997)
Ghahramani, Z., Jordan, M.I.: Factorial hidden markov models. Machine Learning 29, 245–275 (1997)
Coupled hidden Markov models for complex action recognition. Matthew brand and nuria oliver and alex pentland. Perceptual Computing/Learning and Common Sense Technical Report 407, MIT Media Lab 20 (1996)
Han, K., Veloso, M.: Automated robot behavior recognition applied to robotic soccer. In: Proceedings of IJCAI 1999 Workshop on Team Behaviors and Plan Recognition (1999)
Jordan, J.M.I., Ghahramani, Z., Jaakkola, T., Saul, L.K.: An introduction to variational methods for graphical models. Machine Learning 37(2), 183–233 (1999)
Jordan, M.I., Ghahramani, Z., Saul, L.K.: Hidden markov decision trees. In: Mozer, M.C., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, p. 501. The MIT Press, Cambridge (1997)
Jordan, M.I., Ghahramani, Z., Saul, L.K.: Hidden markov decision trees. In: Mozer, M.C., Jordan, M.I., Petsche, T. (eds.) Advances in Neural Information Processing Systems, vol. 9, p. 501. The MIT Press, Cambridge (1997)
Bengio, Y., Frasconi, P.: An input output hmm architecuture. In: Tesauro, G., Touretzky, D., Leen, T. (eds.) Advances in Neural Information Processing Systems, pp. 427–434. The MIT Press, Cambridge (1995)
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2003 Springer-Verlag Berlin Heidelberg
About this paper
Cite this paper
Noda, I. (2003). Hidden Markov Modeling of Multi-agent Systems and Its Learning Method. In: Kaminka, G.A., Lima, P.U., Rojas, R. (eds) RoboCup 2002: Robot Soccer World Cup VI. RoboCup 2002. Lecture Notes in Computer Science(), vol 2752. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-45135-8_8
Download citation
DOI: https://doi.org/10.1007/978-3-540-45135-8_8
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-540-40666-2
Online ISBN: 978-3-540-45135-8
eBook Packages: Springer Book Archive