Intrinsically Motivated Intelligent Sensed Environments

  • Mary Lou Maher
  • Kathryn Merrick
  • Owen Macindoe
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4200)


Intelligent rooms comprise hardware devices that support human activities in a room and software that has some level of control over the devices. “Intelligent” implies that the room is considered to behave in an intelligent manner or includes some aspect of artificial intelligence in its implementation. The focus of this paper is intelligent sensed environments, including rooms or interactive spaces that display adaptive behaviour through learning and motivation. We present motivated agent models that incorporate machine learning in which the motivation component eliminates the need for a benevolent teacher to prepare problem specific reward functions or training examples. Our model of motivation is based on concepts of “curiosity”, “novelty” and “interest”. We explore the potential for this model through the implementation of a curious place.


Reinforcement Learning Supervise Learning Unsupervised Learning Agent Model Motivation Process 
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.


Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.


  1. 1.
    Aylett, R.A., et al.: Agent-based continuous planning. In: Proceedings of the 19th Workshop of the UK Planning and Scheduling Special Interest Group (PLANSIG 2000) (2000)Google Scholar
  2. 2.
    Berlyne, D.E.: Aesthetics and psychobiology. Prentice-Hall, Englewood Cliffs (1971)Google Scholar
  3. 3.
    Brdiczka, O., Reignier, P., Crowley, J.: Supervised learning of an abstract context model for an intelligent environment. In: Proceedings of the Joint sOc-EUSAI Conference, Grenoble (2005)Google Scholar
  4. 4.
    Brooks, R.A., Coen, M., Dang, D., DeBonet, J., Kramer, J., Lozano-Perez, T., Mellor, J., Pook, P., Stauffer, C., Stein, L., Torrance, M., Wessler, M.: The Intelligent Room Project. In: Proceedings of the Second International Cognitive Technology Conference (CT 1997), Aizu, Japan, pp. 271–279 (1997)Google Scholar
  5. 5.
    Canamero, L.: Modeling motivations and emotions as a basis for intelligent behaviour. In: Proceedings of the First International Symposium on Autonomous Agents. ACM Press, New York (1997)Google Scholar
  6. 6.
    Coen, M.H.: Design Principles for Intelligent Environments. In: Proceedings of the Fifteenth National / Tenth Conference on Artificial Intelligence / Innovative Applications of Artificial Intelligence, Madison, Wisconsin, United States, pp. 547–554 (1998)Google Scholar
  7. 7.
    Deci, E., Ryan, R.: Intrinsic motivation and self-determination in human behaviour. Plenum Press, New York (1985)Google Scholar
  8. 8.
    Gemeinboeck, P.: Negotiating the In-Between: Space, Body and the Condition of the Virtual. Crossings - Electronic Journal of Art and Technology 4(1) (2004)Google Scholar
  9. 9.
    Dieterich, H., Malinowski, U., Khme, T., Schneider-Hufschmidt, M.: State of the art in adaptive user interfaces. In: Schneider-Hufschmidt, M., Khme, T., Malinowski, U. (eds.) Adaptive User Interfaces: Principle and Practice. North-Holland, Amsterdam (1993)Google Scholar
  10. 10.
    Dzeroski, S., De Raedt, L., Blockeel, H.: Relational reinforcement learning. In: International Conference on Inductive Logic Programming (1998)Google Scholar
  11. 11.
    Gershenson, C.: Artificial Societies of Intelligent Agents, Bachelor of Engineering Thesis, Fundacion Arturo Rosenblueth (2001)Google Scholar
  12. 12.
    Graesser, A., Van Lehn, K., Rose, C., Jordan, P., Harter, D.: Intelligent tutoring systems with conversational dialogue. AI Magazine 22(4), 39–52 (2001)Google Scholar
  13. 13.
    Green, R.G., Beatty, W.W., Arkin, R.M.: Human motivation: physiological, behavioural and social approaches. Allyn and Bacon, Inc., Massachussets (1984)Google Scholar
  14. 14.
    Hammond, T., Gajos, K., Davis, R., Shrobe, H.: An Agent-Based System for Capturing and Indexing Software Design Meetings. In: Gero, J.S., Brazier, F.M.T. (eds.) Agents in Design 2002, Key Centre of Design Computing and Cognition, University of Sydney, pp. 203–218 (2002)Google Scholar
  15. 15.
    Horvitz, E., Breese, J., Heckerman, D., Hovel, D., Rommelse, K.: The Lumiere project: Bayesian user modelling for inferring the goals and needs of software users. In: Proceedings of the Fourteenth Conference on Uncertainty in Artificial Intelligence, pp. 256–265 (1998)Google Scholar
  16. 16.
    Kandel, E.R., Schwarz, J.H., Jessell, T.M.: Essentials of neural science and behaviour. Appleton and Lang, Norwalk (1995)Google Scholar
  17. 17.
    Kaplan, F., Oudeyer, P.-Y.: Motivational principles for visual know-how development. In: Proceedings of the 3rd international workshop on Epigenetic Robotics: Modeling cognitive development in robotic systems, Lund University Cognitive Studies (2003)Google Scholar
  18. 18.
    Kohonen, T.: Self-organisation and associative memory. Springer, Berlin (1993)Google Scholar
  19. 19.
    Krueger, M., Gionfriddo, T., et al.: Videoplace: an artificial reality. In: Human Factors in Computing Systems, CHI 1985. ACM Press, New York (1985)Google Scholar
  20. 20.
    Luck, M., d’Inverno, M.: Motivated behaviour for goal adoption. In: Zhang, C., Lukose, D. (eds.) DAI 1998. LNCS (LNAI), vol. 1544, pp. 58–73. Springer, Heidelberg (1998)CrossRefGoogle Scholar
  21. 21.
    Maher, M.L., Merrick, K., Macindoe, O.: Can designs themselves be creative? In: Computational and Cognitive Models of Creative Design, Heron Island, pp. 111–135 (2005)Google Scholar
  22. 22.
    Marsland, S., Nehmzow, U., Shapiro, J.: A real-time novelty detector for a mobile robot. In: EUREL European Advanced Robotics Systems Masterclass and Conference (2000)Google Scholar
  23. 23.
    Merceron, A.: Languages and Logic. Pearson Education, Australia (2001)Google Scholar
  24. 24.
    Norman, T.J., Long, D.: Goal creation in motivated agents. In: Intelligent agents: theories, architectures and languages. Springer, Heidelberg (1995)Google Scholar
  25. 25.
    Russel, J., Norvig, P.: Artificial intelligence: a modern approach. Prentice Hall Inc., Englewood Cliffs (1995)Google Scholar
  26. 26.
    Saunders, R., Gero, J.S.: Designing for Interest and Novelty: Motivating Design Agents. In: de Vries, B., van Leeuwen, J., Achten, H. (eds.) CAADFutures 2001, pp. 725–738. Kluwer, Dordrecht (2001)Google Scholar
  27. 27.
    Saunders, R., Gero, J.S.: Curious agents and situated design evaluations. In: Gero, J.S., Brazier, F.M.T. (eds.) Agents In Design: Key Centre of Design Computing and Cognition, pp. 133–149. University of Sydney, Sydney (2002)Google Scholar
  28. 28.
    Schmidhuber, J.: A possibility for implementing curiosity and boredom in model-building neural controllers. In: The International Conference on Simulation of Adaptive behaviour: From Animals to Animats (1991)Google Scholar
  29. 29.
    Schmill, M., Cohen, P.: A motivational system that drives the development of activity. In: AAAMAS. ACM, Bologna (2002)Google Scholar
  30. 30.
    Singh, S., Barto, A.G., Chentanez, N.: Intrinsically Motivated Reinforcement Learning (2004) (accessed, April 7, 2004),
  31. 31.
    Sloman, A., Croucher, M.: Why robots will have emotions. In: Proceedings of the 7th International Joint Conference on Artificial Intelligence, Vancouver (1981)Google Scholar
  32. 32.
    Stanley, J.C.: Computer simulation of a model of habituation. Nature 261, 146–148 (1976)CrossRefGoogle Scholar
  33. 33.
    Sutton, R.S., Barto, A.G.: Reinforcement learning: an introduction. MIT Press, Cambridge (2000)Google Scholar
  34. 34.
    Wang, Y., Huber, M., et al.: User-guided reinforcement learning of robot assistive tasks for an intelligent environment. In: Proceedings of the IEEE/RJS International Conference on Intelligent Robots and Systems. IEEE, Las Vegas (2003)Google Scholar
  35. 35.
    Watkins, C.: Learning from delayed rewards, PhD Thesis, Cambridge University (1989)Google Scholar
  36. 36.
    White, R.W.: Motivation reconsidered: The concept of competece. Phychological Review 66, 297–333 (1959)CrossRefGoogle Scholar

Copyright information

© Springer-Verlag Berlin Heidelberg 2006

Authors and Affiliations

  • Mary Lou Maher
    • 1
  • Kathryn Merrick
    • 2
  • Owen Macindoe
    • 1
  1. 1.Key Centre of Design Computing and CognitionUniversity of SydneySydneyAustralia
  2. 2.School of Information TechnologiesUniversity of Sydney and National ICT Australia 

Personalised recommendations