Skip to main content
Log in

Robotic Ubiquitous Cognitive Ecology for Smart Homes

  • Published:
Journal of Intelligent & Robotic Systems Aims and scope Submit manuscript

Abstract

Robotic ecologies are networks of heterogeneous robotic devices pervasively embedded in everyday environments, where they cooperate to perform complex tasks. While their potential makes them increasingly popular, one fundamental problem is how to make them both autonomous and adaptive, so as to reduce the amount of preparation, pre-programming and human supervision that they require in real world applications. The project RUBICON develops learning solutions which yield cheaper, adaptive and efficient coordination of robotic ecologies. The approach we pursue builds upon a unique combination of methods from cognitive robotics, machine learning, planning and agent-based control, and wireless sensor networks. This paper illustrates the innovations advanced by RUBICON in each of these fronts before describing how the resulting techniques have been integrated and applied to a proof of concept smart home scenario. The resulting system is able to provide useful services and pro-actively assist the users in their activities. RUBICON learns through an incremental and progressive approach driven by the feedback received from its own activities and from the user, while also self-organizing the manner in which it uses available sensors, actuators and other functional components in the process. This paper summarises some of the lessons learned by adopting such an approach and outlines promising directions for future work.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Similar content being viewed by others

References

  1. Cook, D.J., Holder, L.B.: Sensor selection to support practical use of health-monitoring smart environments. Wiley Int. Rev. Data Min. and Knowl. Disc. 1, 339–351 (2011)

    Article  Google Scholar 

  2. Mahmoud, S.M., Lotfi, A., Langensiepen, C.: Abnormal behaviours identification for an elder’s life activities using dissimilarity measurements. In: Proceedings of the 4th International Conference on PErvasive Technologies Related to Assistive Environments, PETRA ’11, (New York, NY, USA), pp. 25:1–25:5, ACM (2011)

  3. Gaddam, A., Mukhopadhyay, S., Sen Gupta, G.: Elder care based on cognitive sensor network. Sensors J. IEEE 11(3), 574–581 (2011)

    Article  Google Scholar 

  4. Lotfi, A., Langensiepen, C., Mahmoud, S.M., Akhlaghinia, M.: Smart homes for the elderly dementia sufferers: Identification and prediction of abnormal behaviour. J. Ambient Intell. Humanized Comput. 3(3), 205–218 (2012)

    Article  Google Scholar 

  5. Cesta, A., Cortellessa, G., Rasconi, R., Pecora, F., Scopelliti, M., Tiberio, L.: Monitoring elderly people with the robocare domestic environment: Interaction synthesis and user evaluation. Comput. Intell. 27(1), 60–82 (2011)

    Article  MathSciNet  Google Scholar 

  6. Dressler, F.: Self-organization in autonomous sensor/actuator networks. In: Proc. of the Int Conf on Architecture of Computing Systems (2006)

  7. Kim, J.-H., Kim, Y.-D., Lee, K.-H.: The Third Generation of Robotics: Ubiquitous Robot. In: Proceedings of the 2nd Int Conf on Autonomous Robots and Agents (2004)

  8. Broxvall, M., Seo, B., Kwon, W. The PEIS Kernel: A Middleware for Ubiquitous Robotics. In: Proceedings of the IROS-07 Workshop on Ubiquitous Robotic Space Design and Applications. San Diego, California (2007)

  9. Saffiotti, A., Broxvall, M.: PEIS Ecologies: Ambient Intelligence Meets Autonomous Robotics. In: Proceedings of the 2005 Joint Conference on Smart Objects and Ambient Intelligence: Innovative Context-Aware Services: Usages and Technologies, pp. 277–281, ACM (2005)

  10. Broxvall, M.: A Middleware for Ecologies of Robotic Devices. In: Proceedings of the 1st international conference on Robot communication and coordination, p. 30, IEEE press (2007)

  11. Amato, G., Broxvall, M., Chessa, S., Dragone, M., Gennaro, C., López, R., Maguire, L., Mcginnity, M., Micheli, A., Renteria, A., O’Hare, G., Pecora, F.: Robotic ubiquitous cognitive network. In: Ambient Intelligence - Software and Applications, vol. 153 of Advances in Intelligent and Soft Computing, pp. 191–195, Springer Berlin Heidelberg (2012)

  12. Lundh, R: Plan-based configuration of a group of robots. PhD thesis, Örebro University (2006)

  13. Lundh, R., Karlsson, L., Saffiotti, A.: Plan-based Configuration of an Ecology of Robots. In: Robotics and Automation, 2007 IEEE International Conference on, pp. 64–70, IEEE (2007)

  14. Gritti, A.S.M., Broxvall, M.: Reactive Self-configuration of an Ecology of Robots. Rome, Italy (2007)

  15. Alam, M., Reaz, M., Mohd Ali, M.: Speed: An inhabitant activity prediction algorithm for smart homes. IEEE Trans. Syst. Man Cybern. Syst. Hum. 42(4), 985–990 (2012)

    Article  Google Scholar 

  16. Puteh, S., Langensiepen, C., Lotfi, A.: Fuzzy ambient intelligence for intelligent office environments. In: Fuzzy Systems (FUZZ-IEEE), 2012 IEEE International Conference on, pp. 1–6 (2012)

  17. Puteh, S., Lotfi, A., Langensiepen, C.S.: Activities recognition in intelligent office environment. In: Bota, J.A., Charitos, D. (eds.) Intelligent Environments (Workshops) (2013)

  18. Liming, C., Nugent, C., Hui, W.: A knowledge-driven approach to activity recognition in smart homes. IEEE Trans. Knowl. Data Eng. 24(6), 961–974 (2012)

    Article  Google Scholar 

  19. Zhang, S., McClean, S., Scotney, B.: Probabilistic learning from incomplete data for recognition of activities of daily living in smart homes. IEEE Trans. Inf. Technol. Biomed. 16(3), 454–462 (2012)

    Article  Google Scholar 

  20. Roggen, D., Forster, K., Calatroni, A., Troster, G.: The adarc pattern analysis architecture for adaptive human activity recognition systems. J. Ambient Intell. Humanized Comput. 4(2), 169–186 (2013)

    Article  Google Scholar 

  21. De, D., Tang, S., Song, W.-Z., Cook, D., Das, S.K.: Activity-aware sensor network in smart environments. Pervas. Mobile Comput. 8(5), 730–750 (2012)

    Article  Google Scholar 

  22. Kurz, M., Gerold, H., Ferscha, A., Calatroni, A., Roggen, D., Trster, G., Sagha, H., Chavarriaga, R., Milln, J.d.R., Bannach, D., Kunze, K., Lukowicz, P.: The OPPORTUNITY Framework and Data Processing Ecosystem for Opportunistic Activity and Context Recognition. Interna. J. Sensors, Wirel. Communicat. Cont. 1(2), 102–125 (2012)

    Article  Google Scholar 

  23. Roggen, D., Calatroni, A., Frster, K, Trster, G., Lukowicz, P., Bannach, D., Ferscha, A., Kurz, M., Hlzl, G., Sagha, H., Bayati, H., Milln, J.d.R., Chavarriaga, R. Activity recognition in opportunistic sensor environments. Procedia Computer Science, vol. 7, pp. 173–174 (2011). Proceedings of the 2nd European Future Technologies Conference and Exhibition 2011 (FET 11)

  24. Jaeger, H.: Ksera project: Deliverable d4.1 learning & decision making algorithms in pervasive environments. tech. rep. (2010)

  25. Hongmei, H., Zhenhuan, Z., Makinen, E.: A neural network model to minimize the connected dominating set for self-configuration of wireless sensor networks. IEEE Transact. Neural Networks 20, 973–982 (2009)

    Article  Google Scholar 

  26. Li, Y., Parker, L.: Detecting and monitoring time-related abnormal events using a wireless sensor network and mobile robot. In: IEEE/RSJ Int. Conf. on Intel. Robots and Systems, 2008, pp. 3292–3298 (2008)

  27. Nakano, H., Utani, A., Miyauchi, A., Yamamoto, H.: Synchronization-based data gathering scheme using chaotic pulse-coupled neural networks in wireless sensor networks. In: Proceedings of the IJCNN 2008, pp. 1115–1121 (2008)

  28. Sun, Y., Li, L.: Hybrid learning algorithm for effective coverage in wireless sensor networks. In: Proc. of the ICNC 2008, vol. 5, pp. 227–231 (2008)

  29. Kulkarni, R., Forster, A., Venayagamoorthy, G.: Computational intelligence in wireless sensor networks: A survey. IEEE Commun. Surv. Tutorials 13, 68–96 (2011)

    Article  Google Scholar 

  30. Kolen, J., Kremer, S. (eds.): A Field Guide to Dynamical Recurrent Networks. IEEE Press (2001)

  31. Moustapha, A., Selmic, R.: Wireless sensor network modeling using modified recurrent neural networks: Application to fault detection. IEEE Trans. Instrum. Meas. 57, 981–988 (2008)

    Article  Google Scholar 

  32. Lukosevicius, M., Jaeger, H.: Reservoir computing approaches to recurrent neural network training. Computer Science Review 3(3), 127–149 (2009)

    Article  MATH  Google Scholar 

  33. Verstraeten, D., Schrauwen, B., D’Haene, M., Stroobandt, D.: An experimental unification of reservoir computing methods. Neural Netw. 20(3), 391–403 (2007)

    Article  MATH  Google Scholar 

  34. Gallicchio, C., Micheli, A.: Architectural and markovian factors of echo state networks. Neural Netw. 24(5), 440–456 (2011)

    Article  Google Scholar 

  35. Jaeger, H., Haas, H.: Harnessing nonlinearity: Predicting chaotic systems and saving energy in wirelesscommunication. Science 304(5667), 78–80 (2004)

    Article  Google Scholar 

  36. Jaeger, H.: The echo state approach to analysing and training recurrent neural networks. tech. rep., GMD - German National Research Institute for Computer Science (2001)

  37. Bacciu, D., Barsocchi, P., Chessa, S., Gallicchio, C., Micheli, A.: An experimental characterization of reservoir computing in ambient assisted living applications. Neural Comput. Applic. 24(6), 1451–1464 (2014)

    Article  Google Scholar 

  38. Bacciu, D., Chessa, S., Gallicchio, C., Micheli, A., Barsocchi, P.: An experimental evaluation of reservoir computation for ambient assisted living. In: Apolloni, B., Bassis, S., Esposito, A., Morabito, F.C. (eds.) Neural Nets and Surroundings, vol. 19 of Smart Innovation, Systems and Technologies, pp. 41–50, Springer Berlin Heidelberg (2013)

  39. Bacciu, D., Gallicchio, C., Micheli, A., Chessa, S., Barsocchi, P.: Predicting user movements in heterogeneous indoor environments by reservoir computing. In: Bhatt, M., Guesgen, H.W., Augusto, J.C. (eds.) Proceedings of the IJCAI Workshop on Space, Time and Ambient Intelligence (STAMI) 2011, pp. 1–6 (2011)

  40. Gallicchio, C., Micheli, A., Barsocchi, P., Chessa, S.: User movements forecasting by reservoir computing using signal streams produced by mote-class sensors. In: Mobile Lightweight Wireless Systems (Mobilight 2011), vol. 81 of Lecture Notes of the Institute for Computer Sciences, Social Informatics and Telecommunications Engineering, pp. 151– 168, Springer Berlin Heidelberg (2012)

  41. Palumbo, F., Barsocchi, P., Gallicchio, C., Chessa, S., Micheli, A.: Multisensor data fusion for activity recognition based on reservoir computing. In: Botía, J., Alvarez-Garcia, J., Fujinami, K., Barsocchi, P., Riedel, T. (eds.) Evaluating AAL Systems Through Competitive Benchmarking vol. 386 of Communications in Computer and Information Science, pp. 24–35, Springer Berlin Heidelberg (2013)

  42. Knight, S., Rabideau, G., Chien, S., Engelhardt, B., Sherwood, R.: Casper: Space exploration through continuous planning. Intell. Syst. 16(5), 70–75 (2001)

    Google Scholar 

  43. Do, M.B., Kambhampati, S.: Sapa: A multi-objective metric temporal planner. J. Artif. Intell. Res. (JAIR) 20, 155–194 (2003)

    MATH  Google Scholar 

  44. Gerevini, A., Saetti, A., Serina, I.: An approach to temporal planning and scheduling in domains with predictable exogenous events. J. Artif. Int. Res. 25, 187–231 (2006)

    MATH  Google Scholar 

  45. Doherty, P., Kvarnström, J., Heintz, F.: A temporal logic-based planning and execution monitoring framework for unmanned aircraft systems. Auton. Agent. Multi-Agent Syst. 19(3), 332–377 (2009)

    Article  Google Scholar 

  46. Barreiro, J., Boyce, M., Frank, J., Iatauro, M., Kichkaylo, T., Morris, P., Smith, T., Do, M.: EUROPA: A platform for AI planning. In: Proc of ICAPS-ICKEPS (2012)

  47. Eyerich, P., Mattmüller, R., Röger, G.: Using the context-enhanced additive heuristic for temporal and numeric planning. In: Proc. of the 19th Int. Conf. on Automated Planning and Scheduling (ICAPS) (2009)

  48. Finzi, A., Ingrand, F., Muscettola, N.: Model-based executive control through reactive planning for autonomous rovers. In: Proc. of the IEEE/RSJ Int. Conf. on Intelligent Robots and Systems (IROS) (2004)

  49. McGann, C., Py, F., Rajan, K., Thomas, H., Henthorn, R., McEwen, R.: A Deliberative Architecture for AUV Control. In: Proc. of the Int. Conf. on Robotics and Automation (ICRA), (Pasadena) (May 2008)

  50. McGann, C., Py, F., Rajan, K., Ryan, J.P., Henthorn, R.: Adaptive Control for Autonomous Underwater Vehicles. In: Proc. of the 23rd AAAI Conference on Artificial Intelligence, (Chicago, IL) (2008)

  51. Köckemann, U., Pecora, F., Karlsson, L.: Towards planning with very expressive languages via problem decomposition into multiple csps. In: Proc. of the ICAPS Workshop on Constraint Satisfaction Techniques for Planning and Scheduling Problems (COPLAS) (2012)

  52. Fratini, S., Pecora, F., Cesta, A.: Unifying planning and scheduling as timelines in a component-based perspective. Archives of Control Sciences 18(2), 231–271 (2008)

    MathSciNet  MATH  Google Scholar 

  53. Ghallab, M., Laruelle, H.: Representation and control in IxTeT, a temporal planner. In: AIPS, pp. 61–67 (1994)

  54. Lemai, S., Ingrand, F.: Interleaving temporal planning and execution in robotics domains. In: Proceedings of the 19th national conference on Artifical intelligence, AAAI’04, pp. 617–622, AAAI Press (2004)

  55. Di Rocco, M., Pecora, F., Kumar, P., Saffiotti, A.: Configuration planning with multiple dynamic goals. In: Proc. of AAAI Spring Symposium on Designing Intelligent Robots (2013)

  56. Rashidi, P.: The resident in the loop: Adapting the smart home to the user, vol. 39, p. 949959. IEEE Transactions on Systems, Man, and Cybernetics journal, Part A. (2009)

  57. Tapia, E., Intille, S., Larson, K.: Activity recognition in the home using simple and ubiquitous sensors. Pervasive Computing, p. 158175 (2004)

  58. Watkins, C.: Learning from delayed rewards. PhD thesis. University of Cambridge, England (1989)

    Google Scholar 

  59. Eunju, K., Sumi, H., Cook, D.: Human activity recognition and pattern discovery. In: Neural Computing and Applications, vol. 9 of Pervasive Computing, pp. 48–53, IEEE (2010)

  60. Rashidi, P., Cook, D., Holder, L.B., Schmitter-Edgecombe, M.: Discovering activities to recognize and track in a smart environment. IEEE Trans. Knowl. Data Eng. 23(4), 527–539 (2011)

    Article  Google Scholar 

  61. Cook, D., Krishnan, N.C., Rashidi, P.: Activity discovery and activity recognition: A new partnership. IEEE T. Cybernetics 43(3), 820–828 (2013)

    Article  Google Scholar 

  62. Duch, W., Oentaryo, R.J., Pasquier, M.: Cognitive architectures: where do we go from here. Frontiers in Artificial Intelligence and Applications 171, 122–136 (2008)

    Google Scholar 

  63. Vernon, D., Metta, G., Sandini, G.: A survey of artificial cognitive systems: implications for the autonomous development of mental capabilities in computational agents. IEEE Transactions on Evolutionary Computation 11(2), 151–180 (2007)

    Article  Google Scholar 

  64. Langley, P., Laird, J.E., Rogers, S.: Cognitive architectures: research issues and challenges. Cogn. Syst. Res. 10, 141–160 (2009)

    Article  Google Scholar 

  65. Laird, J.E.: Extending the soar cognitive architecture. In: Proceedings of the Artificial General Intelligence Conference, (Memphis) (2008)

  66. Laird, J.E.: The soar cognitive architecture. Artificial Intelligence and Simulation of Behaviour Quarterly, vol. 134 (2012)

  67. Langley, P., Choi, D.: A unified cognitive architecture for physical agents. In: Proceedings of the twenty-first aaai conference on artificial intelligence, (Boston) (2006)

  68. Langley, P., Choi, D.: Learning recursive control programs from problem solving. J. Mach. Learn. Res. 7, 493–518 (2006)

    MathSciNet  MATH  Google Scholar 

  69. O’Reilly, R., Braver, T., Cohen, J.: A biologically-based computational model of working memory. In: Miyake, A., Shah, P. (eds.) Models of Working Memory, pp. 375–411. Cambridge University Press (1999)

  70. Edelman, G.: Neural darwinism: Selection and reentrant signaling in higher brain function. Neuron 10(2), 115–125 (1993)

    Article  MathSciNet  Google Scholar 

  71. Anderson, J.R., Bothell, D., Byrne, M.D., Douglass, S., Lebiere, C., Qin, Y.: An integrated theory of the mind. Psychol. Rev. 111(4), 1036–1060 (2004)

    Article  Google Scholar 

  72. Peebles, D., Banks, A.: Modelling dynamic decision making with the act-r cognitive architecture. Journal of Artificial General Intelligence 2(5), 52–68 (2010)

    Google Scholar 

  73. Sun, R.: The importance of cognitive architectures: An analysis based on clarion. Journal of Experimental and Theoretical Artificial Intelligence 19, 159–193 (2007)

    Article  Google Scholar 

  74. Sun, R.: Motivational representations within a computational cognitive architecture. Cognitive Computation 1(1), 91–103 (2009)

    Article  Google Scholar 

  75. Ray, A.K., Leng, G., McGinnity, T.: Development of cognitive capabilities for smart home using a self-organizing fuzzy neural network. In: 10th IFAC Symposium on Robot Contro 10, 447–454 (2012). Dubrovnik, Croatia

    Google Scholar 

  76. Ray, A.K., Leng, G., McGinnity, T., Coleman, S., Maguire, L.: Development of a self sustaining cognitive architecture. Biologically Inspired Cognitive Architecture, Elsevier 6, 96–108 (2013)

    Article  Google Scholar 

  77. Leng, G., McGinnity, T.M., Prasad, G.: Design for self-organizing fuzzy neural networks based on genetic algorithms. IEEE Transactions on Fuzzy Systems 14(6), 755–766 (2006)

    Article  Google Scholar 

  78. Amato, G., Broxvall, M., Chessa, S., Dragone, M., Gennaro, C., Vairo, C.: When wireless sensor networks meet robots. In: Proceedings of the Seventh International Conference on Systems and Networks Communications (ICSNC 2012), pp. 35–40, ThinkMind (TM) Digital Library, Lisbon, Portugal (2012)

  79. Bacciu, D., Barsocchi, P., Chessa, S., Gallicchio, C., Micheli, A.: An experimental characterization of reservoir computing in ambient assisted living applications. Neural Comput. & Applic. 24(6), 1451–1464 (2014)

    Article  Google Scholar 

  80. Bacciu, D., Di Rocco, M., Gallicchio, C., Micheli, A., Saffiotti, A.: Learning context-aware mobile robot navigation in home environments. In: To Appear in the Proceedings of the 5th International Conference on Information, Intelligence, Systems and Applications (IISA 2014), 7-9 July 2014, Chania, Greece IEEE (2014)

  81. Gay, D., Levis, P., von Behren, R., Welsh, M., Brewer, E., Culler, D.: The nesc language: A holistic approach to networked embedded systems. In: Proceedings of the ACM SIGPLAN 2003 conference on Programming language design and implementation, PLDI ’03, pp. 1–11, ACM (2003)

  82. Ray, A.K., Leng, G., McGinnity, T., Coleman, S., Maguire, L.: Dynamically reconfigurable online self-organising fuzzy neural network with variable number of inputs for smart home. In: NCTA 2013 - International Joint Conference on Computational Intelligence. Algarve, Portugal (2013)

    Google Scholar 

  83. Leng, G., Ray, A.K., McGinnity, T., Coleman, S., Maguire, L.: Online sliding window based self-organising fuzzy neural network for cognitive reasoning. in COGNITIVE 2013, The Fifth International Conference on Advanced Cognitive Technologies and Applications, (Valencia, Spain), pp. 114–119, IARIA (2013)

  84. Dragone, M., Abdel-Naby, S., Swords, D., O’Hare, G., Broxvall, M.: A programming framework for multi-agent coordination of robotic ecologies. In: ProMAS 2012, 72–89 (2012)

  85. Dragone, M.: Building self-adaptive software systems with component, services & agents technologies: Self-osgi. In: 4th International Conference, ICAART 2012, Vilamoura, Portugal, February 6-8, 2012. Agents and Artificial Intelligence Communications in Computer and Information Science, 300–316 (2012)

  86. Di Rocco, M., Pecora, F., Saffiotti, A.: When robots are late: Configuration planning for multiple robots with dynamic goals, in IROS (2013)

  87. Di Rocco, M., Pecora, F., Saffiotti, A.: Closed loop configuration planning with time and resources. In: Proceedings of the ICAPS 2013 Workshop on Planning and Robotics (2013)

  88. Robotic ubiquitous cognitive network (rubicon) website (2014)

  89. Amato, G., Chessa, S., Gennaro, C., Pallini, D., Vairo, C.: A data logger forwireless sensor network. In: Proceedings of 2nd International Conference On Sensor Networks (Sensornets), (PRT), pp. 65–68, SciTePress - Science and Technology Publications (2013)

  90. Sokolova, M., Lapalme, G.: A systematic analysis of performance measures for classification tasks. Inform. Process. Management 45(4), 427–437 (2009)

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to M. Dragone.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Amato, G., Bacciu, D., Broxvall, M. et al. Robotic Ubiquitous Cognitive Ecology for Smart Homes. J Intell Robot Syst 80 (Suppl 1), 57–81 (2015). https://doi.org/10.1007/s10846-015-0178-2

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10846-015-0178-2

Keywords

Navigation