Abstract
Activity recognition for shepherding is a way for an artificial intelligence system to learn and understand shepherding behaviours. The problem we describe is one of recognising behaviours within a shepherding environment, where a cognitive agent (the shepherd) influences agents within the system (sheep) through a shepherding actuator (sheepdog), to achieve an intent. Shepherding is pervasive in everyday life with AI agents, collections of animals, and humans all partaking in different forms. Activity recognition in this context is the generation of a transformation from sensor stream data to the perceived behaviour of an agent under observation from the perspective of an external observer. We present a method of classifying behaviour through the use of spatial data and codify action, behaviour, and intent states through a multi-level classification mapping process.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
References
Allen, J.F.: Maintaining knowledge about temporal intervals. Commun. ACM 26(11), 832–843 (1983). https://doi.org/10.1145/182.358434
Anderson, S., Bredeche, N., Eiben, A., Kampis, G., van Steen, M.: Adaptive Collective Systems: Herding Black Sheep. Fundamentals of Collective Adaptive Systems (2013)
Aroor, A., Epstein, S.L., Korpan, R.: Online learning for crowd-sensitive path planning. In: Proceedings of the 17th International Conference on Autonomous Agents and MultiAgent Systems (2018)
Artikis, A., Sergot, M., Paliouras, G.: An event calculus for event recognition. IEEE Trans. Knowl. Data Eng. 27(4), 895–908 (2015). https://doi.org/10.1109/TKDE.2014.2356476
Azkune, G.: Learning for dynamic and personalised knowledge-based activity models. Ph.D. Thesis, Universidad de Deusto (2015)
Bakar, U.A.B.U.A., Ghayvat, H., Hasanm, S.F., Mukhopadhyay, S.C.: Activity and Anomaly Detection in Smart Home: A Survey, pp. 191–220. Springer International Publishing, Cham (2016). https://doi.org/10.1007/978-3-319-21671-3_9
Baumann, M., Büning, H.K.: Learning shepherding behavior. Ph.D. Thesis, University of Paderborn (2016)
Bettini, C., Brdiczka, O., Henricksen, K., Indulska, J., Nicklas, D., Ranganathan, A., Riboni, D.: A survey of context modelling and reasoning techniques. Pervasive Mob. Comput. 6(2), 161–180 (2010). https://doi.org/10.1016/j.pmcj.2009.06.002
Bikakis, A., Antoniou, G., Hasapis, P.: Strategies for contextual reasoning with conflicts in ambient intelligence. Knowl. Inf. Syst. 27(1), 45–84 (2011). https://doi.org/10.1007/s10115-010-0293-0
Breiman, L.: Random forests. Mach. Learn.45(1), 5–32 (2001). https://doi.org/10.1023/A:1010933404324
Bruno, B., Mastrogiovanni, F., Saffiotti, A., Sgorbissa, A.: Using fuzzy logic to enhance classification of human motion primitives. In: Laurent, A., Strauss, O., Bouchon-Meunier, B., Yager, R.R. (eds.) Information Processing and Management of Uncertainty in Knowledge-Based Systems, pp. 596–605. Springer International Publishing, Cham (2014)
Chen, L., Khalil, I.: Activity Recognition: Approaches, Practices and Trends, pp. 1–31. Atlantis Press, Paris (2011). https://doi.org/10.2991/978-94-91216-05-3_1
Chen, L., Hoey, J., Nugent, C.D., Cook, D.J., Yu, Z.: Sensor-based activity recognition. IEEE Trans. Syst. Man Cybern. Part C (Appl. Rev.) 42(6), 790–808 (2012). https://doi.org/10.1109/TSMCC.2012.2198883
Chen, L., Nugent, C.D., Wang, H.: A knowledge-driven approach to activity recognition in smart homes. IEEE Trans. Knowl. Data Eng. 24(6), 961–974 (2012). https://doi.org/10.1109/TKDE.2011.51
Chen, L., Nugent, C., Okeyo, G.: An ontology-based hybrid approach to activity modeling for smart homes. IEEE Trans. Human-Machine Syst. 44(1), 92–105 (2014). https://doi.org/10.1109/THMS.2013.2293714
Cicirelli, F., Fortino, G., Giordano, A., Guerrieri, A., Spezzano, G., Vinci, A.: On the design of smart homes: a framework for activity recognition in home environment. J. Medical Syst. 40(9), 200 (2016). https://doi.org/10.1007/s10916-016-0549-7
Cook, D.J., Crandall, A.S., Thomas, B.L., Krishnan, N.C.: Casas: A smart home in a box. Computer 46(7), 62–69 (2013). https://doi.org/10.1109/MC.2012.328
Cowling, P., Gmeinwieser, C.: Ai for herding sheep. In: Proceedings of the Sixth AAAI Conference on Artificial Intelligence and Interactive Digital Entertainment, AIIDE’10, pp. 2–7. AAAI Press, California (2010). http://dl.acm.org/citation.cfm?id=3014666.3014668
Craig I. Schlenoff Sebti Foufou, S.B.B.: An approach to ontology-based intention recognition using state representations. In: 4th International Conference on Knowledge Engineering and Ontology Development (KEOD 2012) (2014)
Crispim-Junior, C.F., Buso, V., Avgerinakis, K., Meditskos, G., Briassouli, A., Benois-Pineau, J., Kompatsiaris, I.Y., Bremond, F.: Semantic event fusion of different visual modality concepts for activity recognition. IEEE Trans. Pattern Anal. Mach. Intell. 38(8), 1598–1611 (2016). https://doi.org/10.1109/TPAMI.2016.2537323
de Carvalho, A.C.P.L.F., Freitas, A.A.: A Tutorial on Multi-label Classification Techniques, pp. 177–195. Springer, Berlin (2009). https://doi.org/10.1007/978-3-642-01536-6_8
Eiter, T., Ianni, G., Krennwallner, T., Polleres, A.: Reasoning Web. Springer, Berlin (2008). https://doi.org/10.1007/978-3-540-85658-0_1
Ferber, J.: Multi-Agent Systems: An Introduction to Distributed Artificial Intelligence, 1st edn. Addison-Wesley Longman, Boston (1999)
Fleury, A., Vacher, M., Noury, N.: SVM-based multimodal classification of activities of daily living in health smart homes: sensors, algorithms, and first experimental results. IEEE Trans. Inf. Technol. Biomedicine 14(2), 274–283 (2010). https://doi.org/10.1109/TITB.2009.2037317
Guo, K., Li, Y., Lu, Y.: An alternative-service recommending algorithm based on semantic similarity. China Commun. 14(8), 124–136 (2017). https://doi.org/10.1109/CC.2017.8014353
Hamilton, W.D.: Geometry for the selfish herd. J. Theoret. Biol. 31(2), 295–311 (1971)
Hasan, M., Roy-Chowdhury, A.K.: Context aware active learning of activity recognition models. In: 2015 IEEE International Conference on Computer Vision (ICCV), pp. 4543–4551 (2015). https://doi.org/10.1109/ICCV.2015.516
He, X.: Swarm robotics: The future of medicine? (2015). https://medtechboston.medstro.com/blog/2015/10/06/swarm-robotics-what-you-need-to-know-about-the-future-of-medicine/
Helaoui, R., Niepert, M., Stuckenschmidt, H.: Recognizing interleaved and concurrent activities using qualitative and quantitative temporal relationships. Pervasive Mob. Comput. 7(6), 660–670 (2011). https://doi.org/10.1016/j.pmcj.2011.08.004
Helaoui, R., Riboni, D., Stuckenschmidt, H.: A probabilistic ontological framework for the recognition of multilevel human activities. In: Proceedings of the 2013 ACM International Joint Conference on Pervasive and Ubiquitous Computing, UbiComp ’13, pp. 345–354. ACM, New York (2013). https://doi.org/10.1145/2493432.2493501
Ikizler, N., Forsyth, D.: Searching video for complex activities with finite state models. In: 2007 IEEE Conference on Computer Vision and Pattern Recognition, pp. 1–8 (2007). https://doi.org/10.1109/CVPR.2007.383168
Ilyas, A., Engstrom, L., Athalye, A., Lin, J.: Proceedings of the 35th International Conference on Machine Learning, PMLR 80, 2137–2146, (2018)
Incel, O., Kose, M., Ersoy, C.: A review and taxonomy of activity recognition on mobile phones. BioNanoScience 3(2), 145–171 (2013). https://doi.org/10.1007/s12668-013-0088-3
Iqbal, M., Pao, H.K.: Activity recognition from minimal distinguishing subsequence mining. AIP Confer. Proc. 1867(1), 020,046 (2017). https://doi.org/10.1063/1.4994449. https://aip.scitation.org/doi/abs/10.1063/1.4994449
Iwamoto, S., Ohmura, R.: Towards concurrent task verification in context-aware applications. In: Adjunct Proceedings of the 2015 ACM International Joint Conference on Pervasive and Ubiquitous Computing and Proceedings of the 2015 ACM International Symposium on Wearable Computers, UbiComp/ISWC’15 Adjunct, pp. 1473–1477. ACM, New York (2015). https://doi.org/10.1145/2800835.2801618
Kishore, S., Bhattacharjee, S., Swetapadma, A.: A hybrid method for activity monitoring using principal component analysis and back-propagation neural network. In: 2017 International Conference On Smart Technologies For Smart Nation (SmartTechCon), pp. 885–889 (2017). https://doi.org/10.1109/SmartTechCon.2017.8358499
Kleiminger, W., Mattern, F., Santini, S.: Predicting household occupancy for smart heating control: a comparative performance analysis of state-of-the-art approaches. Energy Build. 85, 493–505 (2014). https://doi.org/https://doi.org/10.1016/j.enbuild.2014.09.046. http://www.sciencedirect.com/science/article/pii/S037877881400783X
Kuankid, S., Rattanawong, T., Aurasopon, A.: Classification of the cattle’s behaviors by using accelerometer data with simple behavioral technique. In: Signal and Information Processing Association Annual Summit and Conference (APSIPA), 2014 Asia-Pacific, pp. 1–4 (2014)
Kumar, R.C., Bharadwaj, S.S., Sumukha, B.N., George, K.: Human activity recognition in cognitive environments using sequential ELM. In: 2016 Second International Conference on Cognitive Computing and Information Processing (CCIP), pp. 1–6 (2016). https://doi.org/10.1109/CCIP.2016.7802880
Kunze, K.: Real-life activity recognition – focus on recognizing reading activities. In: Iwamura, M., Shafait, F. (eds.) Camera-Based Document Analysis and Recognition, pp. 179–185. Springer International Publishing, Cham (2014)
Lasecki, W.S., Marcus, A., Rzeszotarski, J.M., Bigham, J.P.: Using microtask continuity to improve crowdsourcing. Technical Report (2014)
Lee, W., Kim, D.: Autonomous shepherding behaviors of multiple target steering robots. Sensors 17(12), 2729 (2017)
Lettmann, T., Baumann, M., Eberling, M., Kemmerich, T.: Modeling Agents and Agent Systems, pp. 157–181. Springer, Berlin (2011). https://doi.org/10.1007/978-3-642-24016-4_9
Li, F., Dustdar, S.: Incorporating unsupervised learning in activity recognition. In: Proceedings of the 4th AAAI Conference on Activity Context Representation: Techniques and Languages, AAAIWS’11-04, pp. 38–41. AAAI Press, Cambridge (2011). http://dl.acm.org/citation.cfm?id=2908613.2908620
Li, M., Hu, Z., Liang, J., Li, S.: Shepherding behaviors with single shepherd in crowd management. In: Xiao, T., Zhang, L., Ma, S. (eds.) System Simulation and Scientific Computing, pp. 415–423. Springer, Berlin (2012)
Liao, L., Fox, D., Kautz, H.: Hierarchical conditional random fields for GPS-based activity recognition. In: Proceedings of the International Symposium of Robotis Research (ISRR 2005). Springer, Berlin (2005)
Licitra, R.A., Hutcheson, Z.D., Doucette, E.A., Dixon, W.E.: Single agent herding of n-agents: a switched systems approach. IFAC-PapersOnLine 50(1), 14374–14379 (2017). 20th IFAC World Congress. https://doi.org/10.1016/j.ifacol.2017.08.2020. http://www.sciencedirect.com/science/article/pii/S2405896317326587
Lien, J.M., Pratt, E.: Interactive planning for shepherd motion. In: AAAI Spring Symposium: Agents that Learn from Human Teachers (2009)
Lien, J.M., Bayazit, O.B., Sowell, R.T., Rodriguez, S., Amato, N.M.: Shepherding behaviors. In: IEEE International Conference on Robotics and Automation, vol. 4, pp. 4159–4164. Citeseer (2004)
Liu, Y., Nie, L., Liu, L., Rosenblum, D.S.: From action to activity. Neurocomputing 181(C), 108–115 (2016). https://doi.org/10.1016/j.neucom.2015.08.096
Loke, S.W.: Representing and reasoning with situations for context-aware pervasive computing: a logic programming perspective. Knowl. Eng. Rev. 19(3), 213–233 (2004). https://doi.org/10.1017/S0269888905000263
Luz, G., Barros, K., Arajo, F.V., Barbosa da Silva, G., Augusto Ferreira da Silva, P., Condori, R., Brasil, L.: Nanorobotics in drug delivery systems for treatment of cancer: a review. J. Materials Sci. Eng. A 6 (2016). https://doi.org/10.17265/2161-6213/2016.5-6.005
McVicar, K.E.: C3: The challenge of change. IEEE Trans. Aerosp. Electron. Syst. AES-20(4), 401–413 (1984). https://doi.org/10.1109/TAES.1984.4502061
Mould, N., Regens, J.L., III, C.J.J., Edger, D.N.: Video surveillance and counterterrorism: the application of suspicious activity recognition in visual surveillance systems to counterterrorism. J. Polic. Intell. Count. Terror. 9(2), 151–175 (2014). https://doi.org/10.1080/18335330.2014.940819.
Nadimi, E.S., Jørgensen, R.N., Blanes-Vidal, V., Christensen, S.: Monitoring and classifying animal behavior using ZigBee-based mobile ad hoc wireless sensor networks and artificial neural networks. Comput. Electron. Agric. 82, 44–54 (2012). https://doi.org/10.1016/j.compag.2011.12.008
Okeyo, G., Chen, L., Wang, H., Sterritt, R.: Ontology-enabled activity learning and model evolution in smart homes. In: Yu, Z., Liscano, R., Chen, G., Zhang, D., Zhou, X. (eds.) Ubiquitous Intelligence and Computing, pp. 67–82. Springer, Berlin (2010)
Okeyo, G., Chen, L., Wang, H., Sterritt, R.: Ontology-Based Learning Framework for Activity Assistance in an Adaptive Smart Home, pp. 237–263. Atlantis Press, Paris (2011). https://doi.org/10.2991/978-94-91216-05-3_11
Okeyo, G., Chen, L., Wang, H., Sterritt, R.: A hybrid ontological and temporal approach for composite activity modelling. In: 2012 IEEE 11th International Conference on Trust, Security and Privacy in Computing and Communications, pp. 1763–1770 (2012). https://doi.org/10.1109/TrustCom.2012.34
Okeyo, G.O., Chen, L., Wang, H.: An agent-mediated ontology-based approach for composite activity recognition in smart homes. J. UCS 19, 2577–2597 (2013)
Okeyo, G., Chen, L., Wang, H., Sterritt, R.: Dynamic sensor data segmentation for real-time knowledge-driven activity recognition. Pervasive Mob. Comput. 10, 155–172 (2014). https://doi.org/10.1016/j.pmcj.2012.11.004
Paranjape, A.A., Chung, S.J., Kim, K., Shim, D.H.: Robotic herding of a flock of birds using an unmanned aerial vehicle. IEEE Trans. Rob. 34(4), 901–915 (2018)
Parker, L.: Multiple Mobile Robot Systems, pp. 921–941. Springer, Berlin (2008). https://doi.org/10.1007/978-3-540-30301-5_41
Quero, J., Orr, C., Zang, S., Nugent, C., Salguero, A., Espinilla, M.: Real-time recognition of interleaved activities based on ensemble classifier of long short-term memory with fuzzy temporal windows. Proceedings 2(19), 1225 (2018). https://doi.org/10.3390/proceedings2191225
Ranasinghe, S., Machot, F.A., Mayr, H.C.: A review on applications of activity recognition systems with regard to performance and evaluation. Int. J. Distrib. Sens. Netw. 12(8), 1550147716665520 (2016). https://doi.org/10.1177/1550147716665520
Rashidi, P., Cook, D.J.: COM: A method for mining and monitoring human activity patterns in home-based health monitoring systems. ACM Trans. Intell. Syst. Technol. 4(4), 64:1–64:20 (2013). https://doi.org/10.1145/2508037.2508045
Rashidi, P., Cook, D.J., Holder, L.B., Schmitter-Edgecombe, M.: Discovering activities to recognize and track in a smart environment. IEEE Trans. Knowl. Data Eng. 23(4), 527–539 (2011). https://doi.org/10.1109/TKDE.2010.148
Riboni, D., Bettini, C.: Context-aware activity recognition through a combination of ontological and statistical reasoning. In: Zhang, D., Portmann, M., Tan, A.H., Indulska, J. (eds.) Ubiquitous Intelligence and Computing, pp. 39–53. Springer, Berlin (2009)
Riboni, D., Pareschi, L., Radaelli, L., Bettini, C.: Is ontology-based activity recognition really effective? In: 2011 IEEE International Conference on Pervasive Computing and Communications Workshops (PERCOM Workshops), pp. 427–431 (2011). https://doi.org/10.1109/PERCOMW.2011.5766927
Roggen, D., Tröster, G., Lukowicz, P., Ferscha, A., del R. Millán, J., Chavarriaga, R.: Opportunistic human activity and context recognition. Computer 46(2), 36–45 (2013). https://doi.org/10.1109/MC.2012.393
Sadri, F.: Logic-based approaches to intention recognition. In: Handbook of Research on Ambient Intelligence and Smart Environments: Trends and Perspectives. Citseer (2009)
Saguna, S., Zaslavsky, A., Chakraborty, D.: Complex activity recognition using context-driven activity theory and activity signatures. ACM Trans. Comput.-Hum. Interact. 20(6), 32:1–32:34 (2013). https://doi.org/10.1145/2490832
Salah, A., Oudeyer, P., c li, C.M., del Solar, J.R.: Guest editorial behavior understanding and developmental robotics. IEEE Trans. Auton. Ment. Dev. 6(2), 77–79 (2014). https://doi.org/10.1109/TAMD.2014.2328731
Sato, K., Fujinami, K.: Active learning-based classifier personalization: A case of on-body device localization. In: 2017 IEEE 6th Global Conference on Consumer Electronics (GCCE), pp. 1–2 (2017). https://doi.org/10.1109/GCCE.2017.8229317
Satyanarayanan, M.: Pervasive computing: vision and challenges. IEEE Pers. Commun. 8(4), 10–17 (2001). https://doi.org/10.1109/98.943998
Singh, H., Campbell, B., Elsayed, S., Perry, A., Hunjet, R., Abbass, H.: Modulation of force vectors for effective shepherding of a swarm: A bi-objective approach. In: 2019 IEEE Congress on Evolutionary Computation (CEC), pp. 2941–2948. IEEE, Piscataway (2019). https://doi.org/10.1109/CEC.2019.8790228
Steinhauer, H.J., Chua, S.L., Guesgen, H.W., Marsland, S.R.: Utilising temporal information in behaviour recognition. In: AAAI Spring Symposium: It’s All in the Timing (2010)
Strömbom, D., Mann, R.P., Wilson, A.M., Hailes, S., Morton, A.J., Sumpter, D.J.T., King, A.J.: Solving the shepherding problem: heuristics for herding autonomous, interacting agents. J. R. Soc. Interf. 11(100) (2014). https://browzine.com/articles/52614503
Sun, Y., Rossi, L., Luan, H., Shen, C.C.: Modeling and analyzing large swarms with covert leaders. In: 2013 IEEE 7th International Conference on Self-Adaptive and Self-Organizing Systems, pp. 169–178 (2013). https://doi.org/10.1109/SASO.2013.32
Tinbergen, N.: The Study of Instinct. Clarendon Press, Oxford (1951)
Vail, D.L., Veloso, M.M., Lafferty, J.D.: Conditional random fields for activity recognition. In: Proceedings of the 6th International Joint Conference on Autonomous Agents and Multiagent Systems, AAMAS ’07, pp. 235:1–235:8. ACM, New York (2007). https://doi.org/10.1145/1329125.1329409
Wang, X.R., Miller, J.M., Lizier, J.T., Prokopenko, M., Rossi, L.F.: Measuring information storage and transfer in swarms. In: European Conference on Artificial Life, Paris (2011)
Weiss, G. (ed.): Multiagent Systems: A Modern Approach to Distributed Artificial Intelligence. MIT Press, Cambridge (1999)
Weisstein, E.W.: Law of cosines. MathWorld–A Wolfram Web Resource (2019). http://mathworld.wolfram.com/LawofCosines.html
Williams, H.J., Holton, M.D., Shepard, E.L.C., Largey, N., Norman, B., Ryan, P.G., Duriez, O., Scantlebury, M., Quintana, F., Magowan, E.A., Marks, N.J., Alagaili, A.N., Bennett, N.C., Wilson, R.P.: Identification of animal movement patterns using tri-axial magnetometry. Mov. Ecol. 5(1), 6 (2017)
Wooldridge, M.: An Introduction to MultiAgent Systems. Wiley, New York (2009). https://books.google.com.au/books?id=X3ZQ7yeDn2IC
Ye, J., Stevenson, G.: Semantics-driven multi-user concurrent activity recognition. In: Augusto, J.C., Wichert, R., Collier, R., Keyson, D., Salah, A.A., Tan, A.H. (eds.) Ambient Intelligence, pp. 204–219. Springer International Publishing, Cham (2013)
Ye, J., Stevenson, G., Dobson, S.: A top-level ontology for smart environments. Pervasive Mob. Comput. 7(3), 359–378 (2011). Knowledge-Driven Activity Recognition in Intelligent Environments. https://doi.org/10.1016/j.pmcj.2011.02.002. http://www.sciencedirect.com/science/article/pii/S1574119211000277.
Ye, J., Dobson, S., McKeever, S.: Situation identification techniques in pervasive computing: a review. Pervasive Mob. Comput. 8(1), 36 – 66 (2012). https://doi.org/10.1016/j.pmcj.2011.01.004. http://www.sciencedirect.com/science/article/pii/S1574119211000253
Ye, J., Dasiopoulou, S., Stevenson, G., Meditskos, G., Kontopoulos, E., Kompatsiaris, I., Dobson, S.: Semantic web technologies in pervasive computing. Pervasive Mob. Comput. 23(C), 1–25 (2015). https://doi.org/10.1016/j.pmcj.2014.12.009
Ye, J., Fang, L., Dobson, S.: Discovery and recognition of unknown activities. In: Proceedings of the 2016 ACM International Joint Conference on Pervasive and Ubiquitous Computing: Adjunct, UbiComp ’16, pp. 783–792. ACM, New York (2016). https://doi.org/10.1145/2968219.2968288
Yordanova, K., Krüger, F., Kirste, T.: Context aware approach for activity recognition based on precondition-effect rules. In: 2012 IEEE International Conference on Pervasive Computing and Communications Workshops, pp. 602–607 (2012). https://doi.org/10.1109/PerComW.2012.6197586
Zheng, V.W., Hu, D.H., Yang, Q.: Cross-domain activity recognition. In: Proceedings of the 11th International Conference on Ubiquitous Computing, UbiComp ’09, pp. 61–70. ACM, New York (2009). https://doi.org/10.1145/1620545.1620554
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2021 Springer Nature Switzerland AG
About this chapter
Cite this chapter
Hepworth, A.J. (2021). Activity Recognition for Shepherding. In: Abbass, H.A., Hunjet, R.A. (eds) Shepherding UxVs for Human-Swarm Teaming. Unmanned System Technologies. Springer, Cham. https://doi.org/10.1007/978-3-030-60898-9_7
Download citation
DOI: https://doi.org/10.1007/978-3-030-60898-9_7
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-030-60897-2
Online ISBN: 978-3-030-60898-9
eBook Packages: EngineeringEngineering (R0)