A probabilistic interval-based event calculus for activity recognition

  • Alexander ArtikisEmail author
  • Evangelos Makris
  • Georgios Paliouras


Activity recognition refers to the detection of temporal combinations of ‘low-level’ or ‘short-term’ activities on sensor data. Various types of uncertainty exist in activity recognition systems and this often leads to erroneous detection. Typically, the frameworks aiming to handle uncertainty compute the probability of the occurrence of activities at each time-point. We extend this approach by defining the probability of a maximal interval and the credibility rate for such intervals. We then propose a linear-time algorithm for computing all probabilistic temporal intervals of a given dataset. We evaluate the proposed approach using a benchmark activity recognition dataset, and outline the conditions in which our approach outperforms time-point-based recognition.


Action languages Complex event recognition Probabilistic logic programming 

Mathematics Subject Classification (2010)



Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.



This work was supported by the INFORE project, which has received funding from the European Union’s Horizon 2020 research and innovation programme, under grant agreement No 825070. We would also like to thank Elias Alevizos for his feedback at the early stages of this work, and Evangelos Michelioudakis for his support in the experiments.


  1. 1.
    Agrawal, J., Diao, Y., Gyllstrom, D., Immerman, N.: Efficient pattern matching over event streams. In: SIGMOD, pp. 147–160 (2008)Google Scholar
  2. 2.
    Akman, V., Selim, T., Erdoğan, J.L., Lifschitz, V., Turner, H.: Representing the Zoo World and the Traffic World in the language of the causal calculator. Artif. Intell. 153(1), 105–140 (2004)zbMATHGoogle Scholar
  3. 3.
    Albanese, M., Chellappa, R., Cuntoor, N., Moscato, V., Picariello, A., Subrahmanian, V.S., Udrea, O.: PADS: A probabilistic activity detection framework for video data. IEEE Trans. Pattern Anal. Mach. Intell. 32(12), 2246–2261 (2010)Google Scholar
  4. 4.
    Alevizos, E., Skarlatidis, A., Artikis, A., Paliouras, G.: Probabilistic complex event recognition: A survey. ACM Comput. Surv. 50(5), 71:1–71:31 (2017)Google Scholar
  5. 5.
    Allen, J.F.: Maintaining knowledge about temporal intervals. Commun. ACM 26(11), 832–843 (1983)zbMATHGoogle Scholar
  6. 6.
    Allison, L.: Longest biased interval and longest non-negative sum interval. Bioinformatics 19(10), 1294–1295 (2003)Google Scholar
  7. 7.
    Artikis, A., Sergot, M.J., Paliouras, G.: An event calculus for event recognition. IEEE Trans. Knowl. Data Eng. 27(4), 895–908 (2015)Google Scholar
  8. 8.
    Artikis, A., Skarlatidis, A., Portet, F., Paliouras, G.: Logic-based event recognition. Knowl. Eng. Rev. 27(4), 469–506 (2012)Google Scholar
  9. 9.
    Bacchus, F., Halpern, J.Y., Levesque, H.J.: Reasoning about noisy sensors in the situation calculus. In: Proceedings of the 14th International Joint Conference on Artificial Intelligence (IJCAI), pp. 1933–1940. Morgan Kaufmann (1995)Google Scholar
  10. 10.
    Baral, C., Nam, T.H., Tuan, L.: Reasoning about actions in a probabilistic setting. In: Proceedings of the Eighteenth National Conference on Artificial Intelligence and Fourteenth Conference on Innovative Applications of Artificial Intelligence, pp. 507–512 (2002)Google Scholar
  11. 11.
    Brendel, W., Fern, A., Todorovic, S.: Probabilistic event logic for interval-based event recognition. In: The 24th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011, Colorado Springs, CO, USA, 20-25 June 2011, pp. 3329–3336 (2011)Google Scholar
  12. 12.
    Bryant, R.E.: Graph-based algorithms for boolean function manipulation. IEEE Trans. Comput. 35(8), 677–691 (1986)zbMATHGoogle Scholar
  13. 13.
    Busany, N., Gal, A., Senderovich, A., Weidlich, M.: Interval-based queries over multiple streams with missing timestamps (2017)Google Scholar
  14. 14.
    Cervesato, I., Montanari, A.: A calculus of macro-events: Progress report. In: TIME, pp. 47–58 (2000)Google Scholar
  15. 15.
    Chesani, F., Mello, P., Montali, M., Torroni, P.: A logic-based, reactive calculus of events. Fundamenta Informaticae 105(1–2), 135–161 (2010)MathSciNetzbMATHGoogle Scholar
  16. 16.
    Chittaro, L., Montanari, A.: Efficient temporal reasoning in the cached event calculus. Comput. Intell. 12(3), 359–382 (1996)MathSciNetGoogle Scholar
  17. 17.
    Chittaro, L., Montanari, A.: Temporal representation and reasoning in artificial intelligence: Issues and approaches. Ann. Math. Artif. Intell. 28(1), 47–106 (2000)MathSciNetzbMATHGoogle Scholar
  18. 18.
    Clark, K.: Negation as failure. In: Gallaire, H., Minker, J. (eds.) Logic and Databases, pp. 293–322. Plenum Press (1978)Google Scholar
  19. 19.
    Craven, R.: Execution Mechanisms for the Action Language C+. Ph.D. thesis University of London (2006)Google Scholar
  20. 20.
    Cugola, G., Margara, A.: Processing flows of information: From data stream to complex event processing. ACM Comput. Surv. 44(3), 15:1–15:62 (2012)Google Scholar
  21. 21.
    D’Asaro, F.A., Bikakis, A., Dickens, L., Miller, R.: Foundations for a probabilistic event calculus. In: Balduccini, M., Janhunen, T. (eds.) Logic Programming and Nonmonotonic Reasoning - 14th International Conference, LPNMR 2017, Espoo, Finland, July 3-6, 2017, Proceedings, Lecture Notes in Computer Science, vol. 10377, pp. 57–63. Springer (2017)Google Scholar
  22. 22.
    D’Odorico, T.: An Ontological Analysis of Vague Motion Verbs, with an Application to Event Recognition. Ph.D. thesis, University of Leeds, UK (2013).
  23. 23.
    D’Odorico, T., Bennett, B.: Automated reasoning on vague concepts using formal ontologies, with an application to event detection on video data. In: Michael, L., Ortiz, C., Johnston, B. (eds.) Commonsense 2013—Proceedings of the 11th International Symposium on Logical Formalizations of Commonsense Reasoning (2013)Google Scholar
  24. 24.
    Doherty, P., Gustafsson, J., Karlsson, L., Kvarnström, J.: TAL: Temporal action logics language specification and tutorial. Electron. Trans. Artif. Intell. 2(3–4), 273–306 (1998)MathSciNetGoogle Scholar
  25. 25.
    Dousson, C., Maigat, P.L.: Chronicle recognition improvement using temporal focusing and hierarchization. In: IJCAI 2007, Proceedings of the 20th International Joint Conference on Artificial Intelligence, Hyderabad, India, January 6-12, 2007, pp. 324–329 (2007)Google Scholar
  26. 26.
    Dries, A., Kimmig, A., Meert, W., Renkens, J., den Broeck, G.V., Vlasselaer, J., Raedt, L.D.: Problog2: Probabilistic logic programming. In: Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD 2015, Porto, Portugal, September 7-11, 2015, Proceedings, Part III, pp. 312–315 (2015)Google Scholar
  27. 27.
    Eiter, T., Lukasiewicz, T.: Probabilistic reasoning about actions in nonmonotonic causal theories. In: Meek, C., Kjærulff, U. (eds.) UAI, pp. 192–199. Morgan Kaufmann (2003)Google Scholar
  28. 28.
    Fagin, R.: Combining fuzzy information from multiple systems. In: Proceedings of the Fifteenth ACM SIGACT-SIGMOD-SIGART Symposium on Principles of Database Systems, pp. 216–226. ACM Press (1996)Google Scholar
  29. 29.
    Gelfond, M., Lifschitz, V.: Representing action and change by logic programs. J. Log. Program. 17(2/3&4), 301–321 (1993)MathSciNetzbMATHGoogle Scholar
  30. 30.
    Giatrakos, N., Alevizos, E., Artikis, A., Deligiannakis, A., Garofalakis, M.: Complex event recognition in the big data era. VLDB Journal (2019)Google Scholar
  31. 31.
    Gibson, J.: The ecological approach to visual perception (1979)Google Scholar
  32. 32.
    Ginsberg, M.L.: Multivalued logics: A uniform approach to reasoning in artificial intelligence. Comput. Intell. 4, 265–316 (1988)Google Scholar
  33. 33.
    Giunchiglia, E., Lee, J., Lifschitz, V., McCain, N., Turner, H.: Nonmonotonic causal theories. Artif. Intell. 153(1), 49–104 (2004)MathSciNetzbMATHGoogle Scholar
  34. 34.
    Hajishirzi, H., Amir, E.: Sampling first order logical particles. In: Proceedings of the 24th Conference in Uncertainty in Artificial Intelligence (UAI), Helsinki, Finland, pp. 248–255. AUAI Press (2008)Google Scholar
  35. 35.
    Hölldobler, S., Karabaev, E., Skvortsova, O.: FLUCAP: A heuristic search planner for first-order MDPs. J. Artif. Intell. Res. (JAIR) 27(1), 419–439 (2006)zbMATHGoogle Scholar
  36. 36.
    Iocchi, L., Lukasiewicz, T., Nardi, D., Rosati, R.: Reasoning about actions with sensing under qualitative and probabilistic uncertainty. ACM Trans. Comput. Log. 10(1), 5:1–5:41 (2009)MathSciNetzbMATHGoogle Scholar
  37. 37.
    Kardas, K., Cicekli, N.K.: SVAS: surveillance video analysis system. Expert Syst. Appl. 89, 343–361 (2017)Google Scholar
  38. 38.
    van Kasteren, T., Englebienne, G., Kröse, B.: Human activity recognition from wireless sensor network data: Benchmark and software. In: Activity Recognition in Pervasive Intelligent Environments, Atlantis Ambient and Pervasive Intelligence. Atlantis Press (2010)Google Scholar
  39. 39.
    Katzouris, N., Artikis, A., Paliouras, G.: Online learning of event definitions. TPLP 16(5–6), 817–833 (2016)MathSciNetzbMATHGoogle Scholar
  40. 40.
    Katzouris, N., Michelioudakis, E., Artikis, A., Paliouras, G.: Online learning of weighted relational rules for complex event recognition. In: Machine Learning and Knowledge Discovery in Databases - European Conference, ECML PKDD, pp. 396–413 (2018)Google Scholar
  41. 41.
    Kimmig, A., Demoen, B., Raedt, L.D., Costa, V.S., Rocha, R.: On the implementation of the probabilistic logic programming language ProbLog. Theory Pract. Logic Program. 11, 235–262 (2011)MathSciNetzbMATHGoogle Scholar
  42. 42.
    Kowalski, R., Sadri, F.: Reconciling the event calculus with the situation calculus. J. Logic Program. 31(1), 39–58 (1997)MathSciNetzbMATHGoogle Scholar
  43. 43.
    Kowalski, R.A., Sergot, M.J.: A logic-based calculus of events. Gen. Comput. 4(1), 67–95 (1986)zbMATHGoogle Scholar
  44. 44.
    Kvarnström, J.: TALplanner and Other Extensions to Temporal Action Logic. Ph.D. thesis, Linköping (2005)Google Scholar
  45. 45.
    List, T., Bins, J., Vazquez, J., Fisher, R.B.: Performance evaluating the evaluator. In: Proceedings 2nd Joint IEEE Int. Workshop on Visual Surveillance and Performance Evaluation of Tracking and Surveillance, pp. 129–136 (2005)Google Scholar
  46. 46.
    Mateus, P., Pacheco, A., Pinto, J., Sernadas, A., Sernadas, C.: Probabilistic situation calculus. Ann. Math. Artif. Intell. 32(1), 393–431 (2001)MathSciNetzbMATHGoogle Scholar
  47. 47.
    McCarthy, J., Hayes, P.J.: Some Philosophical Problems from the Standpoint of Artificial Intelligence. Stanford University (1968)Google Scholar
  48. 48.
    Michelioudakis, E., Artikis, A., Paliouras, G.: Semi-supervised online structure learning for composite event recognition. Mach. Learn. 108(7), 1085–1110 (2019)MathSciNetzbMATHGoogle Scholar
  49. 49.
    Miller, R., Shanahan, M.: Some alternative formulations of the event calculus. In: Computational Logic: Logic Programming and Beyond, Essays in Honour of Robert A. Kowalski, Part II, Lecture Notes in Computer Science, pp. 452–490. Springer (2002)Google Scholar
  50. 50.
    Moldovan, B., Moreno, P., van Otterlo, M., Santos-Victor, J., De Raedt, L.: Learning relational affordance models for robots in multi-object manipulation tasks. In: Proceedings of International Conference on Robotics and Automation (ICRA), pp. 4373–4378 (2012)Google Scholar
  51. 51.
    Montali, M., Maggi, F.M., Chesani, F., Mello, P., van der Aalst, W.M.P.: Monitoring business constraints with the. Event Calculus ACM TIST, 5(1) (2014)Google Scholar
  52. 52.
    Morariu, V.I., Davis, L.S.: Multi-agent event recognition in structured scenarios. In: The 24th IEEE Conference on Computer Vision and Pattern Recognition, CVPR 2011, Colorado Springs, CO, USA, 20-25 June 2011, pp. 3289–3296 (2011)Google Scholar
  53. 53.
    Mueller, E.T.: Commonsense Reasoning. Morgan Kaufmann (2006)Google Scholar
  54. 54.
    Mueller, E.T.: Event calculus and temporal action logics compared. Artif. Intell. 170(11), 1017–1029 (2006)MathSciNetzbMATHGoogle Scholar
  55. 55.
    Paschke, A.: ECA-RuleML: An approach combining ECA rules with temporal interval-based KR event/action logics and transactional update logics. Tech. Rep. 11 Technische Universität München (2005)Google Scholar
  56. 56.
    Paschke, A., Bichler, M.: Knowledge representation concepts for automated SLA management. Decis. Support. Syst. 46(1), 187–205 (2008)Google Scholar
  57. 57.
    Paschke, A., Kozlenkov, A.: Rule-based event processing and reaction rules. In: Proceedings of the 3rd International Symposium on Rules (RuleML), Lecture Notes in Computer Science, vol. 5858, pp. 53–66. Springer (2009)Google Scholar
  58. 58.
    Pinto, J., Sernadas, A., Sernadas, C., Mateus, P.: Non-determinism and uncertainty in the situation calculus. Int. J. Uncert. Fuzziness Knowl.-Based Syst. 8(2), 127–149 (2000)MathSciNetzbMATHGoogle Scholar
  59. 59.
    Pitsikalis, M., Artikis, A., Dreo, R., Ray, C., Camossi, E., Jousselme, A.: Composite event recognition for maritime monitoring. In: Proceedings of the 13th ACM International Conference on Distributed and Event-based Systems, DEBS, pp. 163–174 (2019)Google Scholar
  60. 60.
    Reiter, R.: Knowledge in Action: Logical Foundations for Specifying and Implementing Dynamical Systems. MIT Press (2001)Google Scholar
  61. 61.
    Richardson, M., Domingos, P.: Markov logic networks. Mach. Learn. 62(1–2), 107–136 (2006)Google Scholar
  62. 62.
    Sadilek, A., Kautz, H.A.: Location-based reasoning about complex multi-agent behavior. J. Artif. Intell. Res. (JAIR) 43, 87–133 (2012)MathSciNetzbMATHGoogle Scholar
  63. 63.
    Schiffel, S., Thielscher, M.: Reconciling situation calculus and fluent calculus. In: Proceedings of the National Conference on Artificial Intelligence, vol. 21, p. 287. AAAI Press (2006)Google Scholar
  64. 64.
    Selman, J., Amer, M.R., Fern, A., Todorovic, S.: PEL-CNF: Probabilistic event logic conjunctive normal form for video interpretation. In: ICCVW, pp. 680–687. IEEE (2011)Google Scholar
  65. 65.
    Shen, Z., Kawashima, H., Kitagawa, H.: Probabilistic event stream processing with lineage. In: Proc. of Data Engineering Workshop (2008)Google Scholar
  66. 66.
    Shet, V.D., Neumann, J., Ramesh, V., Davis, L.S.: Bilattice-based logical reasoning for human detection. In: (CVPR), pp. 1–8. IEEE Computer Society (2007)Google Scholar
  67. 67.
    Siskind, J.M.: Grounding the lexical semantics of verbs in visual perception using force dynamics and event logic. JAIR 15, 31–90 (2001)zbMATHGoogle Scholar
  68. 68.
    Skarlatidis, A., Artikis, A., Filipou, J., Paliouras, G.: A probabilistic logic programming event calculus. TPLP 15(2), 213–245 (2015)zbMATHGoogle Scholar
  69. 69.
    Skarlatidis, A., Paliouras, G., Artikis, A., Vouros, G.A.: Probabilistic event calculus for event recognition. ACM Trans. Comput Logic, 16(2) (2015)Google Scholar
  70. 70.
    Thielscher, M.: From situation calculus to fluent calculus: State update axioms as a solution to the inferential frame problem. Artif. Intell. 111(1), 277–299 (1999)MathSciNetzbMATHGoogle Scholar
  71. 71.
    Thielscher, M.: The qualification problem: A solution to the problem of anomalous models. Artif. Intell. 131(1), 1–37 (2001)MathSciNetzbMATHGoogle Scholar
  72. 72.
    Van Belleghem, K., Denecker, M., De Schreye, D.: On the relation between situation calculus and event calculus. J. Logic Program. 31(1), 3–37 (1997)MathSciNetzbMATHGoogle Scholar
  73. 73.
    Vouros, G.A., Vlachou, A., Santipantakis, G.M., Doulkeridis, C., Pelekis, N., Georgiou, H.V., Theodoridis, Y., Patroumpas, K., Alevizos, E., Artikis, A., Claramunt, C., Ray, C., Scarlatti, D., Fuchs, G., Andrienko, G.L., Andrienko, N.V., Mock, M., Camossi, E., Jousselme, A., Garcia, J.M.C.: Big data analytics for time critical mobility forecasting: Recent progress and research challenges. In: Proceedings of the 21th International Conference on Extending Database Technology, EDBT, pp. 612–623 (2018)Google Scholar
  74. 74.
    Wang, J., Domingos, P.: Hybrid Markov logic networks. In: Proceedings of the 23rd AAAI Conference on Artificial Intelligence, pp. 1106–1111. AAAI Press (2008)Google Scholar
  75. 75.
    Wang, Y.H., Cao, K., Zhang, X.M.: Complex event processing over distributed probabilistic event streams. Comput. Math. Appl. 66(10), 1808–1821 (2013)zbMATHGoogle Scholar
  76. 76.
    Wu, E., Diao, Y., Rizvi, S.: High-performance complex event processing over streams. In: ACM SIGMOD, pp. 407–418 (2006)Google Scholar
  77. 77.
    Zhang, H., Diao, Y., Immerman, N.: Recognizing patterns in streams with imprecise timestamps. VLDB 3(1-2), 244–255 (2010)Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Department of Maritime StudiesUniversity of PiraeusPiraeusGreece
  2. 2.Institute of Informatics & TelecommunicationsNCSR ‘Demokritos’AthensGreece

Personalised recommendations