Spatiotemporal Models of Human Activity for Robotic Patrolling

  • Tomáš VintrEmail author
  • Kerem Eyisoy
  • Vanda Vintrová
  • Zhi Yan
  • Yassine Ruichek
  • Tomáš Krajník
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11472)


We present a method that allows autonomous systems to detect anomalous events in human-populated environments through understating of their structure and how they change over time. We represent the environment by temporary warped space-hypertime continuous models derived from patterns of changes driven by human activities within the observed space. The ability of the method to detect anomalies is evaluated on real-world datasets gathered by robots over the course of several weeks. An earlier version of this approach was already applied to robots that patrolled offices of a global security company (G4S).


  1. 1.
    Barnett, V., Lewis, T.: Outliers in Statistical Data. Wiley, Hoboken (1974)Google Scholar
  2. 2.
    Brockwell, P.J., Davis, R.A.: Introduction to Time Series and Forecasting. Springer, Cham (2016). Scholar
  3. 3.
    Coppola, C., Krajnık, T., Duckett, T., Bellotto, N.: Learning temporal context for activity recognition. In: European Conference on Artificial Intelligence (2016)Google Scholar
  4. 4.
    Evans, M., Hastings, N., Peacock, B.: Bernoulli distribution. In: Statistical Distributions, 3rd edn, pp. 31–33. Wiley, Hoboken (2000)Google Scholar
  5. 5.
    Gould, P.G., Koehler, A.B., Ord, J.K., Snyder, R.D., Hyndman, R.J., Vahid-Araghi, F.: Forecasting time series with multiple seasonal patterns. Eur. J. Oper. Res. 191(1), 207–222 (2008)MathSciNetCrossRefGoogle Scholar
  6. 6.
    Grubbs, F.E.: Procedures for detecting outlying observations in samples. Technometrics 11(1), 1–21 (1969)CrossRefGoogle Scholar
  7. 7.
    Hanheide, M., Hebesberger, D., Krajník, T.: The when, where, and how: an adaptive robotic info-terminal for care home residents. In: Proceedings of the 2017 ACM/IEEE International Conference on Human-Robot Interaction, HRI 2017, pp. 341–349. ACM, New York (2017).
  8. 8.
    Hawes, N., et al.: The strands project: long-term autonomy in everyday environments. IEEE Robot. Autom. Mag. 24(3), 146–156 (2017). Scholar
  9. 9.
    Hyndman, R.J., Athanasopoulos, G.: Forecasting: principles and practice. OTexts (2018)Google Scholar
  10. 10.
    Hyndman, R.J., Koehler, A.B.: Another look at measures of forecast accuracy. Int. J. Forecast. 22(4), 679–688 (2006)CrossRefGoogle Scholar
  11. 11.
    Ilango, V., Subramanian, R., Vasudevan, V.: A five step procedure for outlier analysis in data mining. Eur. J. Sci. Res. 75(3), 327–339 (2012)Google Scholar
  12. 12.
    Krajník, T., Fentanes, J.P., Mozos, O.M., Duckett, T., Ekekrantz, J., Hanheide, M.: Long-term topological localisation for service robots in dynamic environments using spectral maps. In: IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4537–4542. IEEE (2014)Google Scholar
  13. 13.
    Krajník, T., Fentanes, J.P., Santos, J.M., Duckett, T.: FreMEn: frequency map enhancement for long-term mobile robot autonomy in changing environments. IEEE Trans. Robot. 33(4), 964–977 (2017)CrossRefGoogle Scholar
  14. 14.
    Krajnik, T., Fentanes, J.P., Cielniak, G., Dondrup, C., Duckett, T.: Spectral analysis for long-term robotic mapping. In: 2014 IEEE International Conference on Robotics and Automation (ICRA), pp. 3706–3711. IEEE (2014)Google Scholar
  15. 15.
    Krajník, T., Fentanes, J.P., Hanheide, M., Duckett, T.: Persistent localization and life-long mapping in changing environments using the frequency map enhancement. In: 2016 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), pp. 4558–4563. IEEE (2016)Google Scholar
  16. 16.
    Krajník, T., Kulich, M., Mudrová, L., Ambrus, R., Duckett, T.: Where’s waldo at time t? Using spatio-temporal models for mobile robot search. In: IEEE International Conference on Robotics and Automation (ICRA) (2015)Google Scholar
  17. 17.
    Kunze, L., Hawes, N., Duckett, T., Hanheide, M., Krajnik, T.: Artificial intelligence for long-term robot autonomy: a survey. In: IEEE Robotics and Automation Letters, p. 1 (2018).
  18. 18.
    Maronna, R., Martin, R.D., Yohai, V.: Robust Statistics, vol. 1. Wiley, Chichester (2006)CrossRefGoogle Scholar
  19. 19.
    Matthews, B.W.: Comparison of the predicted and observed secondary structure of T4 phage lysozyme. Biochimica et Biophysica Acta (BBA)-Protein Struct. 405(2), 442–451 (1975)CrossRefGoogle Scholar
  20. 20.
    Fentanes, J.P., Lacerda, B., Krajník, T., Hawes, N., Hanheide, M.: Now or later? Predicting and maximising success of navigation actions from long-term experience. In: IEEE International Conference on Robotics and Automation (ICRA) (2015)Google Scholar
  21. 21.
    Rousseeuw, P.J., Leroy, A.M.: Robust Regression and Outlier Detection, vol. 589. Wiley, New York (2005)Google Scholar
  22. 22.
    Santos, J.M., Krajník, T., Duckett, T.: Spatio-temporal exploration strategies for long-term autonomy of mobile robots. Robot. Autonom. Syst. 88, 116–126 (2017)CrossRefGoogle Scholar
  23. 23.
    Santos, J.M., Krajnik, T., Pulido Fentanes, J., Duckett, T.: Lifelong information-driven exploration to complete and refine 4D spatio-temporal maps. Robot. Autom. Lett. 1(2), 684–691 (2016)CrossRefGoogle Scholar
  24. 24.
    Taylor, S.J., Letham, B.: Forecasting at scale. Am. Stat. 72(1), 37–45 (2018)MathSciNetCrossRefGoogle Scholar
  25. 25.
    Vintr, T., Molina, S., Fentanes, J., Cielniak, G., Duckett, T., Krajník, T.: Spatio-temporal models for motion planning in human populated environments. In: Student Conference on Planning in Artificial Intelligence and Robotics (2017)Google Scholar
  26. 26.
    Xu, L., Jordan, M.I.: On convergence properties of the EM algorithm for Gaussian mixtures. Neural Comput. 8(1), 129–151 (1996)CrossRefGoogle Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Artificial Intelligence Center, Faculty of Electrical EngineeringCzech Technical UniversityPragueCzech Republic
  2. 2.Department of Computer Engineering, Faculty of EngineeringMarmara UniversityIstanbulTurkey
  3. 3.Faculty of Informatics and StatisticsUniversity of EconomicsPragueCzech Republic
  4. 4.EPAN Research Group, LE2I-CNRS, University of Technology of Belfort-Montbliard (UTBM)BelfortFrance

Personalised recommendations