Environment Adaption for Companion-Systems

  • Stephan Reuter
  • Alexander Scheel
  • Thomas Geier
  • Klaus Dietmayer
Part of the Cognitive Technologies book series (COGTECH)


One of the key characteristics of a Companion-System is the adaptation of its functionality to the user’s preferences and the environment. On the one hand, a dynamic environment model facilitates the adaption of output modalities in human computer interaction (HCI) to the current situation. On the other hand, continuous tracking of users in the proximity of the system allows for resuming a previously interrupted interaction. Thus, an environment perception system based on a robust multi-object tracking algorithm is required to provide these functionalities. In typical Companion-System applications, persons in the proximity are closely spaced, which leads to statistical dependencies in their behavior. The multi-object Bayes filter allows for modeling these statistical dependencies by representing the multi-object state using random finite sets. Based on the social force model and the knowledge base of the companion system, an approach to modeling object interactions is presented. In this work, the interaction model is incorporated into the prediction step of the sequential Monte Carlo (SMC) of the multi-object Bayes filter. Further, an alternative implementation of the multi-object Bayes filter based on labeled random finite sets is outlined.



This work was done within the Transregional Collaborative Research Centre SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems” funded by the German Research Foundation (DFG).


  1. 1.
    Bar-Shalom, Y., Fortmann, T.: Tracking and Data Association. Academic, New York (1988)MATHGoogle Scholar
  2. 2.
    Biundo, S., Bercher, P., Geier, T., Müller, F., Schattenberg, B.: Advanced user assistance based on AI planning. Cogn. Syst. Res. 12(3-4), 219–236 (2011). doi:10.1016/j.cogsys.2010.12.005. Special Issue on Complex CognitionGoogle Scholar
  3. 3.
    Geier, T., Biundo, S., Reuter, S., Dietmayer, K.: Track-person association using a first-order probabilistic model. In: IEEE 24th International Conference on Tools with Artificial Intelligence, vol. 1, pp. 844–851 (2012). doi:10.1109/ICTAI.2012.118Google Scholar
  4. 4.
    Geier, T., Reuter, S., Dietmayer, K., Biundo, S.: Goal-based person tracking using a first-order probabilistic model. In: Proceedings of the Ninth UAI Bayesian Modeling Applications Workshop (UAI-AW 2012) (2012)Google Scholar
  5. 5.
    Helbing, D., Molnar, P.: Social force model for pedestrian dynamics. Phys. Rev. E 51, 4282–4286 (1995). doi:10.1103/PhysRevE.51.4282CrossRefGoogle Scholar
  6. 6.
    Henderson, L.F.: The statistics of crowd fluids. Nature 229, 381–383 (1971)CrossRefGoogle Scholar
  7. 7.
    Kalman, R.E.: A new approach to linear filtering and prediction problems. Trans. ASME J. Basic Eng. 82(Series D), 35–45 (1960)Google Scholar
  8. 8.
    Ma, W.K., Vo, B.N., Singh, S., Baddeley, A.: Tracking an unknown time-varying number of speakers using TDOA measurements: a random finite set approach. IEEE Trans. Signal Process. 54, 3291–3304 (2006)CrossRefMATHGoogle Scholar
  9. 9.
    Mahler, R.: Multitarget Bayes filtering via first-order multitarget moments. IEEE Trans. Aerosp. Electron. Syst. 39(4), 1152–1178 (2003). doi:10.1109/TAES.2003.1261119CrossRefGoogle Scholar
  10. 10.
    Mahler, R.: PHD filters of higher order in target number. IEEE Trans. Aerosp. Electron. Syst. 43(4), 1523–1543 (2007). doi:10.1109/TAES.2007.4441756CrossRefGoogle Scholar
  11. 11.
    Mahler, R.: Statistical Multisource-Multitarget Information Fusion. Artech House Inc, Norwood (2007)MATHGoogle Scholar
  12. 12.
    Mählisch, M.: Filtersynthese zur simultanen Minimierung von Existenz-, Assoziations- und Zustandsunsicherheiten in der Fahrzeugumfelderfassung mit heterogenen Sensordaten. Ph.D. thesis, Ulm University (2009)Google Scholar
  13. 13.
    Maskell, S.: Sequentially structured Bayesian solutions. Ph.D. thesis, Cambridge University (2004)Google Scholar
  14. 14.
    Munz, M., Mählisch, M., Dietmayer, K.: Generic centralized multi sensor data fusion based on probabilistic sensor and environment models for driver assistance systems. IEEE Intell. Transp. Syst. Mag. 2(1), 6–17 (2010). doi:10.1109/MITS.2010.937293CrossRefGoogle Scholar
  15. 15.
    Murty, K.: An algorithm for ranking all the assignments in order of increasing cost. Oper. Res. 16, 682–687 (1968)CrossRefMATHGoogle Scholar
  16. 16.
    Musicki, D., Evans, R.: Joint integrated probabilistic data association: JIPDA. IEEE Trans. Aerosp. Electron. Syst. 40(3), 1093–1099 (2004). doi:10.1109/TAES.2004.1337482CrossRefGoogle Scholar
  17. 17.
    Pellegrini, S., Ess, A., Schindler, K., van Gool, L.: You’ll never walk alone: Modeling social behavior for multi-target tracking. In: Proceedings of the 12th IEEE International Conference on Computer Vision, pp. 261–268 (2009). doi:10.1109/ICCV.2009.5459260Google Scholar
  18. 18.
    Reid, D.: An algorithm for tracking multiple targets. IEEE Trans. Autom. Control 24(6), 843–854 (1979). doi:10.1109/TAC.1979.1102177CrossRefGoogle Scholar
  19. 19.
    Reuter, S.: Multi-object tracking using random finite sets. Ph.D. thesis, Ulm University (2014)Google Scholar
  20. 20.
    Reuter, S., Dietmayer, K.: Pedestrian tracking using random finite sets. In: Proceedings of the 14th International Conference on Information Fusion, pp. 1–8 (2011)Google Scholar
  21. 21.
    Reuter, S., Dietmayer, K., Handrich, S.: Real-time implementation of a random finite set particle filter. In: Proceedings of the 6th Workshop Sensor Data Fusion: Trends, Solutions, Applications (2011). Scholar
  22. 22.
    Reuter, S., Wilking, B., Wiest, J., Munz, M., Dietmayer, K.: Real-time multi-object tracking using random finite sets. IEEE Trans. Aerosp. Electron. Syst. 49(4), 2666–2678 (2013). doi:10.1109/TAES.2014.6619956CrossRefGoogle Scholar
  23. 23.
    Reuter, S., Vo, B.T., Vo, B.N., Dietmayer, K.: The labeled multi-Bernoulli filter. IEEE Trans. Signal Process. 62(12), 3246–3260 (2014)MathSciNetCrossRefGoogle Scholar
  24. 24.
    Richardson, M., Domingos, P.: Markov logic networks. Mach. Learn. 62(1–2), 107–136 (2006). doi:10.1007/s10994-006-5833-1. Scholar
  25. 25.
    Ristic, B., Arulampalam, S., Gordon, N.: Beyond the Kalman Filter: Particle Filters for Tracking Applications. Artech House Inc., Norwood (2004)MATHGoogle Scholar
  26. 26.
    Sidenbladh, H., Wirkander, S.L.: Tracking random sets of vehicles in terrain. In: Conference on Computer Vision and Pattern Recognition Workshop, p. 98 (2003). doi:10.1109/CVPRW.2003.10097Google Scholar
  27. 27.
    Thrun, S., Burgard, W., Fox, D.: Probabilistic Robotics (Intelligent Robotics and Autonomous Agents). The MIT Press, Cambridge (2005)MATHGoogle Scholar
  28. 28.
    Vo, B.N., Ma, W.K.: The Gaussian mixture probability hypothesis density filter. IEEE Trans. Signal Process. 54(11), 4091–4104 (2006). doi:10.1109/TSP.2006.881190CrossRefMATHGoogle Scholar
  29. 29.
    Vo, B.T., Vo, B.N.: Labeled random finite sets and multi-object conjugate priors. IEEE Trans. Signal Process. 61(13), 3460–3475 (2013)MathSciNetCrossRefGoogle Scholar
  30. 30.
    Vo, B.N., Singh, S., Doucet, A.: Sequential Monte Carlo methods for multitarget filtering with random finite sets. IEEE Trans. Aerosp. Electron. Syst. 41(4), 1224–1245 (2005). doi:10.1109/TAES.2005.1561884CrossRefGoogle Scholar
  31. 31.
    Vo, B.T., Vo, B.N., Cantoni, A.: Analytic implementations of the cardinalized probability hypothesis density filter. IEEE Trans. Signal Process. 55(7), 3553–3567 (2007). doi:10.1109/TSP.2007.894241MathSciNetCrossRefGoogle Scholar
  32. 32.
    Vo, B.T., Vo, B.N., Cantoni, A.: The cardinality balanced multi-target multi-Bernoulli filter and its implementations. IEEE Trans. Signal Process. 57(2), 409–423 (2009). doi:10.1109/TSP.2008.2007924MathSciNetCrossRefGoogle Scholar
  33. 33.
    Zajic, T., Mahler, R.: A particle-systems implementation of the PHD multitarget-tracking filter. In: Signal Processing, Sensor Fusion, and Target Recognition XII, SPIE, vol. 5096, pp. 291–299, Bellingham, WA (2003)Google Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Stephan Reuter
    • 1
  • Alexander Scheel
    • 1
  • Thomas Geier
    • 2
  • Klaus Dietmayer
    • 1
  1. 1.Institute of Measurement, Control, and MicrotechnologyUlm UniversityUlmGermany
  2. 2.Institute of Artificial IntelligenceUlm UniversityUlmGermany

Personalised recommendations