Companion-Systems: A Reference Architecture

  • Thilo HörnleEmail author
  • Michael Tornow
  • Frank Honold
  • Reinhard Schwegler
  • Ralph Heinemann
  • Susanne Biundo
  • Andreas Wendemuth
Part of the Cognitive Technologies book series (COGTECH)


Companion-Technology for cognitive technical systems consists of a multitude of components that implement different properties. A primary point is the architecture which is responsible for the interoperability of all components. It defines the capabilities of the systems crucially. For research concerning the requirements and effects of the architecture, several demonstration scenarios were developed. Each of these demonstration scenarios focuses on some aspects of a Companion-System. For the implementation a middleware concept was used, having the capability to realize the major part of the Companion-Systems. Currently the system architecture takes up only a minor property in projects which are working on related research topics. For the description of an architecture representing the major part of possible Companion-Systems, the demonstration scenarios are studied with regard to their system structure and the constituting components. A monolithic architecture enables a simple system design and fast direct connections between the components, such as: sensors with their processing and fusion components, knowledge bases, planning components, dialog systems and interaction components. Herein, only a limited number of possible Companion-Systems can be represented. In a principled approach, a dynamic architecture, capable of including new components during run time, is able to represent almost all Companion-Systems. Furthermore, an approach for enhancing the architecture is introduced.



This work was supported by the Transregional Collaborative Research Centre SFB/TRR 62 “Companion-Technology for Cognitive Technical Systems” which is funded by the German Research Foundation (DFG). The authors thank the following colleagues for their invaluable support (in alphabetical order): Pascal Bercher, Peter Kurzok, Andreas Meinecke, Bernd Schattenberg, and Felix Schüssel.


  1. 1.
    Atrey, P.K., Hossain, M.A., El Saddik, A., Kankanhalli, M.S.: Multimodal fusion for multimedia analysis: a survey. Multimed. Syst. 16(6), 345–379 (2010). doi:10.1007/s00530-010-0182-0CrossRefGoogle Scholar
  2. 2.
    Bercher, P., Biundo, S., Geier, T., Hoernle, T., Nothdurft, F., Richter, F., Schattenberg, B.: Plan, repair, execute, explain – how planning helps to assemble your home theater. In: Proceedings of the 24th International Conference on Automated Planning and Scheduling (ICAPS 2014), pp. 386–394. AAAI Press, Palo Alto (2014)Google Scholar
  3. 3.
    Bercher, P., Richter, F., Hörnle, T., Geier, T., Höller, D., Behnke, G., Nothdurft, F., Honold, F., Minker, W., Weber, M., Biundo, S.: A planning-based assistance system for setting up a home theater. In: Proceedings of the 29th National Conference on Artificial Intelligence (AAAI 2015), pp. 4264–4265. AAAI Press, Palo Alto (2015)Google Scholar
  4. 4.
    Biundo, S., Wendemuth, A.: Companion-technology for cognitive technical systems. KI – Künstl. Intell. (2015). doi:10.1007/s13218-015-0414-8Google Scholar
  5. 5.
    Caschera, M.C., D’Ulizia, A., Ferri, F., Grifoni, P.: Multimodal systems: an excursus of the main research questions. In: Ciuciu, I., Panetto, H., Debruyne, C., Aubry, A., Bollen, P., Valencia-García, R., Mishra, A., Fensel, A., Ferri, F. (eds.) On the Move to Meaningful Internet Systems: OTM 2015 Workshops. Lecture Notes in Computer Science, vol. 9416, pp. 546–558. Springer, Cham (2015). doi:10.1007/978-3-319-26138-6_59CrossRefGoogle Scholar
  6. 6.
    Glodek, M., Tschechne, S., Layher, G., Schels, M., Brosch, T., Scherer, S., Kächele, M., Schmidt, M., Neumann, H., Palm, G., Schwenker, F.: Multiple classifier systems for the classification of audio-visual emotional states. In: D’Mello, S., Graesser, A., Schuller, B., Martin, J.C. (eds.) Affective Computing and Intelligent Interaction. Lecture Notes in Computer Science, vol. 6975, pp. 359–368. Springer, Berlin/Heidelberg (2011). doi:10.1007/978-3-642-24571-8_47CrossRefGoogle Scholar
  7. 7.
    Glodek, M., Honold, F., Geier, T., Krell, G., Nothdurft, F., Reuter, S., Schüssel, F., Hörnle, T., Dietmayer, K., Minker, W., Biundo, S., Weber, M., Palm, G., Schwenker, F.: Fusion paradigms in cognitive technical systems for human–computer interaction. Neurocomputing 161(0), 17–37 (2015). doi:10.1016/j.neucom.2015.01.076CrossRefGoogle Scholar
  8. 8.
    Handrich, S., Al-Hamadi, A.: Multi hypotheses based object tracking in HCI environments. In: 2012 19th IEEE International Conference on Image Processing (ICIP), pp. 1981–1984 (2012). doi:10.1109/ICIP.2012.6467276Google Scholar
  9. 9.
    Honold, F., Poguntke, M., Schüssel, F., Weber, M.: Adaptive dialogue management and UIDL-based interactive applications. In: Proceedings of the International Workshop on Software Support for User Interface Description Language (UIDL 2011). Thales Research and Technology France, Paris (2011)Google Scholar
  10. 10.
    Honold, F., Schüssel, F., Weber, M.: Adaptive probabilistic fission for multimodal systems. In: Proceedings of the 24th Australian Computer-Human Interaction Conference, OzCHI ’12, pp. 222–231. ACM, New York (2012). doi:10.1145/2414536.2414575Google Scholar
  11. 11.
    Honold, F., Schüssel, F., Weber, M., Nothdurft, F., Bertrand, G., Minker, W.: Context models for adaptive dialogs and multimodal interaction. In: 2013 9th International Conference on Intelligent Environments (IE), pp. 57–64. IEEE, Piscataway (2013). doi:10.1109/IE.2013.54Google Scholar
  12. 12.
    Honold, F., Bercher, P., Richter, F., Nothdurft, F., Geier, T., Barth, R., Hörnle, T., Schüssel, F., Reuter, S., Rau, M., Bertrand, G., Seegebarth, B., Kurzok, P., Schattenberg, B., Minker, W., Weber, M., Biundo, S.: Companion-technology: towards user- and situation-adaptive functionality of technical systems. In: 2014 10th International Conference on Intelligent Environments (IE), pp. 378–381. IEEE, Piscataway (2014). doi:10.1109/IE.2014.60Google Scholar
  13. 13.
    Hörnle, T., Tornow, M.: Reference architecture approach for companion-systems. Presented on the 1st International Symposium on Companion-Technology (2015)Google Scholar
  14. 14.
    Hrabal, D., Kohrs, C., Brechmann, A., Tan, J.W., Rukavina, S., Traue, H.: Physiological effects of delayed system response time on skin conductance. In: Schwenker, F., Scherer, S., Morency, L.P. (eds.) Multimodal Pattern Recognition of Social Signals in Human-Computer-Interaction. Lecture Notes in Computer Science, vol. 7742, pp. 52–62. Springer, Cham (2013). doi:10.1007/978-3-642-37081-6_7CrossRefGoogle Scholar
  15. 15.
    Knappmeyer, M., Kiani, S., Reetz, E., Baker, N., Tonjes, R.: Survey of context provisioning middleware. IEEE Commun. Surv. Tutorials 15(3), 1492–1519 (2013). doi:10.1109/SURV.2013.010413.00207CrossRefGoogle Scholar
  16. 16.
    Krafzig, D., Banke, K., Slama, D.: Enterprise SOA: Service-Oriented Architecture Best Practices. The Coad Series. Prentice Hall Professional Technical Reference, Indianapolis, IN (2005)Google Scholar
  17. 17.
    Layher, G., Liebau, H., Niese, R., Al-Hamadi, A., Michaelis, B., Neumann, H.: Robust stereoscopic head pose estimation in human-computer interaction and a unified evaluation framework. In: Maino, G., Foresti, G. (eds.) Image Analysis and Processing – ICIAP 2011. Lecture Notes in Computer Science, vol. 6978, pp. 227–236. Springer, Berlin/Heidelberg (2011). doi:10.1007/978-3-642-24085-0_24CrossRefGoogle Scholar
  18. 18.
    Niese, R., Al-Hamadi, A., Panning, A., Michaelis, B.: Emotion recognition based on 2D-3D facial feature extraction from color image sequences. J. Multimed. 5(5), 488–500 (2010)CrossRefGoogle Scholar
  19. 19.
    Panning, A., Al-Hamadi, A., Michaelis, B., Neumann, H.: Colored and anchored active shape models for tracking and form description of the facial features under image-specific disturbances. In: 2010 5th International Symposium on I/V Communications and Mobile Network (ISVC), pp. 1–4. IEEE, Piscataway (2010)Google Scholar
  20. 20.
    Papazoglou, M.P., Traverso, P., Dustdar, S., Leymann, F.: Service-oriented computing: state of the art and research challenges. Computer 40(11), 38–45 (2007)CrossRefGoogle Scholar
  21. 21.
    Reuter, S., Dietmayer, K.: Pedestrian tracking using random finite sets. In: 2011 Proceedings of the 14th International Conference on Information Fusion (FUSION), pp. 1–8 (2011)Google Scholar
  22. 22.
    Rösner, D., Frommer, J., Friesen, R., Haase, M., Lange, J., Otto, M.: LAST MINUTE: a multimodal corpus of speech-based user-companion interactions. In: LREC, pp. 2559–2566 (2012)Google Scholar
  23. 23.
    Schaub, F.M.: Dynamic privacy adaptation in ubiquitous computing. Dissertation, Universität Ulm. Fakultät für Ingenieurwissenschaften und Informatik (2014)Google Scholar
  24. 24.
    Schröder, M.: The SEMAINE API: towards a standards-based framework for building emotion-oriented systems. Adv. Hum. Comput. Interact. 2010, 21 (2010). doi:10.1155/2010/319406Google Scholar
  25. 25.
    Sharma, R., Pavlovic, V., Huang, T.: Toward multimodal human-computer interface. Proc. IEEE 86(5), 853–869 (1998). doi:10.1109/5.664275CrossRefGoogle Scholar
  26. 26.
    Siegert, I., Hartmann, K., Philippou-Hübner, D., Wendemuth, A.: Human behaviour in HCI: complex emotion detection through sparse speech features. In: Salah, A., Hung, H., Aran, O., Gunes, H. (eds.) Human Behavior Understanding, Lecture Notes in Computer Science, vol. 8212, pp. 246–257. Springer, Berlin/Heidelberg (2013). doi:10.1007/978-3-319-02714-2_21CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  • Thilo Hörnle
    • 1
    Email author
  • Michael Tornow
    • 2
  • Frank Honold
    • 3
  • Reinhard Schwegler
    • 1
  • Ralph Heinemann
    • 2
  • Susanne Biundo
    • 1
  • Andreas Wendemuth
    • 2
  1. 1.Institute of Artificial IntelligenceUlm UniversityUlmGermany
  2. 2.Institute of Information Technology and CommunicationsOtto-von-Guericke UniversityMagdeburgGermany
  3. 3.Institute of Media InformaticsUlm UniversityUlmGermany

Personalised recommendations