Multimedia Tools and Applications

, Volume 73, Issue 1, pp 345–375 | Cite as

HiMotion: a new research resource for the study of behavior, cognition, and emotion

  • Hugo Gamboa
  • Hugo SilvaEmail author
  • Ana Fred


The HiMotion research project was designed to create a multimodal database and several support tools for the study of human behavior, cognition and emotion, in the context of computer-based tasks designed to elicit cognitive load and specialized affective responses. The database includes both human-computer interaction (HCI) and psychophysiological data, collected through an experimental setup that we devised for synchronized recording of keyboard, mouse, and central/ peripheral nervous system measurements. Currently we provide a battery of five different cognitive tasks, and a video bank for affective elicitation, together with a set of introductory and self-reporting screens. We have conducted two experiments, one involving a population of 27 subjects, which followed the cognitive tasks protocol, and another involving a population of 20 subjects, which followed the video bank visualization protocol. We provide an overview of several studies that have used the HiMotion database to test multiple hypothesis in the behavioral and affective domains, highlighting the usefulness of our contribution.


Human-computer interaction Physiological data Multimodal database Affective computing 



In the context of the HiMotion project there were several entities and people that helped or provided their support to the development of the project. First of all we express the gratitude to INSTICC in the person of Prof. Joaquim Filipe, by the support to the project by providing funding resources for scholarships and electrophysiology equipment. Part of the data acquisition was performed in the facilities of the Escola Superior de Tecnologia de Setúbal, where part of the subjects that voluntary participated in the projected were recruited. The research was preformed in the Pattern and Image Analysis group of Instituto de Telecomunicações, to which we would also like to thank. Besides the researchers, the HiMotion project had collaboration in diverse extents of Ricardo Gamboa, David Cordeiro, João Almeida, and Filipe Canento; we address a special thanks for their involvement and collaboration to the project. We would also like to thank to the psychologist, Dr. Hans Welling from Student Counseling Center at Instituto Superior Técnico and Dr. Teresa Paiva from the University of Lisbon Medical School which, kindly gave advise in preliminary stages of the design of the experiments. This work was also partially funded by the National Strategic Reference Framework (NSRF-QREN) programme under contract no. 3475 “Affective Mouse”, by PLUX—Wireless Biosignals, S.A., and by the Fundação para a Ciência e Tecnologia (FCT) under the grant SFRH/BD/65248/2009, whose support the authors gratefully acknowledge.


  1. 1.
    Aidos H, Fred A, Silva H, Carreiras C (2013) Evidence accumulation approach applied to EEG analysis. In: Proceedings of the 2nd international conference on pattern recognition applications and methods (ICPRAM), pp 479–484Google Scholar
  2. 2.
    Batliner A, Fischer K, Huber R, Spilker J, Nöth E (2003) How to find trouble in communication. Speech Commun 40(1–2):117–143CrossRefzbMATHGoogle Scholar
  3. 3.
    Bradley M, Lang P (1999) International affective digitized sounds (IADS): stimuli, instruction manual and affective ratings. Tech. Rep. B-2, The Center for Research in Psychophysiology, University of Florida, Gainesville, FLGoogle Scholar
  4. 4.
    Butcher JN, Mineka S, Hooley JM (2009) Abnormal psychology, 14th edn. PearsonGoogle Scholar
  5. 5.
    Canento F, Fred A, Silva H, Gamboa H, Lourenço A (2011) Multimodal biosignal sensor data handling for emotion recognition. In: Proceedings of the IEEE sensors conference, pp 28–31Google Scholar
  6. 6.
    Canento F, Silva H, Fred A (2012) Applicability of multi-modal electrophysiological data acquisition and processing to emotion recognition. In: Proceedings of the 2nd international workshop on computing paradigms for mental health (MindCare)Google Scholar
  7. 7.
    Carreiras C, Aidos H, Silva H, Fred A (2013) Exploratory EEG analysis using clustering and phase-locking factor. In: Proceedings of the 6th international conference on bio-inspired systems and signal processing (BIOSTEC), pp 79–88Google Scholar
  8. 8.
    Carvalho S, Leite J, Galdo-Álvarez S, Gonçalves OF (2012) The emotional movie database (EMDB): a self-report and psychophysiological study. Appl Psychophysiol Biofeedback 37(4):279–294CrossRefGoogle Scholar
  9. 9.
    Douglas-Cowie E, Cowie R, Sneddon I, Cox C, Lowry O, Mcrorie M, Martin JC, Devillers L, Abrilian S, Batliner A, Amir N, Karpouzis K (2007) The HUMAINE database: addressing the collection and annotation of naturalistic and induced emotional data. In: Proceedings of the 2nd international conference on Affective Computing and Intelligent Interaction (ACII). SpringerGoogle Scholar
  10. 10.
    Fridlund AJ, Cacioppo JT (1986) Guidelines for human electromyographic research. Psychophysiology 23(5):567–589CrossRefGoogle Scholar
  11. 11.
    Fulton J (2000) MENSA—The genius test. Carlton BooksGoogle Scholar
  12. 12.
    Gamboa H (2008) Multi-modal behavioural biometrics based on HCI and electrophysiology. PhD thesis, Universidade Técnica de Lisboa, Instituto Superior TécnicoGoogle Scholar
  13. 13.
    Gamboa H, Ferreira V (2003) WIDAM—web interaction display and monitoring. In: 5th International Conference on Enterprise Information Systems (ICEIS), pp 21–27Google Scholar
  14. 14.
    Gamboa H, Fred A (2004) A behavioral biometric system based on human-computer interaction. In: Jain AK, Ratha NK (eds) SPIE 5404—biometric technology for human identification. Orlando, USAGoogle Scholar
  15. 15.
    Gamboa H, Fred A, Jain A (2007) Webbiometrics: user verification via web interaction. In: Biometrics symposium, pp 1–6Google Scholar
  16. 16.
    Goldberger A, Amaral L, Glass L, Hausdorff J, Ivanov P, Marck R, Mietus J, Moody G, Peng C, Stanley H (2000) PhysioBank, physiotoolkit, and physionet: components of a new research resource for complex physiologic signals. Circulation 101(23):215–220CrossRefGoogle Scholar
  17. 17.
    Healey J, Picard R (2005) Detecting stress during real-world driving tasks using physiological sensors. IEEE Trans Intell Transp Syst 6(2):156–166CrossRefGoogle Scholar
  18. 18.
    Hedman E, Quigley K, Picard R (2012) Using ranked SCRs in ambulatory measurements: a new approach to distinguishing real-world salient events. In: Proceedings of the 52nd annual meeting of society for psychophysiological researchGoogle Scholar
  19. 19.
    Herwig U, Satrapi P, Schonfeldt-Lecuona C (2003) Using the international 10–20 EEG system for positioning of transcranial magnetic stimulation. Brain Topogr 16(2):95–99CrossRefGoogle Scholar
  20. 20.
    Hewig J, Hagemann D, Seifert J, Gollwitzer M, Naumann E, Bartussek D (2005) A revised film set for the induction of basic emotions. Cogn Emot 19(7):1095–1109CrossRefGoogle Scholar
  21. 21.
    Kaplan RM, Saccuzzo DR (2001) Psychological testing: principles, applications, and issues, 5th edn. WadsworthGoogle Scholar
  22. 22.
    Koelstra S, Muhl C, Soleymani M, Lee JS, Yazdani A, Ebrahimi T, Pun T, Nijholt A, Patras I (2012) DEAP: a database for emotion analysis; using physiological signals. IEEE Trans Affect Comput 3(1):18–31CrossRefGoogle Scholar
  23. 23.
    Lang P, Bradley M, Cuthbert B (2008) International affective picture system (IAPS): affective ratings of pictures and instruction manual. Tech. Rep. A-8, University of Florida, Gainesville, FLGoogle Scholar
  24. 24.
    Lewis M, Haviland-Jones JM, Barrett LF (eds) (2010) Handbook of emotions, 3rd edn. The Guilford PressGoogle Scholar
  25. 25.
    Medina LAS, Fred ALN (2010) Genetic algorithm for clustering temporal data—application to the detection of stress from ECG signals. In: Proc 2nd Int’l Conf. on Agents and Artificial Intelligence (ICAART), pp 135–142Google Scholar
  26. 26.
    Pantic M, Valstar M, Rademaker R, Maat L (2005) Web-based database for facial expression analysis. In: IEEE International Conference on Multimedia and Expo (ICME), pp 6–8Google Scholar
  27. 27.
    Picard RW (2000) Affective computing, 1st edn. The MIT PressGoogle Scholar
  28. 28.
    Rottenberg J, Johnson SL (eds) (2007) Emotion and psychopathology: bridging affective and clinical science. American Psychological Association (APA)Google Scholar
  29. 29.
    Silva H (2007) Feature selection in pattern recognition systems. MSc thesis, Instituto Superior T\(\acute{\rm e}\)cnico, Universidade T\(\acute{{\rm e}}\)cnica de LisboaGoogle Scholar
  30. 30.
    Silva H, Gamboa H, Fred A (2007) Applicability of lead V2 ECG measurements in biometrics. In: Proc. of the int’l eHealth, Telemedicine and Health ICT forum (Med-e-Tel), pp 177–180Google Scholar
  31. 31.
    Silva H, Gamboa H, Fred A (2007) One lead ECG based human identification with feature subspace ensembles. In: Proc. of the 5th int’l con. on machine learning and data mining, pp 770–783Google Scholar
  32. 32.
    Sneddon I, McRorie M, McKeown G, Hanratty J (2012) The belfast induced natural emotion database. IEEE Trans Affect Comput 3(1):32–41CrossRefGoogle Scholar
  33. 33.
    Soleymani M, Lichtenauer J, Pun T, Pantic M (2012) A multimodal database for affect recognition and implicit tagging. IEEE Trans Affect Comput 3(1):42–55CrossRefGoogle Scholar
  34. 34.
    Spreen O, Strauss E (1998) A compendium of neuropsychological tests: administration, norms, and commentary, 2nd edn. Oxford University Press, USAGoogle Scholar
  35. 35.
    Tolkmitt FJ, Scherer KR (1986) Effect of experimentally induced stress on vocal parameters. J Exp Psychol Hum Percept Perform 12(3):302–313CrossRefGoogle Scholar
  36. 36.
    Zeng Z, Pantic M, Roisman GI, Huang TS (2009) A survey of affect recognition methods: audio, visual, and spontaneous expressions. IEEE Trans Pattern Anal Mach Intell 31(1):39–58CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media New York 2013

Authors and Affiliations

  1. 1.CEFITEC - Centro de Física e Investigação TecnológicaFaculdade de Ciência e TecnologiaCaparicaPortugal
  2. 2.IT - Instituto de TelecomunicaçõesInstituto Superior TécnicoLisboaPortugal

Personalised recommendations