Abstract
Various emotion elicitation methods are used in many studies assessing the interaction of emotion and cognition. However, most of the experimental and meta-analysis studies comparing emotion elicitation methods have examined the success of emotion evoking in terms of valence and arousal. There are few experimental studies dealing with comparisons within discrete emotion approach. The present study was mainly aimed to determine the most effective emotion elicitation method for discrete emotions. Therefore, three emotion elicitation methods (film, IAPS, imagery) used to evoke three discrete emotions (fear, disgust, happiness) were examined in the framework of discrete, dimensional, and response component model of emotion. Within-subject design was used to examine the coherence of emotional response components such as subjective experience, facial expression, and physiological responses. The results suggested that film was the most effective among all emotion elicitation methods. All emotions elicited using the film method were accompanied not only by intense subjective experiences but also by physiological responses. IAPS and imagination methods successfully elicited subjective experiences of all emotions, except for fear – IAPS condition. Besides all, there was a lack of coherence between the components of emotional responses, and the most explicit results were obtained from subjective experiences.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data Availability
Data is available via the Open Science Framework: https://osf.io/n5q98/. In this study, most materials were taken from specific films or picture databases. Therefore, no materials can be shared but only the name and number of the stimuli.
Notes
Fear: 1022, 1050, 1080, 1300, 1321, 1525, 1932, 6231, 6260, 6300. Disgust: 3250, 7360, 7380, 9300, 9301, 9302, 9320, 9322 9325, 9326. Happiness: 1463, 2070, 2091, 2158, 2340, 2344, 2352, 2395, 2550, 8380.
The effect size (r) was z-score divided from square root number of total observations calculated for statistically significant Wilcoxon signed-rank test results (Fields, 2017).
References
Arıkan İyilikci, E., & Amado, S. (2018). The uncertainty appraisal enhances the prominent deck B effect in the Iowa gambling task. Motivation and Emotion, 42(1), 1–16. https://doi.org/10.1007/s11031-017-9643-5
Arıkan İyilikci, E.., Boğa, M., Yüvrük, E., Özkılıç, Y., İyilikci, O., & Amado, S. (Submitted). EGEFILM Emotion Eliciting Film Clips Set.
Bernat, E., Patrick, C. J., Benning, S. D., & Tellegen, A. (2006). Effects of picture content and intensity on affective physiological response. Psychophysiology, 43(1), 93–103.
Boğa, M., Günay, B., & Kapucu, A. (2021). The influence of discrete negative and positive stimuli on recognition memory of younger vs older adults. Experimental Aging Research, 47(1), 21–39.https://doi.org/10.1080/0361073X.2020.1843894
Bradley, M. M., & Lang, P. J. (2000). Measuring emotion: Behavior, feeling, and physiology. In R. D. Lane & L. Nadel (Eds.), Cognitive neuroscience of emotion (pp. 242–276). Oxford University Press.
Bradley, M. M., & Lang, P. J. (2007). The international affective picture system (IAPS) in the study of emotion and attention. In J. A. Coan & J. J. B. Allen (Eds.), Series in affective science. Handbook of emotion elicitation and assessment (pp. 29–46). Oxford University Press.
Cabral, J. C. C., Tavares, P. D. S., Weydmann, G. J., das Neves, V. T., & de Almeida, R. M. M. (2018). Eliciting negative affects using film clips and real-life methods. Psychological Reports, 121(3), 527–547. https://doi.org/10.1177/0033294117730844
den Uyl, M. J., & van Kuilenburg, H. (2005). The FaceReader: Online facial expression recognition. Proceedings of Measuring Behavior, 30(2), 589–590.
Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124–129. https://doi.org/10.1037/h0030377
Evers, C., de Ridder, D. T., & Adriaanse, M. A. (2009). Assessing yourself as an emotional eater: Mission impossible? Health Psychology, 28(6), 717. https://doi.org/10.1037/a0016700
Evers, C., Hopp, H., Gross, J. J., Fischer, A. H., Manstead, A. S. R., & Mauss, I. B. (2014). Emotion response coherence: A dual-process perspective. Biological Psychology, 98, 43–49. https://doi.org/10.1016/j.biopsycho.2013.11.003
Fanti, K. A., Kyranides, M. N., & Panayiotou, G. (2017). Facial reactions to violent and comedy films: Association with callous-unemotional traits and impulsive aggression. Cognition and Emotion, 31(2), 209–224. https://doi.org/10.1080/02699931.2015.1090958
Gabert-Quillen, C. A., Bartolini, E. E., Abravanel, B. T., & Sanislow, C. A. (2015). Ratings for emotion film clips. Behavior Research Methods, 47(3), 773–787. DOI: 10.3758/s13428-014-0500-0
Ge, Y., Zhao, G., Zhang, Y., Houston, R. J., & Song, J. (2019). A standardised database of Chinese emotional film clips. Cognition and Emotion, 33(5), 976–990. https://doi.org/10.1080/02699931.2018.1530197
Gilpin, G., Gain, J., & Lipinska, G. (2021). The physiological signature of sadness: A comparison between text, film and virtual reality. Brain and Cognition, 152, 105734. https://doi.org/10.1016/j.bandc.2021.105734
Gollnisch, G., & Averill, J. R. (1993). Emotional imagery: Strategies and correlates. Cognition and Emotion, 7(5), 407–429. https://doi.org/10.1080/02699939308409196
Gross, J. J., & Levenson, R. W. (1993). Emotional suppression: Physiology, self-report, and expressive behavior. Journal of Personality and Social Psychology, 64(6), 970. https://doi.org/10.1037/0022-3514.64.6.970
Gross, J. J., & Levenson, R. W. (1995). Emotion elicitation using films. Cognition and Emotion, 9(1), 87–108. https://doi.org/10.1080/02699939508408966
Grühn, D., & Sharifian, N. (2016). Lists of emotional stimuli. In Herbert L. Meiselman (Ed.), Emotion measurement (pp. 145–164). Woodhead Publishing. https://doi.org/10.1016/B978-0-08-100508-8.00007-2
Jabbi, M., Bastiaansen, J., & Keysers, C. (2008). A common anterior insula representation of disgust observation, experience and imagination shows divergent functional connectivity pathways. PLoS One, 3(8), e2939. https://doi.org/10.1371/journal.pone.0002939
Jallais, C., & Gilet, A. L. (2010). Inducing changes in arousal and valence: Comparison of two mood induction procedures. Behavior Research Methods, 42(1), 318–325. https://doi.org/10.3758/BRM.42.1.318
Jang, E. H., Byun, S., Park, M. S., & Sohn, J. H. (2019). Reliability of physiological responses induced by basic emotions: A pilot study. Journal of Physiological Anthropology, 38(1), 15. https://doi.org/10.1186/s40101-019-0209-y.
Jenkins, L. M., & Andrewes, D. G. (2012). A new set of standardised verbal and non- verbal contemporary film stimuli for the elicitation of emotions. Brain Impaırment, 13(2), 212–227. https://doi.org/10.1017/BrImp.2012.18
Joseph, D. L., Chan, M. Y., Heintzelman, S. J., Tay, L., Diener, E., & Scotney, V. S. (2020). The manipulation of affect: A meta-analysis of affect induction procedures. Psychological Bulletin, 146(4), 355-375. https://doi.org/10.1037/bul0000224
Keltner, D., & Cordaro, D. T. (2017). Understanding multimodal emotional expressions: Recent advances in basic emotion theory. In J.-M. Fernández-Dols & J. A. Russell (Eds.), The science of facial expression (pp. 57–75). Oxford University Press.https://doi.org/10.1093/acprof:oso/9780190613501.001.0001
Kreibig, S. D. (2010). Autonomic nervous system activity in emotion: A review. Biological Psychology, 84(3), 394–421. https://doi.org/10.1016/j.biopsycho.2010.03.010.
Lang, P. J., Kozak, M. J., Miller, G. A., Levin, D. N., & McLean Jr., A. (1980). Emotional imagery: Conceptual structure and pattern of somato-visceral response. Psychophysiology, 17(2), 179–192. https://doi.org/10.1016/j.biopsycho.2010.03.010.
Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (2008). International affective picture system (IAPS): Affective ratings of pictures and instruction manual. Technical report A-8, : University of Florida.
Larsen, J. T., Norris, C. J., & Cacioppo, J. T. (2003). Effects of positive and negative affect on electromyographic activity over zygomaticus major and corrugator supercilii. Psychophysiology, 40(5), 776–785. https://doi.org/10.1111/1469-8986.00078
Lench, H. C., Flores, S. A., & Bench, S. W. (2011). Discrete emotions predict changes in cognition, judgment, experience, behavior, and physiology: A meta-analysis of experimental emotion elicitations. Psychological Bulletin, 137(5), 834–855. https://doi.org/10.1037/a0024244
Lieberman, M. D. (2019). Boo! The consciousness problem in emotion. Cognition and Emotion, 33(1), 24–30. https://doi.org/10.1080/02699931.2018.1515726
Marcusson-Clavertz, D., Kjell, O. N., Persson, S. D., & Cardeña, E. (2019). Online validation of combined mood induction procedures. PLoS One, 14(6), e0217848. https://doi.org/10.1371/journal.pone.0217848
Mauss, B., & Robinson, M. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209–237. https://doi.org/10.1080/02699930802204677
Mauss, I. B., Levenson, R. W., McCarter, L., Wilhelm, F. H., & Gross, J. J. (2005). The tie that binds? Coherence amongemotion experience, behavior, and physiology. Emotion, 5(2), 175–190. https://doi.org/10.1037/1528-3542.5.2.175
Mikels, J. A., Fredrickson, B. L., Larkin, G. R., Lindberg, C. M., Maglio, S. J., & Reuter-Lorenz, P. A. (2005). Emotional category data on images from the international affective picture system. Behavior Research Methods, 37(4), 626–630. https://doi.org/10.3758/BF03192732
Novák, O., & Šolcová, I. P. (2017). Films VS pictures in experimental emotion elicitation. Poster, ICPS (International convention of psychological science), Vienna, Austria. https://doi.org/10.13140/RG.2.2.20408.80644.
Panksepp, J. (1998). Affective neuroscience: The foundations of human and animal emotions. Oxford University Press.
Reisenzein, R. (2000). Exploring the strength of association between the components of emotion syndromes: The case of surprise. Cognition and Emotion, 14(1), 1–38. https://doi.org/10.1080/026999300378978
Reisenzein, R. (2019). Cognition and emotion: A plea for theory. Cognition and Emotion, 33(1), 109–118. https://doi.org/10.1080/02699931.2019.1568968
Reisenzein, R., Studtmann, M., & Horstmann, G. (2013). Coherence between emotion and facial expression: Evidence from laboratory experiments. Emotion Review, 5(1), 16–23. https://doi.org/10.1177/1754073912457228
Rottenberg, J., Ray, R. D., & Gross, J. J. (2007). Emotion elicitation using films. In J. A. Coan & J. J. B. Allen (Eds.), Handbook of emotion elicitation and assessment (pp. 9–28). Oxford University Press.
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161. https://doi.org/10.1037/h0077714
Russell, J. A., & Barrett, L. F. (1999). Core affect, prototypical emotional episodes, and other things called emotion: Dissecting the elephant. Journal of Personality and Social Psychology, 76(5), 805. https://doi.org/10.1037/0022-3514.76.5.805
Schaefer, A., Nils, F., Sanchez, X., & Philippot, P. (2010). Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers. Cognition and Emotion, 24(7), 1153–1172. https://doi.org/10.1080/02699930903274322
Schneider, W., Eschman, A., & Zuccolotto, A. (2002). E-prime: User’s guide. Psychology Software Incorporated.
Sharman, L., & Dingle, G. A. (2015). Extreme metal music and anger processing. Frontiers in Human Neuroscience, 9, 272. https://doi.org/10.3389/fnhum.2015.00272
Siedlecka, E., & Denson, T. F. (2019). Experimental methods for inducing basic emotions: A qualitative review. Emotion Review, 11(1), 87–97. https://doi.org/10.1177/1754073917749016
Smillie, L. D., Cooper, A. J., Wilt, J., & Revelle, W. (2012). Do extraverts get more bang for the buck? Refining the affective-reactivity hypothesis of extraversion. Journal of Personality and Social Psychology, 103(2), 306. https://doi.org/10.1037/a0028372
Smith, E. E., & Kosslyn, S. M. (2009). Cognitive psychology: Mind and brain. Pearson International Edition.
Söylemez, S., Koyuncu, M., Wolf, O. T., & Türkan, B. N. (2022). How Shame and Guilt Influence Perspective Taking: A Comparison of Turkish and German Cultures, Journal of Cognition and Culture, 22(1-2), 20-40. doi: https://doi.org/10.1163/15685373-12340123
Suhr, Y. T. (2017). FaceReader, a promising instrument for measuring facial emotion expression? A comparison to facial electromyography and self-reports (Master's thesis).
Uhrig, M. K., Trautmann, N., Baumgärtner, U., Treede, R. D., Henrich, F., Hiller, W., & Marschall, S. (2016). Emotion elicitation: A comparison of pictures and films. Frontiers in Psychology, 7, 180. https://doi.org/10.3389/fpsyg.2016.00180
Van Diest, I., Winters, W., Devriese, S., Vercamst, E., Han, J. N., Van de Woestijne, K. P., & Van den Bergh, O. (2001). Hyperventilation beyond fight/flight: Respiratory responses during emotional imagery. Psychophysiology, 38(6), 961–968. https://doi.org/10.1111/1469-8986.3860961
Vrana, S. R., & Rollock, D. (2002). The role of ethnicity, gender, emotional contetnt, and contextual differences in physiological, expressive, and self-reported emotional responses to imagery. Cognition and Emotion, 16(1), 165–192. https://doi.org/10.1080/02699930143000185
Westermann, R., Spies, K., Stahl, G., & Hesse, F. W. (1996). Relative effectiveness and validity of mood induction procedures: A meta-analysis. European Journal of Social Psychology, 26(4), 557–580. https://doi.org/10.1002/(SICI)1099-0992(199607)26:4<557::AID-EJSP769>3.0.CO;2-4
Wu, Y., Gu, R., Yang, Q., & Luo, Y. (2019). How do amusement, anger and fear influence heart rate and heart rate variability? Frontiers in Neuroscience, 13, 1131. https://doi.org/10.3389/fnins.2019.01131
Yüvrük, E., Kapucu, A., & Amado, S. (2020). The effects of emotion on working memory: Valence versus motivation. Acta Psychologica, 202, 102983. https://doi.org/10.3389/fnins.2019.01131
Zhang, X., Yu, H. W., & Barrett, L. F. (2014). How does this make you feel? A comparison of four affect induction procedures. Frontiers in Psychology, 5, Article 689. https://doi.org/10.3389/fpsyg.2014.00689.
Zupan, B., & Babbage, D. R. (2017). Film clips and narrative text as subjective emotion elicitation techniques. The Journal of Social Psychology, 157(2), 194–210. https://doi.org/10.1080/00224545.2016.1208138
Zupan, B., & Eskritt, M. (2020). Eliciting emotion ratings for a set of film clips: A preliminary archive for research in emotion. The Journal of Social Psychology, 160(6), 768–789. https://doi.org/10.1080/00224545.2020.1758016
Acknowledgements
We would like to thank Prof. Dr. Nevin Yildirim Koyuncu from English Literature Department at Ege University for her editing contributions of the manuscript. We thank Emine Aktaş and Yağızcan Kurt for their valuable contributions to this project.
Funding
This study was supported by Projects 15-EDB-022 and 16-EDB-019 at Ege University, Scientific Research Center, Projects Coordination Unit.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Informed Consent
Informed consent was obtained from all individual participants included in the study.
Conflicting Interests
The authors declare that they have no confict of interest. This research is conducted within the context of a master’s thesis by the first author MB.
Additional information
Publisher’s Note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Appendices
Appendix 1. Pilot study for IAPS (The average ratings of subjective experiences)
Intensity of Target Emotion (0–8) | Valence &Arousal (1–9) | ||||||
|---|---|---|---|---|---|---|---|
IAPS No | N | Fear M (SD) | Disgust M (SD) | Happiness M (SD) | Valence M (SD) | Arousal M (SD) | |
Fear | 1022 | 20 | 4.1 (3.3) | 2.5 (3.4) | 0.2 (0.7) | 3.1 (3.3) | 5.1 (1.9) |
1050 | 20 | 4.8 (2.7) | 2.1 (3.1) | 0.3 (1.0) | 2.9 (1.4) | 5.8 (1.5) | |
1080 | 20 | 4.1 (3.4) | 2.6 (3.0) | 0.2 (0.8) | 3.3 (1.3) | 5.3 (1.9) | |
1300 | 20 | 4.7 (3.2) | 3.8 (3.7) | 0.0 (0.0) | 1.8 (0.9) | 6.5 (1.9) | |
1321 | 20 | 4.1 (3.1) | 0.4 (0.8) | 0.5 (1.5) | 4.3 (1.1) | 4.5 (1.9) | |
1525 | 20 | 5.3 (2.5) | 1.5 (2.4) | 0.1 (0.3) | 2.7 (1.3) | 5.7 (1.7) | |
1932 | 20 | 4.4 (3.1) | 1.2 (2.6) | 0.2 (0.8) | 3.9 (1.3) | 5.2 (1.7) | |
6231 | 20 | 4.1 (3.1) | 1.3 (2.9) | 0.1 (0.3) | 3.5 (2.3) | 5.2 (1.7) | |
6260 | 20 | 3.8 (2.9) | 1.2 (2.8) | 0.6 (1.6) | 3.5 (1.9) | 5.7 (1.5) | |
6300 | 20 | 4.5 (2.6) | 1.1 (2.5) | 0.1 (0.5) | 2.9 (1.3) | 5.7 (1.5) | |
Disgust | 3250 | 20 | 2.1 (2.5) | 6.1 (2.8) | 0.1 (0.2) | 2.2 (1.2) | 6.0 (1.7) |
7360 | 20 | 0.6 (1.6) | 5.7 (2.7) | 0.0 (0.0) | 3.6 (1.2) | 5.0 (1.4) | |
7380 | 20 | 0.8 (2.1) | 6.9 (1.2) | 0.0 (0.0) | 2.7 (1.1) | 5.6 (1.6) | |
9300 | 20 | 1.1 (2.5) | 6.9 (2.1) | 0.0 (0.0) | 1.7 (0.8) | 5.9 (1.7) | |
9301 | 20 | 0.7 (1.9) | 7.5 (0.7) | 0.0 (0.0) | 1.3 (0.6) | 6.5 (1.3) | |
9302 | 20 | 0.0 (2.3) | 7.5 (0.6) | 0.0 (0.0) | 1.4 (0.7) | 6.1 (1.7) | |
9320 | 20 | 0.3 (1.3) | 7.7 (0.8) | 0.0 (0.0) | 1.5 (0.7) | 5.8 (1.9) | |
9322 | 20 | 0.3 (1.1) | 6.9 (1.4) | 0.0 (0.0) | 1.8 (0.8) | 5.5 (1.6) | |
9325 | 20 | 0.8 (2.1) | 7.4 (1.1) | 0.0 (0.0) | 1.5 (0.9) | 6.2 (1.3) | |
9326 | 20 | 0.2 (0.8) | 6.9 (1.3) | 0.0 (0.0) | 1.9 (1.2) | 5.1 (1.7) | |
Happiness | 1463 | 20 | 0.0 (0.0) | 0.0 (0.0) | 6.1 (2.1) | 7.6 (1.6) | 4.4 (2.4) |
2070 | 20 | 0.0 (0.0) | 0.0 (0.0) | 6.5 (2.1) | 7,9 (2.1) | 3.4 (2.7) | |
2091 | 20 | 0.2 (0.7) | 0.0 (0.0) | 6.9 (1.1) | 7.8 (1.0) | 4.2 (2.8) | |
2158 | 20 | 0.0 (0.0) | 0.0 (0.0) | 6.4 (1.4) | 7.0 (1.0) | 4.1 (2.2) | |
2340 | 20 | 0.0 (0.0) | 0.0 (0.0) | 6.7 (1.1) | 7.8 (1.0) | 4.1 (2.7) | |
2344 | 20 | 0.0 (0.0) | 0.1 (0.5) | 6.4 (1.9) | 6.9 (0.9) | 4.7 (1.8) | |
2352 | 20 | 0.1 (0.5) | 0.0 (0.0) | 6.5 (1.9) | 7.5 (1.5) | 4.3 (1.9) | |
2395 | 20 | 0.0 (0.0) | 0.0 (0.0) | 6.0 (1.9) | 7.0 (1.3) | 4.1 (1.8) | |
2550 | 20 | 0.0 (0.0) | 0.0 (0.0) | 7.1 (1.2) | 8.0 (1.2) | 3.0 (2.2) | |
8380 | 20 | 0.0 (0.0) | 0.0 (0.0) | 6.3 (1.5) | 6.8 (0.7) | 3.3 (1.4) | |
Appendix 2. Pilot study for Imagination (The average ratings of subjective experiences)
Intensity of Target Emotion | Valence &Arousal | Imagination | |||||||
|---|---|---|---|---|---|---|---|---|---|
(0–8) | (1–9) | (1–9) | (1–5) | ||||||
N | Fear M (SD) | Disgust M (SD) | Happiness M (SD) | Valence M (SD) | Arousal M (SD) | Vividness M (SD) | Success M (SD) | ||
Fear | Elevator | 11 | 6.4 (2.5) | 0.0 (0.0) | 0.1 (0.3) | 2.4 (1.4) | 7.0 (2.2) | 6.9 (1.9) | 3.6 (1.3) |
Burgler | 10 | 7.7 (0.7) | 0.4 (1.3) | 0.8 (2.5) | 1.2 (0.6) | 8.0 (2.5) | 8.3 (0.9) | 3.9 (0.7) | |
Disgust | Drink | 8 | 0.0 (0.0) | 7.0 (1.0) | 1.1 (1.4) | 1.9 (1.0) | 4.3 (1.9) | 6.9 (1.4) | 3.6 (1.1) |
Disgusting food | 11 | 2.6 (2.9) | 7.5 (0.8) | 1.3 (2.3) | 2.4 (1.9) | 6.8 (2.7) | 7.9 (1.3) | 3.7 (0.6) | |
Happiness | Family | 11 | 0.0 (0.0) | 0.0 (0.0) | 7.6 (0.5) | 8.4 (1.0) | 5.6 (2.6) | 7.2 (1.6) | 4.4 (0.9) |
Holiday | 11 | 0.0 (0.0) | 0.0 (1.2) | 7.8 (0.5) | 8.5 (0.9) | 5.8 (2.6) | 7.5 (1.1) | 4.3 (0.7) | |
Rights and permissions
About this article
Cite this article
Boğa, M., Koyuncu, M., Kaça, G. et al. Comparison of emotion elicitation methods: 3 methods, 3 emotions, 3 measures. Curr Psychol 42, 18670–18685 (2023). https://doi.org/10.1007/s12144-022-02984-5
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12144-022-02984-5