Abstract
Previous research has shown that the use of short film clips is one of the most successful and widely-used methods of emotion elicitation. Nevertheless, there is a high degree of audiovisual variation across film clips, resulting in a trade-off in terms of controllability. To address this complexity, the present study aimed to investigate the potential effects of sound and color on film clips’ emotion-elicitation levels. For this purpose, four different film clips were selected for each of the eight emotion categories: amusement, tenderness, calmness, anger, sadness, disgust, fear, and neutrality. All film clips were manipulated in terms of features of sound (sound versus silent) × color (color versus BW). In total, 128 film clips were tested online on various parameters: intensity of target emotion, valence, arousal, discreteness, and non-target emotions. The results revealed that sound was a mild contributing factor in increasing the degree of target emotions and evoking less boredom and more interest. However, color effects were less pronounced in emotion-elicitation using film clips. Furthermore, the study provided preliminary evidence that for most of the film clips, there was no significant difference in emotional reactivity in response to subtitled versus dubbed film clips. In conclusion, the current study showed that film clips (with a few exceptions) can evoke moderate to high levels of emotions independent of their bottom-up features, such as sound or color.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Data availability
The datasets generated and analyzed during the current study are available in the OSF repository [https://osf.io/3yftb/?view_only=64af7862f3034d9389f29ada8f680bb9].
Notes
In Gabert-Quillen et al. (2015), there were four BW film clips out of a total of 18. These BW film clips included Modern Times (amusement), Gentleman's Agreement (anger), Psycho (fear), and D.O.A. (surprise). In the study conducted by Schaefer et al. (2010), there were six BW film clips out of a total of 70. The BW film clips in this study were American History X (anger), Man Bites Dog (2) (anger), Schindler's List (2) (anger), Schindler's List (3) (anger), Schindler's List (1) (sadness), and Man Bites Dog (1) (disgust).
The elicitation index scores were calculated by averaging the degree and discreteness scores (Zupan & Eskritt, 2020). The degree score represents the intensity of the target emotion. It was calculated by converting the mean target emotion rating into a proportion-based score (i.e., mean target emotion rating divided by 8. It is important to note that in the current study, we used a-priori defined target emotions (e.g., amusement ratings for the amusement category) for this calculation, whereas Zupan and Eskritt (2020) used the highest rated emotion. The discreteness score represents the proportion of instances in which the target emotion was felt at least one point more intensely than the non-target emotions, relative to the total number of non-target emotions (Gabert-Quillen et al., 2015). Specifically, for each film clip version, the number of comparisons in which target emotion was at least 1 point higher than the non-target emotion was divided by the total number of comparisons between target and non-target emotions. This score was then averaged across all participants who had viewed that particular film clip.
In addition to the aforementioned advantages, subtitling may offer a minor advantage in terms of preserving the original sound effects. We observed that certain sound effects might be canceled in the dubbed versions (e.g., The sound of flies buzzing was absent in the dubbed version of the toilet scene from Trainspotting).
Using G*Power version 3.1 (Faul et al., 2009), we performed a power analysis based on a between-subjects factorial design (k = 16) and a small effect size (d = 0.2). It was found that the minimum sample size of 277 participants was needed to reach the desired power level (1-β = .80, α = .05).
In each emotion category, one of the film clips was from a widely-cited database (i.e., amusement: Hangover, Gabert-Quillen et al., 2015; tenderness: Forrest Gump, Schaefer et al., 2010; calmness: Pride and Prejudice, Gabert-Quillen et al., 2015; anger: My Bodyguard, Gross & Levenson, 1995; sadness: The Champ, Gross & Levenson, 1995; disgust: Trainspotting, Schaefer et al., 2010; fear: The Shining, Schaefer et al., 2010; neutrality: Hannah and Her Sisters, Hewig et al., 2005). The remaining three film clips in each category were the scenes that were obtained via a survey in a Turkish sample (Arıkan İyilikci et al., 2023). All film clips were validated in a Turkish sample by Arıkan İyilikci et al. (2023).
The items were amusement, anger, anxiety, boredom, calmness, confusion, desire, disgust, fear, guilt, happiness, interest, joy, love, pride, sadness, shame, surprise, tenderness, and unhappiness. The original questionnaire contains 18 items; but following Arıkan İyilikci et al. (2023) we modified the form by excluding two items (i.e., contempt, and embarrassment) and adding four new items (i.e., boredom, calmness, desire, and tenderness).
We put gender as a third variable into the 2 (sound) × 2 (color) ANOVAs. The results replicated previous studies that women, compared to men, reported higher levels of negative emotions in response to negatively-valenced film clips (e.g., Gabert-Quillen et al., 2015; Gross & Levenson, 1995; Hagemann et al., 1999). Additionally, we found that women reported higher levels of tenderness after watching tenderness-eliciting film clips, which is also consistent with prior research (e.g., Arıkan İyilikci et al., 2023; Lishner et al., 2011). However, there was no interaction between gender, sound, or color.
For instance, previous research has demonstrated that the silent version of the anger-eliciting film clip My Bodyguard from the set of Gross and Levenson (1995) evoked intense levels of anger (e.g., Hewig et al., 2005: M = 7.24, SD = 2.12 on a 0-9 scale). Similarly, the silent version of the sadness-eliciting film clip The Champ from the set of Gross and Levenson (1995) evoked above moderate level of sadness on a 0-9 scale (e.g., Hagemann et al., 1999: M = 6.64, SD = 2.35; Hewig et al., 2005: M = 7.21, SD = 2.07). When we compared the current ratings in response to these pre-existing stimuli with those obtained in previous research, we observed that our results were highly comparable: My Bodyguard, M = 7.44, SD = 0.98; and The Champ, M = 6.96, SD = 1.58 on a 0-8 scale. To our knowledge, the other film clips in the current study were not used as silent in previous research, thus it was not possible to provide comparisons for the remaining stimuli.
References
Ack Baraly, K. T., Muyingo, L., Beaudoin, C., Karami, S., Langevin, M., & Davidson, P. S. (2020). Database of emotional videos from Ottawa (DEVO). Collabra: Psychology, 6(1). https://doi.org/10.1525/collabra.180
Arıkan İyilikci, E., Boğa, M., Yüvrük, E., Özkılıç, Y., İyilikci, O., & Amado, S. (2023). An extended emotion-eliciting film clips set (EGEFILM): Assessment of emotion ratings for 104 film clips in a Turkish sample. Behavior Research Methods. https://doi.org/10.3758/s13428-022-02055-4
Bekhtereva, V., & Müller, M. M. (2017). Bringing color to emotion: The influence of color on attentional bias to briefly presented emotional images. Cognitive, Affective, & Behavioral Neuroscience, 17, 1028–1047. https://doi.org/10.3758/s13415-017-0530-z
Bisson, M., Van Heuven, W., Conklin, K., & Tunney, R. (2014). Processing of native and foreign language subtitles in films: An eye tracking study. Applied Psycholinguistics, 35(2), 399–418. https://doi.org/10.1017/S0142716412000434
Boğa, M., Koyuncu, M., Kaça, G., & Bayazıt, T. O. (2022). Comparison of emotion elicitation methods: 3 methods, 3 emotions, 3 measures. Current Psychology, 1–16. https://doi.org/10.1007/s12144-022-02984-5
Bolivar, V. J., Cohen, A. J., & Fentress, J. C. (1994). Semantic and formal congruency in music and motion pictures: Effects on the interpretation of visual action. Psychomusicology: A Journal of Research in Music Cognition, 13(1–2), 28–59. https://doi.org/10.1037/h0094102
Boyatzis, C. J., & Varghese, R. (1994). Children’s emotional associations with colors. The Journal of Genetic Psychology, 155(1), 77–85. https://doi.org/10.1080/00221325.1994.9914760
Bradley, M. M., & Lang, P. J. (1994). Measuring emotion: The self-assessment manikin and the semantic differential. Journal of Behavior Therapy and Experimental Psychiatry, 25(1), 49–59. https://doi.org/10.1016/0005-7916(94)90063-9
Cano, M. E., Class, Q. A., & Polich, J. (2009). Affective valence, stimulus attributes, and P300: color vs. black/white and normal vs. scrambled images. International Journal of Psychophysiology: Official Journal of the International Organization of Psychophysiology, 71(1), 17–24. https://doi.org/10.1016/j.ijpsycho.2008.07.016
Carvalho, S., Leite, J., Galdo-Álvarez, S., & Gonçalves, O. F. (2012). The emotional movie database (EMDB): A self-report and psychophysiological study. Applied Psychophysiology and Biofeedback, 37(4), 279–294. https://doi.org/10.1007/s10484-012-9201-6
Codispoti, M., Surcinelli, P., & Baldaro, B. (2008). Watching emotional movies: Affective reactions and gender differences. International Journal of Psychophysiology, 69(2), 90–95. https://doi.org/10.1016/j.ijpsycho.2008.03.004
Codispoti, M., De Cesarei, A., & Ferrari, V. (2012). The influence of color on emotional perception of natural scenes. Psychophysiology, 49(1), 11–16. https://doi.org/10.1111/j.1469-8986.2011.01284.x
Cohen, A. J. (1993). Associationism and musical soundtrack phenomena. Contemporary Music Review, 9, 163–178. https://doi.org/10.1080/07494469300640421
Cowen, A. S., & Keltner, D. (2017). Self-report captures 27 distinct categories of emotion bridged by continuous gradients. Proceedings of the National Academy of Sciences, 114(38), E7900–E7909. https://doi.org/10.1073/pnas.1702247114
Delorme, A., Richard, G., & Fabre-Thorpe, M. (2000). Ultra-rapid categorisation of natural scenes does not rely on colour cues: A study in monkeys and humans. Vision Research, 40(16), 2187–2200. https://doi.org/10.1016/S0042-6989(00)00083-3
Deng, Y., Chang, L., Yang, M., Huo, M., & Zhou, R. (2016). Gender differences in emotional response: Inconsistency between experience and expressivity. PloS One, 11(6), e0158666. https://doi.org/10.1371/journal.pone.0158666
Detenber, B. H., & Winch, S. P. (2001). The impact of color on emotional responses to newspaper photographs. Visual Communication Quarterly, 8(3), 4–14. https://doi.org/10.1080/15551390109363461
Detenber, B. H., Simons, R. F., & Reiss, J. E. (2000). The emotional significance of color in television presentations. Media Psychology, 2(4), 331–355. https://doi.org/10.1207/S1532785XMEP0204_02
Dominy, N., & Lucas, P. (2001). Ecological importance of trichromatic vision to primates. Nature, 410, 363–366. https://doi.org/10.1038/35066567
d’Ydewalle, G., & De Bruycker, W. (2007). Eye movements of children and adults while reading television subtitles. European Psychologist, 12(3), 196–205. https://doi.org/10.1027/1016-9040.12.3.196
d'Ydewalle, G., Van Rensbergen, J., & Pollet, J. (1987). Reading a message when the same message is available auditorily in another language: The case of subtitling. In Eye movements from physiology to cognition (pp. 313–321). Elsevier. https://doi.org/10.1016/B978-0-444-70113-8.50047-3
Ekman, P. (1992). Facial expressions of emotion: an old controversy and new findings. Philosophical Transactions of the Royal Society of London. Series B: Biological Sciences, 335(1273), 63–69. https://doi.org/10.1098/rstb.1992.0008
Ekman, P., & Friesen, W. V. (1971). Constants across cultures in the face and emotion. Journal of Personality and Social Psychology, 17(2), 124–129. https://doi.org/10.1037/h0030377
Ellis, R. J., & Simons, R. F. (2005). The impact of music on subjective and physiological indices of emotion while viewing films. Psychomusicology: A Journal of Research in Music Cognition, 19(1), 15–40. https://doi.org/10.1037/h0094042
Faul, F., Erdfelder, E., Buchner, A., & Lang, A.-G. (2009). Statistical power analyses using G*Power 3.1: Tests for correlation and regression analyses. Behavior Research Methods, 41, 1149–1160. https://doi.org/10.3758/BRM.41.4.1149
Fernández Megías, C., Pascual Mateos, J. C., Soler Ribaudi, J., & Fernández-Abascal, E. G. (2011). Validación española de una batería de películas para inducir emociones [Spanish validation of an emotion-eliciting set of films]. Psicothema, 23(4), 778–785.
Ferrer, R. A., Grenen, E. G., & Taber, J. M. (2015). Effectiveness of internet-based affect induction procedures: A systematic review and meta-analysis. Emotion, 15(6), 752–762. https://doi.org/10.1037/emo0000035
Frey, H. P., Honey, C., & König, P. (2008). What’s color got to do with it? The influence of color on visual attention in different categories. Journal of Vision, 8(14), 1–17. https://doi.org/10.1167/8.14.6
Gabert-Quillen, C. A., Bartolini, E. E., Abravanel, B. T., & Sanislow, C. A. (2015). Ratings for emotion film clips. Behavior Research Methods, 47(3), 773–787. https://doi.org/10.3758/s13428-014-0500-0
Ge, Y., Zhao, G., Zhang, Y., Houston, R. J., & Song, J. (2019). A standardised database of Chinese emotional film clips. Cognition and Emotion, 33(5), 976–990. https://doi.org/10.1080/02699931.2018.1530197
Gilman, T. L., Shaheen, R., Nylocks, K. M., Halachoff, D., Chapman, J., Flynn, J. J., & Coifman, K. G. (2017). A film set for the elicitation of emotion in research: A comprehensive catalog derived from four decades of investigation. Behavior Research Methods, 49(6), 2061–2082. https://doi.org/10.3758/s13428-016-0842-x
Gross, J. J., & Levenson, R. W. (1995). Emotion elicitation using films. Cognition & Emotion, 9(1), 87–108. https://doi.org/10.1080/02699939508408966
Grühn, D., & Sharifian, N. (2016). Lists of emotional stimuli. In Emotion measurement (pp. 145–164). Woodhead Publishing. https://doi.org/10.1016/B978-0-08-100508-8.00007-2
Hagemann, D., Naumann, E., Maier, S., Becker, G., Lürken, A., & Bartussek, D. (1999). The assessment of affective reactivity using films: Validity, reliability and sex differences. Personality and Individual Differences, 26(4), 627–639. https://doi.org/10.1016/S0191-8869(98)00159-7
Hewig, J., Hagemann, D., Seifert, J., Gollwitzer, M., Naumann, E., & Bartussek, D. (2005). A revised film set for the induction of basic emotions. Cognition and Emotion, 19(7), 1095. https://doi.org/10.1080/02699930541000084
Hull, R., Tosun, S., & Vaid, J. (2017). What’s so funny? Modelling incongruity in humour production. Cognition and Emotion, 31(3), 484–499. https://doi.org/10.1080/02699931.2016.1182363
Jenkins, L., & Andrewes, D. (2012). A new set of standardised verbal and non-verbal contemporary film stimuli for the elicitation of emotions. Brain Impairment, 13(2), 212–227. https://doi.org/10.1017/BrImp.2012.18
Joseph, D. L., Chan, M. Y., Heintzelman, S. J., Tay, L., Diener, E., & Scotney, V. S. (2020). The manipulation of affect: A meta-analysis of affect induction procedures. Psychological Bulletin, 146(4), 355–375. https://doi.org/10.1037/bul0000224
Jurásová, K., & Špajdel, M. (2013). Development and assessment of film excerpts used for emotion elicitation. Activitas Nervosa Superior Rediviva, 55(3), 135–140.
Kaya, N., & Epps, H. H. (2004). Relationship between color and emotion: A study of college students. College Student Journal, 38(3), 396–405.
Kock, M., & Louven, C. (2019). The power of sound design in a moving picture: An empirical study with emoTouch for iPad. Empirical Musicology Review, 13(3–4), 1325–148. https://doi.org/10.18061/emr.v13i3-4.6572
Koolstra, C. M., Peeters, A. L., & Spinhof, H. (2002). The pros and cons of dubbing and subtitling. European Journal of Communication, 17(3), 325–354. https://doi.org/10.1177/0267323102017003694
Kuzinas, A., Noiret, N., Bianchi, R., & Laurent, É. (2016). The effects of image hue and semantic content on viewer’s emotional self-reports, pupil size, eye movements, and skin conductance response. Psychology of Aesthetics, Creativity, and the Arts, 10(3), 360–371. https://doi.org/10.1037/a0040274
Leveridge, F., Mevel, P.-A., & Tsikandilakis, M. (2023). Emotional responses to aesthetically integrated and standard subtitles in a fantasy-thriller audiovisual context. Preprints. https://doi.org/10.13140/RG.2.2.27663.46246
Lishner, D. A., Batson, C. D., & Huss, E. (2011). Tenderness and sympathy: Distinct empathic emotions elicited by different forms of need. Personality and Social Psychology Bulletin, 37(5), 614–625. https://doi.org/10.1177/0146167211403157
Matamala, A., Perego, E., & Bottiroli, S. (2017). Dubbing versus subtitling yet again?: An empirical study on user comprehension and preferences in Spain. Babel, 63(3), 423–441. https://doi.org/10.1075/babel.63.3.07mat
Mauss, I. B., & Robinson, M. D. (2009). Measures of emotion: A review. Cognition and Emotion, 23(2), 209–237. https://doi.org/10.1080/02699930802204677
McHugo, G. J., Smith, C. A., & Lanzetta, J. T. (1982). The structure of self-reports of emotional responses to film segments. Motivation and Emotion, 6, 365–385. https://doi.org/10.1007/BF00998191
Meinel, L. S., & Bullerjahn, C. (2022). More horror due to specific music placement? Effects of film music on psychophysiological responses to a horror film. Psychology of Music, 50(6), 1837–1852. https://doi.org/10.1177/03057356211073478
Nothwang, H. G. (2016). Evolution of mammalian sound localization circuits: A developmental perspective. Progress in Neurobiology, 141, 1–24. https://doi.org/10.1016/j.pneurobio.2016.02.003
Perego, E., Del Missier, F., Porta, M., & Mosconi, M. (2010). The cognitive effectiveness of subtitle processing. Media Psychology, 13(3), 243–272. https://doi.org/10.1080/15213269.2010.502873
Perego, E., Del Missier, F., & Bottiroli, S. (2015). Dubbing versus subtitling in young and older adults: Cognitive and evaluative aspects. Perspectives, 23(1), 1–21. https://doi.org/10.1080/0907676X.2014.912343
Perego, E., Del Missier, F., & Stragà, M. (2018). Dubbing vs. subtitling: Complexity matters. Target. International Journal of Translation Studies, 30(1), 137–157. https://doi.org/10.1075/target.16083.per
Pessoa, D. M., Maia, R., de Albuquerque Ajuz, R. C., De Moraes, P. Z., Spyrides, M. H., & Pessoa, V. F. (2014). The adaptive value of primate color vision for predator detection. American Journal of Primatology, 76(8), 721–729. https://doi.org/10.1002/ajp.22264
Polzella, D. J., Hammar, S. H., & Hinkle, C. W. (2005). The effect of color on viewers’ ratings of paintings. Empirical Studies of the Arts, 23(2), 153–163. https://doi.org/10.2190/ECK9-C18E-RQN9-7WNE
Riniolo, T. C., & Capuana, L. J. (2022). Directly comparing subtitling and dubbing using Netflix: Examining enjoyment issues in the natural setting. Current Psychology, 41(7), 4252–4258. https://doi.org/10.1007/s12144-020-00948-1
Rottenberg, J., Ray, R. D., & Gross, J. J. (2007). Emotion elicitation using films. In J. A. Coan & J. J. B. Allen (Eds.), Handbook of emotion elicitation and assessment (pp. 9–28). Oxford University Press. https://doi.org/10.1080/02699939508408966
Russell, J. A. (1980). A circumplex model of affect. Journal of Personality and Social Psychology, 39(6), 1161–1178. https://doi.org/10.1037/h0077714
Sato, W., Noguchi, M., & Yoshikawa, S. (2007). Emotion elicitation effect of films in a Japanese sample. Social Behavior and Personality, 35(7), 863–874. https://doi.org/10.2224/sbp.2007.35.7.863
Schaefer, A., Nils, F., Sanchez, X., & Philippot, P. (2010). Assessing the effectiveness of a large database of emotion-eliciting films: A new tool for emotion researchers. Cognition and Emotion, 24(7), 1153–1172. https://doi.org/10.1080/02699930903274322
Smith, C. A., & Ellsworth, P. C. (1985). Patterns of cognitive appraisal in emotion. Journal of Personality and Social Psychology, 48(4), 813–838. https://doi.org/10.1037/0022-3514.48.4.813
Szarkowska, A., & Gerber-Morón, O. (2018). Viewers can keep up with fast subtitles: Evidence from eye movements. PloS One, 13(6), e0199331. https://doi.org/10.1371/journal.pone.0199331
Tan, S. L., Spackman, M. P., & Bezdek, M. A. (2007). Viewers’ interpretations of film characters’ emotions: Effects of presenting film music before or after a character is shown. Music Perception, 25(2), 135–152. https://doi.org/10.1525/mp.2007.25.2.135
Thayer, J. F., & Levenson, R. W. (1983). Effects of music on psychophysiological responses to a stressful film. Psychomusicology: A Journal of Research in Music Cognition, 3(1), 44–52. https://doi.org/10.1037/h0094256
Tomarken, A. J., Davidson, R. J., & Henriques, J. B. (1990). Resting frontal brain asymmetry predicts affective responses to films. Journal of Personality and Social Psychology, 59(4), 791–801. https://doi.org/10.1037/0022-3514.59.4.791
Westermann, R., Spies, K., Stahl, G., & Hesse, F. W. (1996). Relative effectiveness and validity of mood induction procedures: A meta-analysis. European Journal of Social Psychology, 26(4), 557–580. https://doi.org/10.1002/(SICI)1099-0992(199607)26:4%3C557::AID-EJSP769%3E3.0.CO;2-4
Weymar, M., Löw, A., Melzig, C. A., & Hamm, A. O. (2009). Enhanced long-term recollection for emotional pictures: Evidence from high-density ERPs. Psychophysiology, 46(6), 1200–1207. https://doi.org/10.1111/j.1469-8986.2009.00869.x
Wissmath, B., Weibel, D., & Groner, R. (2009). Dubbing or subtitling?: Effects on spatial presence, transportation, flow, and enjoyment. Journal of Media Psychology: Theories, Methods, and Applications, 21(3), 114–125. https://doi.org/10.1027/1864-1105.21.3.114
Zupan, B., & Eskritt, M. (2020). Eliciting emotion ratings for a set of film clips: A preliminary archive for research in emotion. The Journal of Social Psychology, 160(6), 768–789. https://doi.org/10.1080/00224545.2020.1758016
Acknowledgements
The authors would like to thank Dr. Belgüzar Nilay Türkan and Dr. Yıldız Özkılıç for their help with the data collection.
Author information
Authors and Affiliations
Corresponding author
Ethics declarations
Ethical approval
The study was conducted in full accordance with the approved ethics procedure granted by the Scientific Research and Publication Ethics Committee of Ege University (EGEBAYEK).
Informed consent
All participants gave voluntary consent to take part in the study and informed consent was required from all individual participants included in the study.
Conflicts of interest
We have no conflicts of interest to disclose.
Additional information
Publisher's note
Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.
Supplementary Information
Below is the link to the electronic supplementary material.
Rights and permissions
Springer Nature or its licensor (e.g. a society or other partner) holds exclusive rights to this article under a publishing agreement with the author(s) or other rightsholder(s); author self-archiving of the accepted manuscript version of this article is solely governed by the terms of such publishing agreement and applicable law.
About this article
Cite this article
İyilikci, E.A., Demirel, A., Işık, F. et al. How do sound and color features affect self-report emotional experience in response to film clips?. Curr Psychol 43, 10185–10216 (2024). https://doi.org/10.1007/s12144-023-05127-6
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s12144-023-05127-6