Abstract
Self-reported measures are widely used to measure consumers’ emotional responses to advertising stimuli or consumption-related experiences, and are a consistently popular method for practitioners and researchers. There is, however, a problem known as “cognitive bias” which often arises from self-reported measures. Consequently, several researchers highlight the demand for the measurement of emotion to go beyond self-reported measures, and call for collaboration with other research fields to advance consumer behavior research in the study of emotion. This research collaborates with researchers in the field of human-computer interaction and suggests an alternative method: the Voice Emotion Response in Mandarin Chinese, which is one of the most widely spoken languages in the world. The findings show that the Voice Emotion Response, as compared to self-report, is more strongly related to recall. Preliminary outcomes reveal that this approach can potentially enhance the effectiveness of measuring emotions.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ambler, T. (2000). Persuasion, pride and prejudice: how ads work. International Journal of Advertising, 19, 299–315.
Baggett, H. L., Saab, P. G., & Carver, C. S. (1996). Appraisal, coping, task performance, and cardiovascular responses during the evaluated speaking task. Personality and Social Psychology Bulletin, 22(5), 483–494.
Bagozzi, R. P., Gopinath, M., & Nyer, P. U. (1999). The role of emotions in marketing. Journal of the Academy of Marketing Science, 27, 184–206.
Banse, R., & Scherer, K. R. (1996). Acoustic profiles in vocal emotion expression. Journal of Personality and Social Psychology, 70(3), 614–636.
Banziger, T., & Scherer, K. R. (2005). The role of intonation in emotional expressions. Speech Communication, 46, 252–267.
Boush, D. M. (1993). How advertising slogans can prime evaluations of brand extensions. Psychology and Marketing, 10(1), 67–78.
Breazeal, C. (2003). http://robotic.media.mit.edu/projects/Leonardo/Leo-robot.html.
Burkhardt, F., & Sendlmeier, W.F. (2000). Verification of acoustical correlates of emotional speech using formant-synthesis. In SpeechEmotion-2000, 151–156.
Bush, G., Luu, P., & Posner, M. I. (2000). Cognitive and emotional influences in anterior cingulate cortex. Trends in Cognitive Sciences, 4(6), 215–222.
Cacioppo, J. T., & Petty, R. E. (1983). Social psychophysiology. New York: The Guildford Press.
Cacioppo, J. T., Petty, R. E., Losch, M. E., & Kim, H. S. (1986). Electromyographic activity over facial muscle regions can differentiate the valence and intensity of affective reactions. Journal of Personality and Social Psychology, 50, 260–268.
Calvo, M. G., & Eysenck, M. W. (1998). Cognitive bias to internal sources of information in anxiety. International Journal of Psychology, 33(4), 287–299.
Chamberlain, L., & Broderick, A. J. (2007). The application of physiological observation methods to emotion research. Qualitative Market Research, 10, 199–216.
Chiang Blair, T., & Chiou, S.-C. (2014). The impact of brand knowledge on consumers of different genders and from different cultures. Asia pacific Management Review, 19(1), 47–59.
Chien, C. S., Wang, W.-C., Cheng, Y.-M., Pao, T.-L., Chen, Y.-T., Yeh, J.-H., & Moutinho, L. (2007). Applying recognition of emotions in speech to extend the borders of brand slogan research. Portuguese Journal of Management Studies, 12(2), 115–131.
Coricelli, G., Dolan, R. J., & Sirigu, A. (2007). Brain, emotion and decision making: the paradigmatic example of regret. Trends in Cognitive Sciences, 11(6), 258–265.
Critchley, H. D. (2002). Book review: electrodermal responses: what happens in the brain. The Neuroscientist, 8, 132–142.
Davis, D. F., Golicic, S. L., Boerstler, C. N., Choi, S., & Oh, H. (2013). Does marketing research suffer from methods myopia? Journal of Business Research, 66, 1245–1250.
Dawson, M.E., Schell, A.M., Filion, D.L. (2000). The electrodermal system. In Handbook of psychophysiology, J. Ekman, P., & Friesen, W.V. (1975). Unmasking the face. A guide to recognizing emotions from facial clues. New Jersey: Prentice-Hall.
Ekman, P., & Friesen, W. V. (1975). Unmasking the face (A guide to recognizing emotions from facial clues). New Jersey: Prentice-Hall.
Fiske, S., & Taylor, S.E. (1991). Social Cognition second edition (New York: McGraw Hill).
Hazlett, R. L., & Hazlett, S. Y. (1999). Emotional response to television commercials: facial emg vs. self-report. Journal of Advertising Research, 39, 7–23.
Helliker, K. (2006). This is your brain on a strong brand: MRIs show even insurers can ecite. Wall Street Journal, November 28(B1-B4).
Hopkins, R., & Fletcher, J. E. (1994). Electrodermal measurement: particularly effective for forecasting message influence on sales appeal. In A. Lang (Ed.), Measuring psychological responses to media (pp. 113–132). Hillsdale: Lawrence Erlbaum.
Hupp, O., Groppel-Klein, A., Dieckmann, A., Broeckelmann, P., & Walter, K. (2008). Beyond verbal scales: measurement of emotions in advertising effectiveness Research. Yearbook of Marketing and Consumer Research, 6, 72–99.
John, O. P., & Robins, R. W. (1994). Accuracy and bias in self-perception: individual differences, self-enhancement and the role of narcissism. Journal of Personality and Social Psychology, 66, 206–219.
Kelly, M. (2002). The science of shopping. http://www.cbc.ca/marketplace/pre-2007/files/money/science_shopping/index.html [On-line].
King, M. F., & Bruner, G. C. (2000). Social desirability bias: a neglected aspect of validity testing. Psychology and Marketing, 17, 79–103.
Klebba, J. M. (1985). Physiological measures of research: a review of brain activity, electrodermal response, pupil dilation, and voice analysis methods and studies. Current Issues and Research in Advertising, 8, 53–76.
LaBarbera, P. A., & Tucciarone, J. D. (1995). GSR reconsidered: a behavior-based approach to evaluating and improving the sales potency of advertising. Journal of Advertising Research, 35, 33–53.
Lang, A. (1990). Involuntary attention and physiological arousal evoked by structural features and emotional content in tv commercials. Communication Research, 17, 275–299.
Lang, P. J. (2010). Emotion and motivation: toward consensus definitions and a common research purpose. Emotion Review, 2(3), 229–233.
Lang, A., Borse, J., Wise, K., & David, P. (2002). Captured by the world wide web: orienting to structural and content features of computer-presented information. Communication Research, 29, 215–245.
Lazarus, R. S. (1991). Emotion and adaption. New York: University Press.
LeDoux, J. E. (1996). The emotional brain. New York: Simon and Schuster.
Lee, N., Broderick, A. J., & Chamberlain, L. (2007). What is ‘neuromarketing’? A discussion and agenda for future research. International Journal of Psychophysiology, 63, 199–204.
Munn, H. L. (1960). Brand perception as related to age, income, and education. Journal of Marketing, 24(3), 29–34.
Murray, I., & Arnott, J. L. (1993). Towards the simulation of emotion in synthetic speech: a review of the literature on human vocal emotion. Journal of the Acoustical Society of America, 93, 1097–1108.
Nenycz-Thiel, M., Beal, V., Ludwichowska, G., & Romaniuk, J. (2013). Investigating the accuracy of self-reports of brand usage behavior. Journal of Business Research, 66(2), 224–232.
O’Guinn, Allen, T. C., & Semenik, R. (2003). Advertising. New York: South-Western College Publishing.
Pao, T. L., Chen, Y. T., & Yeh, J. H. (2008). Emotion recognition and evaluation from Mandarin speech signals. International Journal of Innovative Computing, Information and Control (IJICIC), 4(7), 1695–1709.
Park, C., & Thorson, E. (1990). Influences on emotional response to commercials of different executional styles. In S. Agres, J. A. Edell, & T. M. Dubitsky (Eds.), Emotion in advertising: theoretical and practical explorations. Westport: Quorum Books.
Pawle, J., & Cooper, P. (2006). Measuring emotion—lovemarks, the future beyond brands. Journal of Advertising Research, 46, 38–48.
Picard, R. (1997). Affective computing. Cambridge: MIT Press.
Picard, R. W. (2003). What does it mean for a computer to “have” emotions? In R. Trappl, P. Petta & S. Payr (Eds.), Emotions in humans and artifacts r. MIT Press.
Pinker, S. (1994). Apes–lost for words. New Statesman & Society, 7, 30–31.
Plassmann, H., Ambler, T., Braeutigam, S., & Kenning, P. (2007). What can advertisers learn from neuroscience? International Journal of Advertising, 26(2), 151–175.
Plutchik, R. (1994). The psychology and biology of emotion. New York: Harper Collins.
Poels, K., & Dewitte, S. (2006). How to capture the heart? Reviewing 20 years of emotion measurement in advertising. Journal of Advertising Research, 46, 18–37.
Potter, R. F., & Bolls, P. D. (2012). Psychophysiological measurement and meaning: cognitive and emotional processing of media. New York: Routledge.
Rajshekhar, G. J., Park, J., Lee, O., Prasad, V. K., Ivan, R., & Vernon, I. R. (2013). Antecedents of Taiwan Chinese consumers’ purchase intentions toward US- and Japanese-made household appliances. Journal of Global Marketing, 26(4), 203–223.
Sigal, T., & Ram, H. (2012). Gender differences in brand commitment, impulse buying, and hedonic consumption. Journal of Product and Brand Management, 21(3), 176–182.
Teixeira, T., Wedel, M., & Pieters, R. (2012). Emotion-induced engagement in internet video advertisements. Journal of Marketing Research, 49(2), 144–159.
Wahlberg, D. (2004). Advertisers probe brains, raise fears. http://www.cognitiveliberty.org/neuro/neuromarketing_ajc.html [On-line].
Wang, Y. J., & Minor, M. S. (2008). Validity, reliability, and applicability of psychophysiological techniques in marketing research. Psychology and Marketing, 25, 197–232.
Wang, W.C. & Moutinho, L. (2013). Chapter 18: The slogan validator—the application of human-computer interface. In L. Moutinho, E. Bigne, & A. Manrai (Eds), Routledge companion on future of marketing, Taylor and Francis Publication.
Winkielman, P., Berridge, K. C., & Wilbarger, J. L. (2005). Unconscious affective reactions to masked happy versus angry faces influence consumption behavior and judgments of value. Personality and Social Psychology Bulletin, 31, 121–135.
Wurm, L. H., Vakoch, D. A., Strasser, M. R., Calin-Jageman, R., & Ross, S. E. (2001). Speech perception and vocal epression of emotion. Cognition & Emotion, 15(6), 831–852.
Author information
Authors and Affiliations
Corresponding author
Appendices
Appendix 1: The development of the Voice Emotion Response
1.1 Emotional speech corpus
Initially, an emotional corpus must be built to form the basis for eliciting emotions from speech signals. Using the sample of 18 males and 16 females, this study investigates the 5 primary emotions in response to 20 different prompting sentences of one to six words. Designers purposefully created neutral and meaningful sentences to enable participants to easily express themselves through the 5 emotions. This exercise yielded a corpus of 3400 utterances. Human judges then evaluated this preliminary corpus, removing ambiguous emotional utterances for further recognition analysis. Table 4 shows the performance confusion matrix of the judges. The rows and columns represent the simulated and the evaluated categories, respectively. Table 4 shows only the utterances that judges recognized as portraying the given emotion, and subsequently divides them into different subsets according to recognition accuracy. D80, D90, and D100 denote subsets with recognition accuracy of at least 80, 90, and 100 %, respectively (see Table 5). The D80 subset is relatively close to the human recognition rate (Banziger and Scherer 2005). Table 5 also shows the distribution of utterances among the given emotion categories.
The underlying recognition architecture forms the core of the Voice Emotion Response. Figure 2 illustrates the emotion recognition architecture based on the K-NN (K-nearest neighbor) methods.
Explanations of technical terms
Technical terms | Explanation |
Corpus | A large collection of written or spoken language used for studying the language. |
Confusion matrix | A visualization tool typically used in supervised learning. |
K-NN (K-nearest neighbor) | A supervised learning algorithm where the result of new instance query is classified based on the majority of K-nearest neighbor category. The purpose of this algorithm is to classify a new object based on attributes and training samples. |
High-pass filter | A filter that passes high frequencies well but attenuates (reduces the amplitude of) frequencies lower than the cut-off frequency. |
Frame | In signal processing, a frame is a fixed amount of samples or time duration that is cut off at a fixed period. |
Hamming window | A fixed period of time or sample that has some special functions. We used a Hamming window to reduce discontinuity among the windowed frames. |
A windowed frame | A frame that is cut by a window. |
Appendix 2
The questionnaire introduced the emotion in the following way:
Please say aloud once: “Family Mart is Your Home!” How much emotion did you experience at this point when you were saying this advertising slogan aloud?
Not at all happy | 1 | 2 | 3 | 4 | 5 | Very happy |
Not at all angry | 1 | 2 | 3 | 4 | 5 | Very angry |
Not at all sad | 1 | 2 | 3 | 4 | 5 | Very Sad |
Not at all bored | 1 | 2 | 3 | 4 | 5 | Very bored |
Not at all neutral | 1 | 2 | 3 | 4 | 5 | Very neutral |
Diagram of the Voice Emotion Response result
Means for self-reported emotional measures
Slogans | Pepsi | Coca Cola | 7-Eleven | Family Mart | KFC |
Emotions | |||||
Angry | 1.75 | 1.68 | 1.58 | 1.58 | 2.07 |
Happy | 3.50 | 3.96 | 4.23 | 3.77 | 3.49 |
Sad | 1.74 | 1.63 | 1.59 | 1.63 | 1.80 |
Bored | 2.37 | 2.01 | 1.91 | 2.01 | 2.08 |
Neutral | 2.74 | 2.80 | 2.72 | 2.65 | 2.68 |
The correlation matrix of five emotions
Coca Cola five emotions correlations (self-report) | Coca Cola angry | Coca Cola happy | Coca Cola sad | Coca Cola bored | Coca Cola neutral |
Coca Cola angry | 1.000 | −0.251 | 0.672** | 0.368** | 0.242** |
Coca Cola happy | −0.251** | 1.000 | −0.240** | −0.341** | −0.146** |
Coca Cola sad | 0.672** | −0.240** | 1.000 | 0.495** | 0.236** |
Coca Cola bored | 0.368** | −0.341** | 0.495** | 1.000 | 0.309** |
Coca Cola neutral | 0.242** | −0.146 | 0.236** | 0.309** | 1.000 |
Coca Cola five emotions correlations (VER) | VER Coca Cola angry | VER Coca Cola happy | VER Coca Cola sad | VER Coca Cola bored | VER Coca Cola neutral |
VER Coca Cola angry | 1.000 | −0.097 | −0.292** | −0.400** | −0.419** |
VER Coca Cola happy | −0.097 | 1.000 | 0.119 | 0.275** | 0.395** |
VER Coca Cola sad | −0.292** | 0.119 | 1.000 | 0.247** | 0.087 |
VER Coca Cola bored | −0.400** | 0.275** | 0.247** | 1.000 | 0.812** |
VER Coca Cola neutral | −0.419** | 0.395** | 0.087 | 0.812** | 1.000 |
Pepsi five emotions correlations (self-report) | Pepsi angry | Pepsi happy | Pepsi sad | Pepsi bored | Pepsi neutral |
Pepsi angry | 1.000 | −0.038 | 0.784** | 0.320** | 0.295** |
Pepsi happy | −0.038 | 1.000 | 0.018 | −0.333** | −0.088 |
Pepsi sad | 0.784** | 0.018 | 1.000 | 0.312** | 0.244** |
Pepsi bored | 0.320** | −0.333** | 0.312** | 1.000 | 0.157 |
Pepsi neutral | 0.295** | −0.088 | 0.244** | 0.157 | 1.000 |
Pepsi five emotions correlations (VER) | VER Pepsi angry | VER Pepsi happy | VER Pepsi sad | VER Pepsi bored | VER Pepsi neutral |
VER Pepsi angry | 1.000 | −0.088 | −0.224** | −0.548** | −0.457** |
VER Pepsi happy | −0.088 | 1.000 | 0.367** | 0.383** | 0.626** |
VER Pepsi sad | −0.224** | 0.367** | 1.000 | 0.367** | 0.335** |
VER Pepsi bored | −0.224** | 0.367** | 0.367** | 1.000 | 0.335** |
VER Pepsi neutral | −0.457** | 0.626** | 0.335** | 0.811** | 1.000 |
KFC five emotions correlations (self-report) | KFC angry | KFC happy | KFC sad | KFC bored | KFC neutral |
KFC angry | 1.000 | −0.284** | 0.492** | 0.362** | 0.051 |
KFC happy | −0.284** | 1.000 | −0.178* | −0.261** | 0.134 |
KFC sad | 0.492** | −0.178* | 1.000 | 0.415** | 0.135 |
KFC bored | 0.362** | −0.261** | 0.415** | 1.000 | 0.197* |
KFC neutral | 0.051 | 0.134 | 0.135 | 0.197* | 1.000 |
KFC five emotions correlations (VER) | VER KFC angry | VER KFC happy | VER KFC sad | VER KFC bored | VER KFC neutral |
VER KFC angry | 1.000 | 0.089 | −0.230 | −0.628** | −0.405** |
VER KFC happy | 0.089 | 1.000 | 0.336** | 0.344** | 0.666** |
VER KFC sad | −0.230** | 0.336** | 1.000 | 0.373** | 0.379** |
VER KFC bored | −0.628** | 0.344** | 0.373** | 1.000 | 0.801** |
VER KFC neutral | −0.405** | 0.666** | 0.379** | 0.801** | 1.000 |
Seven Eleven five emotions correlations (self-report) | Seven Eleven angry | Seven Eleven happy | Seven Eleven sad | Seven Eleven bored | Seven Eleven neutral |
Seven Eleven angry | 1.000 | −0.200* | 0.629** | 0.476** | 0.197* |
Seven Eleven happy | −0.200* | 1.000 | −0.136 | −0.304** | 0.052 |
Seven Eleven sad | 0.629** | −0.136 | 1.000 | 0.646** | 0.236** |
Seven Eleven bored | 0.476** | −0.304** | 0.646** | 1.000 | 0.179* |
Seven Eleven neutral | 0.197* | 0.052 | 0.236** | 0.179* | 1.000 |
Seven Eleven five emotions correlations (VER) | VER Seven Eleven angry | VER Seven Eleven happy | VER Seven Eleven sad | VER Seven Eleven bored | VER Seven Eleven neutral |
VER Seven Eleven angry | 1.000 | −0.047 | −0.073 | −0.277** | −0.263** |
VER Seven Eleven happy | −0.047 | 1.000 | 0.088 | 0.190* | 0.379** |
VER Seven Eleven sad | −0.073 | 0.088 | 1.000 | 0.132 | −0.016 |
VER Seven Eleven bored | −0.277** | 0.190* | 0.132 | 1.000 | 0.907** |
VER Seven Eleven neutral | −0.263** | 0.379** | −0.016 | 0.907** | 1.000 |
Family Mart five emotions correlations (self-report) | Family Mart angry | Family Mart happy | Family Mart sad | Family Mart bored | Family Mart neutral |
Family Mart angry | 1.000 | −0.192* | 0.774** | 0.555** | 0.119 |
Family Mart happy | −0.192* | 1.000 | −0.227** | −0.334** | 0.108 |
Family Mart sad | 0.774** | −0.227** | 1.000 | 0.664** | 0.111 |
Family Mart bored | 0.555** | −0.334** | 0.664** | 1.000 | 0.171* |
Family Mart neutral | 0.119 | 0.108 | 0.111 | 0.171* | 1.000 |
Family Mart five emotions correlations (VER) | VER Family Mart angry | VER Family Mart happy | VER Family Mart sad | VER Family Mart bored | VER Family Mart neutral |
VER Family Mart angry | 1.000 | −0.095 | −0.331** | −0.613** | −0.492** |
VER Family Mart happy | −0.095 | 1.000 | 0.113 | 0.288** | 0.531** |
VER Family Mart sad | −0.331** | 0.113 | 1.000 | 0.323** | 0.201* |
VER Family Mart bored | −0.613** | 0.288** | 0.323** | 1.000 | 0.840** |
VER Family Mart neutral | −0.492** | 0.531** | 0.201* | 0.840** | 1.000 |
*Significant at 0.05 level (two-tailed)
**Significant at 0.01 level (two-tailed)
Rights and permissions
About this article
Cite this article
Wang, WC., Chien, C.S. & Moutinho, L. Do you really feel happy? Some implications of Voice Emotion Response in Mandarin Chinese. Mark Lett 26, 391–409 (2015). https://doi.org/10.1007/s11002-015-9357-y
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11002-015-9357-y