Affective Computing in Marketing: Practical Implications and Research Opportunities Afforded by Emotionally Intelligent Machines

After years of using AI to perform cognitive tasks, marketing practitioners can now use it to perform tasks that require emotional intelligence. This advancement is made possible by the rise of affective computing, which develops AI and machines capable of detecting and responding to human emotions. From market research, to customer service, to product innovation, the practice of marketing will likely be transformed by the rise of affective computing, as preliminary evidence from the field suggests. In this Idea Corner, we discuss this transformation and identify the research opportunities that it offers.


Introduction
Affective computing-the study and development of artificial intelligence (AI) and machines that detect and respond to affective states-has recently received increased interest outside its original field owing to the emergence of commercial applications.For example, in 2016, Affectiva's emotion-sensing technology was integrated with the videogame Nevermind, allowing it to dynamically adjust to the players' emotions.Similarly, in 2017, France Loisirs, a French bookstore, tested Angus's emotion-sensing technology to detect the emotions of browsing customers and alert employees in real-time, enabling them to offer customer support when needed.
Major international firms, such as Apple and Google, have already invested in affective computing technology to enhance customer experience (Research And Markets, 2020), paving the way for its large-scale marketing applications.In fact, the industry of affective computing technologies is expected to be worth US$123.3 billion by 2026, with an estimated compound annual growth rate of 36.4% in 2020-2026(Research And Markets, 2020).Nevertheless, affective computing has received little attention in marketing literature.Therefore, the need for research on the impact of affective computing on marketing is urgent.
Affective computing relies on the bodily changes (e.g., smile, heart rate acceleration, sweating) characterizing affective states or emotions.Machines sense these changes-biometrics, brain activity, facial expressions, body movement, or voice/ speech can be used as input-and algorithms are trained to identify the corresponding emotion (D'Mello et al., 2018).Machines can also be trained to respond to the identified emotion (e.g., a robot smiling in response to a human's smile).
Affective computing is expected to transform marketing practice because: (1) firms could collect new types of insights into consumer emotions to personalize their experience in an unprecedented manner; (2) human employees would no longer be needed to understand and manage consumer emotions during their interactions; and (3) bots (robots, chatbots, voice assistants) that inform and serve consumers could be augmented with emotional intelligence.
Although affective computing and AI are related, the critical differences between them call for marketing research on affective computing in its own right.Whereas AI mimics human IQ type of intelligence when performing cognitive tasks, affective computing simulates emotional intelligence.The latter may be less easily embraced than the former because emotions are considered to be "our human advantage," distinguishing us from machines (Forbes, 2021).Therefore, studying how emotionally intelligent machines, beyond pure cognitive AI, can be accepted by consumers and employees is essential.
To this end, we advance a framework that distinguishes between three uses of affective computing in marketing: empathic (to detect emotions), collaborative (to detect emotions and assist humans in responding to them), and interactive (to detect and respond to emotions).This framework draws upon the work by Picard (1997), which specifies that affective computing can serve to detect emotions or to detect and express emotions.

Research agenda
We identify research directions for all three uses of affective computing (Table 1 presents an expanded list of the research questions).Collaborative How do frontline employees respond to the use of affective computing to assist them during interactions with consumers?What are the antecedents of employees' acceptance of and trust in affective computing systems?Do frontline employees experience aversion toward affective computing systems?If so, how can this be alleviated?Do employees feel dehumanized by the use of affective computing, as they are partly deprived of using their emotional intelligence?How does collaborative affective computing impact employees' task performance?How does it impact their emotional labor?How does collaborative affective computing impact frontline employees' job satisfaction and motivation?Does it make employees worry that machines/AI could take over their jobs?To what extent are consumers willing to have their facial expressions or voice recorded and analyzed during interactions with firms?What individual differences (e.g., dispositional willingness to self-disclose) and situational factors (e.g., public vs. private service contexts) explain consumers' level of (dis)comfort with such use of affective computing systems?Do consumers perceive any difference in how they are served by employees assisted by affective computing (vs.not assisted)?Does this depend on the nature of the bond (e.g., functional vs. emotional bond) between the consumer and the employee?Does the use of affective computing systems during consumer-firm interactions contribute to more satisfactory consumer experiences?Does it have the same effect on both face-toface and technology-mediated interactions?How do consumers expect firms to inform them about their use of affective computing systems during service interactions?
Interactive How do consumers respond to emotionally augmented service robots?Does the fact that robots display emotions increase their acceptance, likeability, perceived warmth, and perceived competence?Or does it trigger discomfort?What individual differences (e.g., dispositional sensitivity to the uncanny valley) and situational factors (e.g., occurrence of a service failure) moderate consumer response?How does the use of emotionally augmented service robots impact the organizational frontline?How do emotionally augmented robots influence the service climate?How does the augmentation of service robots with emotional intelligence impact the work relationship between robots and (human) employees?How effective are the emotional displays of service robots?Does emotional contagion occur when emotionally augmented robots display emotions to consumers?Does a robot's ability to display appropriate emotions enable it to build rapport with consumers?What does the concept of authenticity of emotional displays mean when robots are the ones displaying emotions?What contributes to consumers' perception of emotional displays of robots as authentic?When and how do consumers derive value from using products and services that adapt in real-time to how they feel?Does emotion-based personalization add more value to hedonic products/services than to utilitarian ones?Under what conditions are consumers willing to trade their emotional data for a more personalized experience?What privacy concerns do consumers have regarding the use of their emotional data?What measures can companies implement to alleviate the privacy concerns of consumers regarding the use of their emotional data?

Empathic affective computing
Empathic affective computing (for affective computing systems that detect emotions) is relevant to marketing managers who want to understand consumers' emotions.Emotion detection is central to market research, as many consumer decisions are driven by emotions.As such, empathic affective computing will likely advance market research by offering a new range of methods for studying which emotions are triggered by a product, an advertisement, or another element of the marketing mix.Academic research should examine the predictive abilities of affective computing methods relative to those of traditional methods.Affective computing methods keep increasing in accuracy and may even outperform survey methods, as anecdotal evidence suggests.For example, the food company Mars used Affectiva's Affdex technology, which identifies emotions based on facial expressions, to determine consumers' emotional response to their ads and found that it predicted short-term sales more accurately than self-reports (Affectiva, n.d.).Academic research is needed to corroborate this finding.
Moreover, empathic affective computing offers opportunities for studying unconscious emotions.These are emotions that affective computing methods can sense but that study participants would not necessarily be cognitively aware of (Winkielman & Berridge, 2004).When and how do unconscious emotions affect consumer attitudes and behaviors?We encourage academics to extend existing knowledge about the role of emotions in the consumer decision-making process by exploring the respective roles of conscious and unconscious emotions in this process.

Collaborative affective computing
Collaborative affective computing (to detect emotions and assist humans in responding to them) can cooperate with employees during consumer interactions by helping them understand the consumer's emotions and prompting them to respond accordingly.
However, a prerequisite is that frontline employees are willing to use affective computing systems.Accordingly, research is needed to identify the factors influencing the acceptance of and trust in these systems.Prior research has shown that in decision-making situations, humans are unwilling to trust forecasts made by algorithms-a phenomenon called algorithm aversion (Dietvorst et al., 2015).Does a similar aversion apply to affective computing systems?If so, how can it be alleviated?
Research should explore if consumers consent to their emotions being tracked by affective computing systems during interactions with the firm.Humana, an insurance company, experimented with Cogito's affective computing method to assist its call center agents (these agents are alerted when an emotion that needs to be addressed is detected in a consumer's voice) and saw the net promotor score and issue resolution increase by 28% and 6%, respectively (Inc, 2017).Although consumers may benefit from collaborative affective computing by being better served, they may feel uncomfortable by having their emotions systematically analyzed.How do consumers evaluate this trade-off?Moreover, we encourage academics to investigate how consumers expect firms to inform them about their use of affective computing systems during service interactions.
Finally, collaborative affective computing raises questions regarding the psychological outcomes experienced by employees.Although the case of Humana suggests that collaborative affective computing improves the task performance of frontline employees, its impact on employees' well-being remains unknown.Frontline employees, who perform emotional labor, are particularly likely to experience stress and burnout.Does collaborative affective computing alleviate the burden of emotional labor?Or does it make employees worry that machines/AI could take over their jobs, thereby reducing their motivation?Do employees feel dehumanized, as they are partly deprived from using their emotional intelligence?

Interactive affective computing
Interactive affective computing (for affective computing systems that detect and respond to emotions) can enrich consumer-bot interactions by augmenting the (ro) bot with emotional intelligence.This technology can be applied to service robots (Mende et al., 2019) or bots built in physical objects (e.g., headphones) or on digital platforms (e.g., Spotify).We review each application.
Interactive affective computing has potential for transforming organizational frontlines by enabling service robots to adapt their emotional responses based on the emotions of the consumers who interact with them.This raises questions regarding the impact of such robots on service climate and on consumer responses.Past research (e.g., Mende et al., 2019) has focused on how the physical appearance of robots influences consumers' propensity to anthropomorphize them; however, how robots' behaviors influence consumer responses remains unknown.Robots augmented with emotional intelligence will behave in a more human-like manner.Would this increase their acceptance and likeability?Or would consumers perceive such robots as excessively anthropomorphized, thereby triggering discomfort?
Another research direction relates to the effectiveness of robots' emotional displays.The positive emotions displayed by (human) employees generate positive emotions in consumers, a phenomenon called emotional contagion (Hennig-Thurau et al., 2006).Does emotional contagion occur when consumers interact with emotionally augmented robots?If so, does it enable building rapport between robots and consumers?
Further, questions arise regarding the authenticity of the emotions displayed by robots augmented with emotional intelligence, which is a critical factor in (human) employee-consumer interactions (Hennig-Thurau et al., 2006).However, what is authentic emotional display in robots?This construct needs to be re-conceptualized.
Interactive affective computing also offers the potential for real-time emotionbased personalization of bot-enabled products, creating opportunities for product innovation.For instance, Neurowear's Mico headphones sense the user's brain activity to identify their affective state and play songs that match it.Similarly, Spotify-which offers voice-controlled access to its streaming platform-is working on a voice recognition system that can identify users' emotions.Research is needed to determine when and how consumers derive value from using products and services that adapt in real-time to their emotions.We encourage academics to study how emotion-based personalization benefits hedonic versus utilitarian products.
Examining the circumstances under which consumers are willing to trade their emotional data for an emotionally personalized experience is also important.The amount and type of data collected by affective computing systems raise privacy concerns, and its implications need to be examined.

Conclusion
Understanding and responding to consumer emotions is fundamental to marketing practice, and affective computing will likely transform this process.As marketing applications of affective computing rise, new research is needed to understand how these applications affect consumers, employees, and companies.
Funding Open access funding provided by Kristiania University College.
Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made.The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material.If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.To view a copy of this licence, visit http:// creat iveco mmons.org/ licen ses/ by/4.0/.

Table 1
Opportunities for future research What is the role of conscious versus unconscious emotions in consumer decision-making?Should market research companies use affective computing methods in addition to or as a substitute for traditional methods?