Abstract
The use of the computer as a model, metaphor, and modelling tool has tended to privilege the 'cognitive' over the 'affective' by engendering theories in which thinking and learning are viewed as information processing and affect is ignored or marginalised. In the last decade there has been an accelerated flow of findings in multiple disciplines supporting a view of affect as complexly intertwined with cognition in guiding rational behaviour, memory retrieval, decision-making, creativity, and more. It is time to redress the imbalance by developing theories and technologies in which affect and cognition are appropriately integrated with one another. This paper describes work in that direction at the MIT Media Lab and projects a large perspective of new research in which computer technology is used to redress the imbalance that was caused (or, at least, accentuated) by the computer itself.
Similar content being viewed by others
References
Simon H A: 'Motivational and emotional controls of cognition', in Models of Thought, pp 29—38, Yale University Press, New Haven (1967).
Norman D A: 'Twelve issues for cognitive science', in Norman D A (Ed): 'Perspectives on cognitive science', pp 265—295, Erlbaum, Hillsdale, NJ (1981).
Isen A M: 'Positive affect and decision making', in Lewis M and Haviland J (Eds): 'Handbook of emotions', Guilford, New York (2000).
Lane R, Reiman E M, Ahern G L, Schwartz G E and Davidson R J: 'Neuroanatomical correlates of happiness, sadness and disgust', American Journal of Psychiatry, 154, pp 926—933 (1997).
Damasio A R, Grabowski T J, Bechara A, Damasio H, Ponto L L B, Parvizi J and Hichwa R D: 'Subcortical and cortical brain activity during the feeling of self-generated emotions', Nature Neuroscience, 3, pp 1049—1056 (2000).
Ashby F G, Isen A M and Turken U: 'A neuropsychological theory of positive affect and its influence on cognition', Psychological Review, 106, No 3, pp 529—550 (1999).
Estrada C, Isen A M and Young M J: 'Positive affect influences creative problem solving reported source of practice satisfaction in physicians', Motivation and Emotion, 18, pp 285—299 (1994).
Minsky M: 'The Emotional Machine', (2003) — http://web.media.mit.edu/~minsky/
Bransford J D, Brown A L and Cocking R R: 'How people learn: brain, mind, experience and school', National Academy Press, Washington, DC (1999).
Piaget J: 'Six Etudes de Psychologie', Pub Gonthier, Paris (1971).
Vygotsky L: 'Thought and Language', The MIT Press, Cambridge, MA (1962).
Vygotsky L: 'Mind in Society: the development of the higher psychological processes', The Harvard University Press, Cambridge, MA. Originally published 1930, Oxford University Press, New York (1978).
Darwin C: 'The Expression of Emotions in Man and Animals', The University of Chicago Press, Chicago, IL (reprinted in 1965) (1872).
James W: 'Writings 1879-1899', Chapter on Emotion, pp 350—365, The Library of America (1992) (originally published in 1890 as Principles of Psychology, Holt, NY (1890)).
Kleinginna Jr, P R and Kleinginna A M: 'A categorised list of emotion definitions, with suggestions for a consensual definition', Motivation and Emotion, 5, No 4, pp 345—379 (1981).
Ortony A, Clore G L and Collins A: 'The Cognitive Structure of Emotions', Cambridge University Press, Cambridge (1988).
Csikszentmihályi M: 'Flow: The Psychology of Optimal Experience', HarperCollins (1991).
Lepper M R and Chabay R W: 'Socialising the intelligent tutor: bringing empathy to computer tutors', in Mandl H and Lesgold A (Eds): 'Learning issues for intelligent tutoring systems', pp 242—257 (1988).
Mandler G: 'Mind and Body: Psychology of Emotion and Stress', W W Norton & Co, New York, NY (1984).
Kort B, Reilly R and Picard R W: 'An affective model of interplay between emotions and learning: reengineering educational pedagogy-building a learning companion', in Proceedings of International Conference on Advanced Learning Technologies (ICALT 2001), Madison Wisconsin (August 2001).
Kort B, Reilly R and Picard R W: 'External representation of learning process and domain knowledge: affective state as a determinate of its structure and function', Workshop on Artificial Intelligence in Education (AI-ED 2001), San Antonio, (May 2001).
Vroom V H: 'Work and Motivation', Wiley, New York (1964).
Keller J M: 'Motivational design of instruction', in Reigeluth C M (Ed): 'Instructional Design Theories and Models: an overview of their current status', Erlbaum, Hillsdale, NJ (1983).
Keller E F: 'A feeling for the organism: the life and work of Barbara McClintock', San Francisco, W H Freeman (1983).
Keller J M: 'Strategies for stimulating the motivation to learn', Performance and Instruction, 26, No 8, pp 362—632 (1987).
Keller J M: 'IMMS: instructional materials motivation survey', Florida State University (1987).
Ames C: 'Classrooms: goals, structures, and student motivation', Journal of Educational Psychology, 84, No 3, pp 261—271 (1992).
Vail P: 'Emotions: the on/off switch for learning', Modern Learning Press (1994).
Bandura A: 'Self-efficacy: toward a unifying theory of behaviour change', Psychological Review, 84, pp 191—215 (1977).
Bandura A: 'Social learning theory', Prentice-Hall, Englewood Cliffs, NJ (1977).
Pajares F: 'Self-efficacy beliefs in academic settings', Review of Educational Research, 66, pp 543—578 (1996).
Schunk D H: 'Self-efficacy and achievement behaviours', Educational Psychology Review, 1, pp 173—208 (1989).
Zimmerman B J: 'Self-efficacy: an essential motive to learn', Contemporary Educational Psychology, 25, pp 82—91 (2000).
Maehr M L: 'Meaning and motivation: toward a theory of personal investment', in Ames R and Ames C (Eds): 'Research on motivation in education', pp 39—73, Academic Press, San Diego, CA (1984).
Dweck C S: 'Motivational processes affecting learning', American Psychologist, 41, pp 1040—1048 (1986).
Ames C and Archer J: 'Achievement goals in the classroom: students' learning strategies and motivation processes', Journal of Educational Psychology, 80, pp 260—267 (1988).
Dweck C S and Leggett E L: 'A social-cognitive approach to motivation and personality', Psychological Review, 95, pp 256—273 (1988).
Elliott E S and Dweck C S: 'Goals: an approach to motivation and achievement', Journal of Personality and Social Psychology, 54, pp 5—12 (1988).
Matsubara Y and Nagamachi M: 'Motivation systems and motivation models for intelligent tutoring', in Frasson C (Ed): Proceedings of the Third International Conference in Intelligent Tutoring Systems, pp 139—147 (1996).
De Vicente A and Pain H: 'Motivation diagnosis', in Intelligent Tutoring Systems (1998).
Whitelock D and Scanlon E: 'Motivation, media and motion: reviewing a computer supported collaborative learning experience', in Brna P <nt>et al</nt> (Eds): 'Proceedings of the European Conference', (1996).
Keller J M and Keller B H: 'Motivational delivery checklist', Florida State University (1989).
Kagan J: Personal communication (2002).
Computing Research Association — http://www.cra.org/
Fernald A: 'Intonation and communicative intent in mother's speech to infants: Is the melody the message?' Child Development, 60, pp 1497—1510 (1989).
Salovey P and Mayer J D: 'Emotional intelligence', Imagination, Cognition and Personality, 9, No 3, pp 185—211 (1990).
Goleman D: 'Emotional Intelligence', Bantam Books, NY (1995).
Picard R W: 'Affective Computing', The MIT Press, Cambridge, MA (1997).
Scherer K R: 'Chapter 10: speech and emotional states', in Darby J K (Ed): 'Speech evaluation in psychiatry', Grune and Stratton Inc (1981).
Hansen J: 'Speech under stress', ICASSP'99, Pheonix, Arizona (1999).
Ekman P and Friesen W: 'Facial action coding system', Consulting Psychologists Press (1977).
Cohn J F, Zlochower A J, Lien J and Kanade T: 'Automated face analysis by feature point tracking has high concurrent validity with manual FACS coding', Psychophysiology, 36, pp 35—43 (1999).
Bartlett M S, Hager J C, Ekman P and Sejnowski T J: 'Measuring facial expressions by computer image analysis', Psychophysiology, 36, pp 253—263 (1999).
Donato G, Bartlett S, Hager M, Ekman J C P and Seinowski T: 'Classifying facial actions', IEEE Transactions on Pattern Analysis and Machine Intelligence, 21, No 10, pp 974—989 (1999).
Tian Y, Kanade T and Cohn J F: 'Recognising action units for facial expression analysis', IEEE Trans Pattern Analysis and Machine Intelligence, 23, No 2 (February 2001).
Kapoor A and Picard R W: 'Real-time, fully automatic upper facial feature tracking', in Proceedings of the 5th International Conference on Automatic Face and Gesture Recognition 2002, Washington DC (May 2002).
Kapoor A and Picard R W: 'A real-time head nod and shake detector', Workshop on Perceptive User Interfaces, Orlando, FL (November 2001).
Mota S and Picard R W: 'Automated posture analysis for detecting learner's interest level', 1st IEEE Workshop on Computer Vision and Pattern Recognition, CVPR HCI 2003 (2003).
Bartlett M S, Littlewort G, Braathen B, Sejnowski T J and Movellan J R: 'A prototype for automatic recognition of spontaneous facial actions', in Becker S and Obermayer K (Eds): 'Advances in Neural Information Processing Systems', The MIT Press, Cambridge, MA (2003).
DeSilva L C, Miyasato T and Nakatsu R: 'Facial emotion recognition using multi-model information', Proceedings of IEEE International Conference on Information, Communication and Signal Processing, Singapore, pp 397—401 (1997).
Huang T S, Chen L S and Tao H: 'Bimodal emotion recognition by man and machine', ATR Workshop on Virtual Communication Environments, Kyoto, Japan (April 1998).
Chen L S, Huang T S, Miyasato T and Nakatsu R: 'Multimodal human emotion/expression recognition', in Proceedings of International Conference on Automatic Face and Gesture Recognition, Nara, Japan, IEEE Computer Society (April 1998).
Kapoor A, Picard R W and Ivanov Y: 'Probabilistic combination of multiple modalities to detect interest', Proceedings of Int Conf on Pattern Recognition, Cambridge, England (August 2004).
Kapoor A, Mota S and Picard R W: 'Towards a learning companion that recognises affect', AAAI Fall Symposium 2001, North Falmouth, MA (November 2001).
Conati C: 'Probabilistic assessment of users' emotions in educational games', Applied Artificial Intelligence, 16, Nos 7 and 8, pp 555—575 (2002).
Zhou X and Conati C: 'Inferring user goals from personality and behaviour in a causal model of user affect', Intelligent User Interfaces 2003, pp 211—218 (2003).
Zhou X and Conati C: 'Modeling students' emotions from cognitive appraisal in educational games', Intelligent Tutoring Systems 2002, pp 944—954 (2002).
Sheldon-Biddle E, Malone L and McBride D: 'Objective measurement of student affect to optimise automated instruction', Proceedings of Workshop on Modelling User Attitudes and Affect, User Modeling '03 (2003).
Scheirer J, Fernandez R and Picard R W: 'Expression glasses: a wearable device for facial expression recognition', CHI'99 Short Papers, Pittsburgh, PA (1999).
Healey J and Picard R W: 'SmartCar: detecting driver stress', Proceedings of ICPR'00, Barcelona, Spain (May 2000).
Tekscan: 'Tekscan Body pressure measurement system user's manual', Tekscan Inc, South Boston, MA USA (1997).
Reeves B and Nass C: 'The Media Equation', Center for the Study of Language and Information, Cambridge University Press (1996).
Picard R W and Scheirer J: 'The Galvactivator: a glove that senses and communicates skin conductivity', Proceedings from the 9th International Conference on Human-Computer Interaction, New Orleans (August 2001).
Dennerlein J T, Becker T, Johnson T, Reynolds C and Picard R W: 'Frustrating computer users increases exposure to physical risk factors', Proceedings of International Ergonomics Association, Seoul, Korea (2003).
Bechara A, Damasio H, Tranel D and Damasio A: 'Deciding advantageously before knowing the advantageous strategy', Science, 275, pp 1293—1295 (1997).
Hirstine W, Iversen P and Ramachandran V S: 'Autonomic responses of autistic children to people and objects', Proceedings of the Royal Society, London, B, 268, pp 1883—1888 (April 2001).
La France M: 'Posture Mirroring and Rapport', in David M (Ed): 'Interaction rhythms: periodicity in communicative behaviour', Human Sciences Press, New York, NY, pp 279—298 (1982).
Liu K and Picard R W: 'Subtle expressivity in a robotic computer', presented at CHI 2003 Workshop on Subtle Expressiveness in Characters and Robots (2003).
Neisser U: 'The imitation of man by machine', Science, 139, pp 193—197 (1963).
Sutton R and Barto A: 'Reinforcement learning: an introduction', The MIT Press, Cambridge, MA (1998).
Shuttleworth S J: 'Cognition, evolution and behaviour', Oxford University Press, New York, NY (1998).
Gould J and Gould C: 'The animal mind', W H Freeman, New York, NY (1999).
Wilkes G: 'Click and treat training kit', Click and Treat Inc, Mesa, AZ (1995).
Marler P: 'Song learning: the interface between behaviour and neuroethology: philosophical transactions', Biological Sciences, 329, No 1253, pp 109—114 (1990).
Markham E: 'Constraints children place on word meanings', Cognitive Science, 14, pp 57—77 (1990).
Breazeal C, Edsinger A, Fitzpatrick P and Scassellati B: 'Active vision systems for sociable robots', in Dautenhan K (Ed): 'IEEE Transactions on Systems, Man and Cybernetics', Part A, 31, No 5, pp 443—453 (2001).
Feinman S, Roberts D, Hseih K, Sawyer D and Swanson D: 'A critical review of social referencing in infancy', in Feinman S (Ed): 'Social referencing and the social construction of reality in infancy', pp 15—54, Plenum Press, New York (1992).
Breazeal C, Hoffman G and Lockerd A: 'Teaching and working with robots as a collaboration', in Proceedings of AAMAS 2004, Columbia NY (2004).
Biswas G, Katzlberger T, Bransford J, Schwartz D and TAG-V (the teachable agents group at Vanderbilt): 'Extending Intelligent Learning Environments with Teachable Agents to Enhance Learning', 10th International Conference on AI in Education: AI-ED in the wired and wireless future (Moore J D, Redfield C L and Johnson W L (Eds)), IOS Press, Amsterdam, pp 389—397 (2001).
Biswas G, Schwartz D, Bransford J and TAG-V (the teachable agents group at Vanderbilt): 'Technology Support for Complex Problem Solving: From SAD Environments to AI, Smart Machines in Education', in Forbus K D and Feltovich P J (Eds): 'The coming revolution in education technology', AAAI/MIT Press, Menlo Park, CA (2001).
Pryor K: 'Clicker Training for Dogs', Sunshine Books Inc, Waltham, MA (1999).
Lindsay S R: 'Applied Dog Behaviour and Training', Iowa State University Press, Ames, IA (2000).
Blumberg B and Downie M: 'Integrated learning for interactive synthetic characters', Transactions on Graphic, 21, No 3, Proceedings of ACM SIGGRAPH 2002 (2002).
Blumberg B: 'D-learning: what learning in dogs tells us about building characters that learn what they ought to learn', in Lakemeyer G and Nebel B (Eds): 'Exploring Artificial Intelligence in the New Millenium', Morgan Kaufman Publishers, San Francisco, CA (2002).
Lorenz K and Leyhausen P: 'Motivation of human and animal behaviour: an ethological view', Van Nostrand Reinhold Co, New York, NY (1973).
Panksepp J: 'Affective Neuroscience', Oxford University Press (1998).
Bull P E: 'Posture and Gesture', International Series in Experimental Social Physchology, Pergamon Press (1987).
Hatfield E, Cacioppo J and Rapson R L: 'Emotional contagion', New York, Cambridge University Press (1994).
Burleson W, Picard R W, Perlin K and Lippincott J: 'A platform for affective agent research', in Proceedings of the AAMAS Workshop on Empathetic Agents, Columbia University, New York, NY (July 2004).
Wentzel K: 'Student motivation in middle school: the role of perceived pedagogical caring', Journal of Educational Psychology, 89, No 3, pp 411—419 (1997).
Pianta R C: 'Beyond the parent: the role of other adults in children's lives', New Directions in Child Development, 57, Jossey-Bass, San Francisco (1992).
Wentzel K and Asher S R: 'Academic lives of neglected, rejected, popular, and controversial children', Child Development, 66, pp 754—763 (1995).
Birch S H and Ladd G W: Interpersonal relationships in the school environment and children's early school adjustment: the role of teachers and peers', in Juvonen J and Wentzel K (Eds): 'Social motivation: understanding children's school adjustment', Cambridge University Press, New York (1996).
Bickmore T and Picard R W: 'Establishing and maintaining longterm human-computer relationships', Transactions on Computer Human Interaction, to appear (2004).
Csikszentmihályi M: 'Finding Flow: the psychology of engagement with everyday life', HarperCollins (1997).
Papert S: 'The Children's Machine', Basic Books (1993).
Kafai Y and Resnick M (Eds): 'Constructionism in practice: designing, thinking, and learning in a digital world', Mahwah, NJ, Lawrence Erlbaum (1996).
Brosterman N: 'Inventing Kindergarten', Harry Abrams Publishing (1997).
Resnick M, Berg R and Eisenberg M: 'Beyond black boxes: bringing transparency and aesthetics back to scientific investigation', Journal of the Learning Sciences, 9, No 1, pp 7—30 (2000).
Lave J and Wenger E: 'Situated Learning: legitimate peripheral participation', University Press, Cambridge (1991).
Turkle S and Papert S: 'Epistemological pluralism', Signs, 16, No 1, pp 128—157 (1990).
Papert S: 'Teaching children to be mathematicians versus teaching about mathematics', International Journal of Mathematics Education in Science and Technology, 3, pp 249—262 (1972).
Papert S: 'Mindstorms: Children, Computers, and Powerful Ideas', Basic Books (1980).
Abelson H and diSessa A: 'Turtle Geometry', The MIT Press (1980).
Austin H: 'A computation theory of physical skill', MIT Artificial Intelligence Laboratory Note AIM-330 (1974).
Burton R et al: 'Skiing as a Model of Instruction', in Rogoff B and Lave J (Eds): 'Everyday Cognition: development in social context', Harvard University Press (1984).
O'Modhrain S: 'Playing by feel: incorporating haptic feedback into computer-based musical instruments', unpublished PhD Dissertation, Stanford University (2000).
Sloane N J A and Wyner A D (Eds): 'Claude Elwood Shannon: Collected Papers', Piscataway, NJ, IEEE Press (1940/1993).
Strohecker C: 'Why knot?', unpublished doctoral dissertation, MIT Media Laboratory, Cambridge, MA (1991).
Strohecker C: 'Learning about topology and learning about learning', Proceedings of the Second International Conference on the Learning Sciences, Association for the Advancement of Computing in Education (1996).
Strohecker C: 'Understanding topological relationships through comparisons of similar knots', AI&Society: Learning with Artifacts, 10, pp 58—69 (1996).
Strohecker C: 'Cognitive zoom: from object to path and back again', Spatial Cognition II, pp 1—15 (2000).
Cavallo D, Blikstein P, Sipitakiat A, Basu A, Camargo A, de Deus Lopes R and Cavallo A: 'The city that we want: generative themes, constructionist technologies and school/social change', in Proceedings of IEEE Technology and Education in Developing Countries 2004, Joennsu, Finland (2004).
Cavallo D, Sipitakiat A, Basu A, Bryant S, Welti-Santos L, Maloney J, Chen S, Asmussen E, Solomon C and Ackermann E: 'RoBallet: exploring learning through expression in the arts through constructing in a technologically immersive environment', in Proceedings of International Conference on the Learning Sciences 2004, Los Angeles, California (2004).
Cavallo D, Sipitakiat A, Basu A and Bryant S: 'Opening pathways to higher education through engineering projects', in Proceedings of the American Society of Engineering Education 2004, Salt Lake City, Utah (2004).
Machover T: 'Toy symphony', Boosey & Hawkes, New York and London (2002/3) — http://www.toysymphony.net
Tomaino C: 'How music can reach the silenced brain', Cerebrum: The Dana Forum on Brain Science, 4, No 1, pp 23—24 (2002).
Lim M M <nt>et al</nt>: 'Ventral striatopallidal oxytocin and vasopresson V1a receptors in the monogamous prairie vole', J Comp Neurol, 468, No 4, pp 555—570 (January 2004).
Barsalou L W, Niedenthal P M, Barbey A K and Ruppert J A: 'Social embodiment, in the psychology of learning and motivation: advances in research theory', Ross B H (Ed), 43, Academic Press, Elsevier Science, USA (2003).
Minsky M: 'The Society of Mind', Simon & Schuster, New York, NY (1985).
Kay A: 'Computer software', Scientific American, 251, No 3, pp 41—47 (September 1984).
Winnicott D W: 'Playing and reality', London, Tavistock (1971).
Greenberg J R and Mitchell S A: 'Object Relations in Psycho-Analytic Theory', Harvard Press (1983).
Resnick M: 'Thinking Like a Tree (and Other Forms of Ecological Thinking', International Journal of Computers for Mathematical Learning, 8, No 1, pp 43—62 (2003).
Wilenski U: 'Statistical mechanics for secondary school: The GasLab Modeling Toolkit', International Journal of Computers for Mathematical Learning, 8, No 1, pp 1—41 (2003).
Papert S: 'Foreword', Special Issue on Multi-Agent Programming, International Journal of Computers for Mathematical Learning, 8, No 1, (2003).
Resnick M: 'Turtles, Termites and Traffic Jams', The MIT Press (1994).
Rogoff B: 'Observing sociocultural activity on three planes: participatory appropriation, guided participation and apprencticeship', in Wertsch J V, Rio P D and Alvarez A (Eds): 'Sociocultural Studies of the Mind', Cambridge University Press, Cambridge, pp 139—164 (1995).
Papert S: 'An exploration in the space of mathematics educations', International Journal of Computers for Mathematical Learning, 1, No 1, pp 95—123 (1996).
Cavallo D: 'Epistemology all the way down', EuroLogo 2003, Porto, Portugal (2003).
Cavallo D: 'Emergent design and leaning environments: building on indigenous knowledge', IBM Systems Journal, 39, Nos 3 and 4, pp 768—781 (2000).
Smith B, Bender W, Driscoll J, Endter I, Turpeinen M and Quan D: 'Silver stringers and junior journalists: active information producers', IBM Systems Journal, 39, Nos 3 and 4 2000.
Monroy-Gomez C: 'eRadio,' MIT SM Thesis (2004).
US Department of Education, National Center for Education Statistics, The Condition of Education 2000, NCES 2000-62, US Government Printing Office, Washington, DC (2000).
About this article
Cite this article
Picard, R.W., Papert, S., Bender, W. et al. Affective Learning — A Manifesto. BT Technology Journal 22, 253–269 (2004). https://doi.org/10.1023/B:BTTJ.0000047603.37042.33
Issue Date:
DOI: https://doi.org/10.1023/B:BTTJ.0000047603.37042.33