Advertisement

Expression of Emotions in Virtual Humans Using Lights, Shadows, Composition and Filters

  • Celso de Melo
  • Ana Paiva
Part of the Lecture Notes in Computer Science book series (LNCS, volume 4738)

Abstract

Artists use words, lines, shapes, color, sound and their bodies to express emotions. Virtual humans use postures, gestures, face and voice to express emotions. Why are they limiting themselves to the body? The digital medium affords the expression of emotions using lights, camera, sound and the pixels in the screen itself. Thus, leveraging on accumulated knowledge from the arts, this work proposes a model for the expression of emotions in virtual humans which goes beyond embodiment and explores lights, shadows, composition and filters to convey emotions. First, the model integrates the OCC emotion model for emotion synthesis. Second, the model defines a pixel-based lighting model which supports extensive expressive control of lights and shadows. Third, the model explores the visual arts techniques of composition in layers and filtering to manipulate the virtual human pixels themselves. Finally, the model introduces a markup language to define mappings between emotional states and multimodal expression.

Keywords

Expression of Emotions Virtual Humans Expression in the Arts Light Expression Screen Expression 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. 1.
    Collingwood, R.: Art and Craft. In: Feagin, S., Maynard, P. (eds.) Aesthetics, pp. 215–220. The Oxford University Press, New York, USA (1997)Google Scholar
  2. 2.
    Bell, C.: The Metaphysical Hypothesis. In: Feagin, S., Maynard, P. (eds.) Aesthetics, pp. 158–159. The Oxford University Press, New York, USA (1997)Google Scholar
  3. 3.
    Arijon, D.: Grammar of the Film Language. Silman-James Press, California, USA (1976)Google Scholar
  4. 4.
    Millerson, G.: Lighting for Television and film, 3rd edn. Focal Press, Oxford, UK (1999)Google Scholar
  5. 5.
    Birn, J.: [digital] Lighting and Rendering, 2nd edn. New Riders, California, USA (2006)Google Scholar
  6. 6.
    Juslin, P., Sloboda, J.: Music and Emotion: theory and research. Oxford University Press, New York, USA (2001)Google Scholar
  7. 7.
    Zettl, H.: Sight Sound Motion: Applied Media Aesthetics. Wadsworth Publishing, California, USA (2004)Google Scholar
  8. 8.
    Sayre, H.: A World of Art, 5th edn. Prentice-Hall, Englewood Cliffs (2007)Google Scholar
  9. 9.
    Gratch, J., Rickel, J., Andre, E., Badler, N., Cassell, J., Petajan, E.: Creating Interactive Virtual Humans: Some Assembly Required. IEEE Intelligent Systems 17(4), 54–63 (2002)CrossRefGoogle Scholar
  10. 10.
    Orthony, A.: The Cognitive Structure of Emotions. Cambridge University Press, New York, USA (1990)Google Scholar
  11. 11.
    Batteux, A.: The Fine Arts Reduced to a Single Principle. In: Feagin, S., Maynard, P. (eds.) Aesthetics, pp. 102–104. The Oxford University Press, New York, USA (1997)Google Scholar
  12. 12.
    Clifford, G.: Art as a Cultural System. In: Feagin, S., Maynard, P. (eds.) Aesthetics, pp. 109–118. The Oxford University Press, New York, USA (1997)Google Scholar
  13. 13.
    Kant, I.: Art and Genius. In: Feagin, S., Maynard, P. (eds.) Aesthetics, pp. 180–192. The Oxford University Press, New York, USA (1997)Google Scholar
  14. 14.
    Elliot, R.: Aesthetic Theory and the Experience of Art. In: Feagin, S., Maynard, P. (eds.) Aesthetics, pp. 278–288. The Oxford University Press, New York, USA (1997)Google Scholar
  15. 15.
    Cassell, J.: Nudge, Nudge, Wink, Wink: Elements of Face-to-Face Conversation for Embodied Conversational Agents. In: Cassell, J., Sullivan, J., Prevost, S., Churchill, E. (eds.) Embodied Conversational Agents, pp. 1–27. The MIT Press, Massachusetts, USA (2000)Google Scholar
  16. 16.
    Noh, J., Neumann, U.: A Survey of Facial Modeling and Animation Techniques. USC Technical Report 99-705 (1998)Google Scholar
  17. 17.
    Schroder, M.: Speech and emotion research: an overview of research frameworks and a dimensional approach to emotional speech synthesis. PhD thesis. vol. 7 of Phonus, Research Report of the Institute of Phonetics, Saarland University (2004)Google Scholar
  18. 18.
    Amaya, K., Bruderlin, A., Calvert, T.: Emotion from motion. In: Proceedings Graphics Interface 1996, pp. 222–229 (1996)Google Scholar
  19. 19.
    Chi, D., Costa, M., Zhao, L., Badler, N.: The EMOTE model for effort and shape. In: Proceedings of SIGGRAPH 2000, pp. 173–182 (2000)Google Scholar
  20. 20.
    Hartmann, B., Mancini, A., Pelachaud, C.: Implementing Expressive Gesture Synthesis for Embodied Conversational Agents. In: Gibet, S., Courty, N., Kamp, J.-F. (eds.) GW 2005. LNCS (LNAI), vol. 3881, Springer, Heidelberg (2006)CrossRefGoogle Scholar
  21. 21.
    de Melo, C., Paiva, A.: Environment Expression: Expressing Emotions through Cameras, Lights and Music. In: Tao, J., Tan, T., Picard, R.W. (eds.) ACII 2005. LNCS, vol. 3784, pp. 715–722. Springer, Heidelberg (2005)CrossRefGoogle Scholar
  22. 22.
    Hunt, R.: The Reproduction of Colour, 6th edn. John Wiley & Sons, Ltd., Sussex, UK (2004)Google Scholar
  23. 23.
    Fraser, T., Banks, A.: Designer’s Color Manual: The Complete Guide to Color Theory and Application. Chronicle Books, California, USA (2004)Google Scholar
  24. 24.
    Moller, R., Haines, E.: Real-Time Rendering, 2nd edn. AK Peters Ltd., Massachusetts, USA (2002)Google Scholar
  25. 25.
    St-Laurent, S.: Shaders for Game Programmers and Artists. Thomson Course Technology, Massachusetts, USA (2004)Google Scholar
  26. 26.
    de Melo, C., Paiva, A.: Multimodal Expression in Virtual Humans. Computer Animation and Virtual Worlds Journal. 17(3), 239–248 (2006)CrossRefGoogle Scholar
  27. 27.
    Picard, R.: Affective Computing. The MIT Press, Massachusetts, USA (1997)Google Scholar
  28. 28.
    SMIL: Synchronized Multimedia Integration Language (SMIL) (2007), http://www.w3.org/AudioVideo/

Copyright information

© Springer-Verlag Berlin Heidelberg 2007

Authors and Affiliations

  • Celso de Melo
    • 1
  • Ana Paiva
    • 1
  1. 1.IST-Technical University of Lisbon and INESC-ID, Av. Prof. Cavaco Silva, Taguspark, 2780-990 Porto SalvoPortugal

Personalised recommendations