Skip to main content

Shake2Talk: Multimodal Messaging for Interpersonal Communication

  • Conference paper

Part of the Lecture Notes in Computer Science book series (LNISA,volume 4813)

Abstract

This paper explores the possibilities of using audio and haptics for interpersonal communication via mobile devices. Drawing on the literature on current messaging practises, a new concept for multimodal messaging has been designed and developed. The Shake2Talk system allows users to construct audio-tactile messages through simple gesture interactions, and send these messages to other people. Such messages could be used to communicate a range of meanings, from the practical (e.g. “home safely”, represented by the sound and sensation of a key turning in a lock) to the emotional (e.g. “thinking of you” represented by a heartbeat). This paper presents the background to this work, the system design and implementation and a plan for evaluation.

Keywords

  • haptics
  • audio
  • vibrotactile
  • multimodal interaction
  • mobile phones
  • messaging
  • remote communication
  • gesture recognition

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-3-540-76702-2_6
  • Chapter length: 12 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   59.99
Price excludes VAT (USA)
  • ISBN: 978-3-540-76702-2
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   79.99
Price excludes VAT (USA)

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Harper, R.: Towards a new Communications Genre. IEEE Computer On-Line Magazine, 99–101 (2005)

    Google Scholar 

  2. Harper, R., Hodges, S.: Beyond talk, beyond sound: Emotional expression and the future of mobile connectivity. In: Höflich, J., Hartmann, M. (eds.) Mobile Communication in Everyday Life: An Ethnographic View, Frank & Timme (2006)

    Google Scholar 

  3. Kasesniemi, E., Rautiainen, P.: Mobile Culture of Children and Teenagers in Finland. In: Katz, J.E., Aakhus, M.A. (eds.) Perpetual Contact: Mobile Communication, Private Talk, Public Performance, pp. 170–192. Cambridge University Press, Cambridge (2002)

    Google Scholar 

  4. Kopomaa, T.: The breakthrough of text messaging in Finland. In: Harper, R., Palen, L., Taylor, A. (eds.) The Inside Text: Social, Cultural and Design Perspectives on SMS, Springer, Heidelberg (2005)

    Google Scholar 

  5. Ling, R., Julsrud, T., Yttri, B.: Nascent communication genres within SMS and MMS. In: Harper, R., Palen, L., Taylor, A. (eds.) The Inside Text: Social, Cultural and Design Perspectives on SMS, Springer, Heidelberg (2005)

    Google Scholar 

  6. Taylor, A., Harper, R.: Age-old Practises in the ’New World’: A study of gift-giving between teenage mobile phone users. In: Proc. CHI 2002, pp. 439–446. ACM Press, New York (2002)

    Google Scholar 

  7. Kindberg, T., Spasojevic, M., Fleck, R.: I saw this and thought of you: Some social uses of camera phones. In: Proc. CHI 2005, pp. 1545–1548. ACM Press, New York (2005)

    Google Scholar 

  8. Thayer, S.: Social Touching. In: Schiff, W., Foulke, E. (eds.) Tactual Perception: A Sourcebook, Cambridge University Press, Cambridge (1982)

    Google Scholar 

  9. Jones, S.E., Yarborough, E.: A Naturalistic Study of The Meanings of Touch. Communication Monographs 52(1), 19–56 (1985)

    CrossRef  Google Scholar 

  10. Haans, A., IJsselsteijn, W.: Mediated social touch: a review of current research and future directions. Virtual Reality 9(2), 149–159 (2006)

    CrossRef  Google Scholar 

  11. DiSalvo, C., et al.: The Hug: An Exploration of Robotic Form For Intimate Communication. In: ROMAN 2003, pp. 403–408. IEEE, Los Alamitos (2003)

    Google Scholar 

  12. Chang, A., et al.: ComTouch: Design of a Vibrotactile Communication Device. In: Proc. DIS 2002, pp. 312–320. ACM press, New York (2002)

    Google Scholar 

  13. Rovers, A.F., van Essen, H.A.: HIM: A Framework for Haptic Instant Messaging. In: Ext. Abstracts CHI 2004, pp. 1313–1316. ACM Press, New York (2004)

    CrossRef  Google Scholar 

  14. Smith, J., MacLean, K.E.: Communicating Emotion Through a Haptic Link: Design Space and Methodology. International Journal of Human-Computer Studies (IJHCS), Special Issue on Affective Evaluation–Innovative Approaches 65(4), 376–387 (2007)

    Google Scholar 

  15. Brown, L.M., Brewster, S.A., Purchase, H.C.: A First Investigation into the Effectiveness of Tactons. In: Proc. World Haptics 2005, pp. 167–176. IEEE, Los Alamitos (2005)

    Google Scholar 

  16. Chan, A., MacLean, K., McGrenere, J.: Learning and Identifying Haptic Icons under Workload. In: Proc. World Haptics 2005, pp. 432–439. IEEE, Los Alamitos (2005)

    Google Scholar 

  17. Brewster, S.A., Wright, P.C., Edwards, A.D.N.: A detailed investigation into the effectiveness of earcons. In: Proc. ICAD 1992, pp. 471–498. Addison-Wesley, Reading (1992)

    Google Scholar 

  18. Gaver, W.W.: The SonicFinder: An Interface that Uses Auditory Icons. Human Computer Interaction 4(1) (1989)

    Google Scholar 

  19. Gaver, W.W.: Sound Support For Collaboration. In: Proc. ECSCW 1991, pp. 293–308. Kluwer Academic Publishers, Dordrecht (1991)

    Google Scholar 

  20. Isaacs, E., Walendowski, A., Ranganathan, D.: Hubbub: A sound-enhanced mobile instant messenger that supports awareness and opportunistic interactions. In: Proc. CHI 2002, pp. 179–186. ACM Press, New York (2002)

    Google Scholar 

Download references

Author information

Authors and Affiliations

Authors

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2007 Springer-Verlag Berlin Heidelberg

About this paper

Cite this paper

Brown, L.M., Williamson, J. (2007). Shake2Talk: Multimodal Messaging for Interpersonal Communication. In: Oakley, I., Brewster, S. (eds) Haptic and Audio Interaction Design. HAID 2007. Lecture Notes in Computer Science, vol 4813. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-540-76702-2_6

Download citation

  • DOI: https://doi.org/10.1007/978-3-540-76702-2_6

  • Publisher Name: Springer, Berlin, Heidelberg

  • Print ISBN: 978-3-540-76701-5

  • Online ISBN: 978-3-540-76702-2

  • eBook Packages: Computer ScienceComputer Science (R0)