Skip to main content

Structures, forms, and stuff: the materiality and medium of interaction

Personal and Ubiquitous Computing volume 18, pages 637–649 (2014)Cite this article

Abstract

Though information is popularly, and often academically, understood to be immaterial, nonetheless, we only encounter it in material forms, in books, on laptops, in our brains, in spoken language, and so forth. In the past decade, HCI has increasingly focused on the material dimensions of interacting with computational devices and information. This paper explores three major strands of this research—tangible user interfaces, theories of computational materiality, and craft-oriented approaches to HCI. We argue that each of these offers a formulation of the materiality of interaction: as physical, as metaphysical, or as tradition communicating. We situate these three formulations in relation to debates on the nature of media, from philosophical aesthetics (the ontology of art, in particular), media studies, and visual cultural studies. We argue that the formulations of materiality, information, and meaning from HCI and those from the humanities have deeper underlying similarities than may be expected and that exploring these similarities have two significant benefits. Such an analysis can benefit these differing threads in different ways, taking their current theories and adding to them. It also serves as a basis to import philosophical art concepts in a robust way into HCI, that is, not simply as prepackaged ideas to be applied to HCI, but rather as ideas always already enmeshed in productive and living debates that HCI is now poised to enter—to the benefit of both HCI and the humanities.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

References

  1. Miller D (2009) Stuff. Polity, Cambridge

    Google Scholar 

  2. Bardzell J (2011) Interaction criticism: an introduction to the practice. Interact Comput 23(6):604–621

    Article  MathSciNet  Google Scholar 

  3. Norman D (1999) Affordance, conventions, and design. Interactions 6(3):38–43

    Article  Google Scholar 

  4. Broken Probes: Towards the Design of Worn Media (in this issue)

  5. Ishii H, Ullmer B (1997) Tangible bits: towards seamless integration interfaces between people, atoms, and bits. In: Proceedings of CHI’97, New York, ACM pp 234–241

  6. Vallgårda A, Redstöm J (2007) Computational composites. In: Proceedings of CHI’97, New York, ACM pp 513–522

  7. Dourish P, Mazmanian M (2011) Media as material: information representation as material foundations for organizational practice. Third international symposium on process organization studies, Corfu, Greece pp 1–24

  8. Schön D (1983) The reflective practitioner. Basic Books, New York

    Google Scholar 

  9. Rosner D, Ryokai K (2009) Reflections on craft: probing the creative process of everyday knitters. In: Proceedings of creativity and cognition’09, New York, ACM pp 195–204

  10. Rosner, D, Blanchette, J F, Buechley, L, Dourish, P, Mazmanian, M (2012) From materials to materiality: connecting practice and theory in HCI. In: Proceedings of CHI’2012 extended abstracts, ACM pp 2787–2790

  11. Bardzell S, Rosner D, Bardzell J (2012) The craft of designing for quality: integrity, creativity, and public sensibility. In: Proceedings of DIS’12, ACM

  12. Janlert LE, Stolterman E (1997) The character of things. Des Stud 18:297–314

    Article  Google Scholar 

  13. Bergström J, Clark B, Frigo A, Mazé R, Redström J, Vallgårda A (2010) Becoming materials: material forms and forms of practice. Digit Creativity 21(3):155–172

    Article  Google Scholar 

  14. Barnard M (2001) Approaches to understanding visual culture. PALGRAVE, New York

    Google Scholar 

  15. Cavell S (1979) The world viewed: reflections on the ontology of film. Harvard University Press, Cambridge

    Google Scholar 

  16. Davies D (2005) Medium in art. In: Levinson J (ed) The Oxford handbook of aesthetics. Oxford University Press, Oxford

    Google Scholar 

  17. Herwitz D (2008) Aesthetics: key concepts in philosophy. Continuum, London

    Google Scholar 

  18. Greenberg C (1961) The new sculpture. In: Art and culture: critical essays, vol 212. Beacon Press, Boston

  19. Manovich L (2001) Language of new media. MIT Press, Cambridge

    Google Scholar 

  20. Benjamin W (2008) The work of art in the age of mechanical reproduction. Penguin Books, New York

    Google Scholar 

  21. Lessing GE (1874) Laocoön. (trans: Frothingham E). Roberts Brothers, Boston

  22. Aristotle (1961) Poetics. (trans: Butcher SH). Macmillan and Co, New York

  23. Smith GC (1995) The marble answering machine. The hand that rocks the cradle (May/June), pp 60–65

  24. Fitzmaurice G, Ishii H, Buxton B (1995) Bricks: laying the foundations for graspable user interfaces. In: Proceedings of CHI’95, ACM pp 442–449

  25. Ishii H (2008) Tangible bits: beyond pixels. In: Proceedings of TEI’08, New York, ACM pp 15–25

  26. Ishii H, Lakatos D, Bonanni L, Labrune J-B (2012) Radical atoms: beyond tangible bits, towards transformable materials. Interactions 19(1):38–51

    Article  Google Scholar 

  27. Hornecker E (2012) Beyond affordances: tangibles’ hybrid nature. In: Proceedings of TEI’12, ACM pp 175–182

  28. Vallgårda A, Sokoler T (2010) A material strategy: exploring material properties in computers. Int J Des 4(3):1–14

    Google Scholar 

  29. Robles E, Wiberg M (2010) Texturing the “material turn” in interaction design. In: Proceedings of TEI’10, New York, ACM pp 137–144

  30. Giving form to computers: the material practice of interaction design (in this issue)

  31. Crafting interaction: the epistemology of modern programming (in this issue)

  32. Hybrid crafting: towards an integrated practice of crafting with physical and digital components (in this issue)

  33. McCullough M (1996) Abstracting craft: the practiced digital hand. MIT Press, Cambridge

    Google Scholar 

  34. Sennett R (2008) The craftsman. Yale University Press, New Haven

    Google Scholar 

  35. Rosner D, Taylor A (2011) Antiquarian answers: book restoration as a resource for design. In: Proceedings of CHI’11, New York, ACM pp 2665–2668

  36. Buechley L, Mako Hill B (2010) LilyPad in the wild: how hardware’s long tail is supporting new engineering and design communities. In: Proceedings of DIS’10, New York, ACM pp 199–207

  37. Buechley L, Hendrix S, Eisenberg M (2009) Paints, paper, and programs: first steps toward the computational sketchbook. In: Proceedings of TEI ‘09, ACM pp 9–12

  38. Bødker S (2006) When second wave HCI meets third wave challenges. In: Proceedings of NordiCHI, New York, ACM

  39. Harrison S, Sengers P, Tatar D (2011) Making epistemological trouble: third-paradigm HCI as successor science. Interact Comput 23(5):385–392

    Article  Google Scholar 

  40. Rogers Y (2012) HCI theory: classical, modern, and contemporary. Synthesis lectures on human-centered informatics. Morgan Claypool

  41. Arias E, Eden H, Fischer G (1997) Enhancing communication, facilitating shared understanding, and creating better artifacts by integrating physical and computational media for design. In: Proceedings of DIS’97, New York, ACM

  42. Fernaeus Y (2007) Let’s make a digital patchwork designing for children’s creative play with programming materials. Dissertation, Stockholm University

  43. Bolter J, Grusin R (2000) Remediation. MIT Press, Cambridge

    Google Scholar 

  44. Carroll N (1985) Specificity of media in the arts. J Aesthet Educ 19(4):5–20

    Article  Google Scholar 

  45. Kracauer, S (2004) [1960] Basic concepts. In: Braudy L, Cohen M (eds) From theory of film, excerpted in film theory and criticism. Oxford University Press, Oxford, pp 143–153

  46. Bazin A (2004) What is cinema?. University of California Press, Berkeley

    Google Scholar 

  47. Levinson J (1980) What a musical work is. J Philos 77(1):5–28

    Article  MathSciNet  Google Scholar 

  48. Doane MA (2007) The indexical and the concept of medium specificity. Differences 18(1):397–408

    Google Scholar 

  49. Ratti C, Wang Y (2004) Tangible user interfaces (TUIs): a novel paradigm for GIS. Trans GIS’04 8(4):407–421

    Google Scholar 

  50. Piper B, Ratti C, Ishii H (2002) Illuminating clay: a 3-d tangible interface for landscape analysis. In: Proceedings of CHI’02, ACM pp 355–362

  51. Ishii H, Ratti C, Piper B, Wang Y, Biderman A, Ben-Joseph E (2004) Brining clay and sand into digital design—continuous tangible user interfaces. BT Technol J 22(4):287–299

    Article  Google Scholar 

  52. Piper B, Ratti C, Ishii H (2002) Illuminating clay: a tangible interface with potential GRASS applications. In: Proceedings of open source GIS-GRASS user conf ‘02, University of Trento, Trento

Download references

Author information

Authors and Affiliations

  1. Indiana University, Bloomington, 919 E 10th Street, Bloomington, IN, 47401, USA

    Shad Gross, Jeffrey Bardzell & Shaowen Bardzell

Authors
  1. Shad Gross
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Jeffrey Bardzell
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Shaowen Bardzell
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Shad Gross.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Gross, S., Bardzell, J. & Bardzell, S. Structures, forms, and stuff: the materiality and medium of interaction. Pers Ubiquit Comput 18, 637–649 (2014). https://doi.org/10.1007/s00779-013-0689-4

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s00779-013-0689-4

Keywords

  • Materiality
  • Medium
  • HCI
  • Design
  • Art
  • Philosophy
  • Aesthetic interaction