Giving Form to Smart Objects: Exploring Intelligence as an Interaction Design Material

  • Marco C. RozendaalEmail author
  • Maliheh Ghajargar
  • Gert Pasman
  • Mikael Wiberg
Part of the Human–Computer Interaction Series book series (HCIS)


Artificial intelligence (AI) has recently been highlighted as a design material in the HCI community. This acknowledgement is a call for interaction designers to consider intelligence as a resource for design. While this view is valid and well-grounded, it brings with it a need to better understand how intelligence as a design material can be used in formgiving practices. This chapter seeks to address this need by suggesting a new approach that integrates AI in the designer’s toolkit. This approach considers intelligence as being part of, and expressed through, an object's character, hereby integrating artificial intelligence into a product's form. We describe and discuss this approach by presenting and reflecting on our experiences in a design course where students were asked to give form to intelligent everyday objects in three iterative design cycles. We discuss the implications of our approach and findings within the frame of third wave HCI.


  1. Allmendinger G, Lombreglia R (2005) Four strategies for the age of smart services. Harv Bus Rev 83(10):131–145Google Scholar
  2. Amazon Echo (2017) Last checked 14 Sept 2017
  3. Apple HomePod (2017) Last checked 14 Sept 2017
  4. Bergström J, Brendon C, Frigo A, Mazé R, Vallgårda A (2010) Becoming materials: material forms and forms of practice. Digital Creativity 21(3):155–172CrossRefGoogle Scholar
  5. Bødker S (2006) When second wave HCI meets third wave challenges. In: Proceedings of the 4th Nordic conference on Human-computer interaction: changing roles (NordiCHI’06). ACM, New York, pp 1–8Google Scholar
  6. Bødker S (2015) Third-wave HCI, 10 years later—participation and sharing. Interactions 22(5):24–31. ACM, New YorkCrossRefGoogle Scholar
  7. Bødker S, Klokmose CN (2012) Dynamics in artifact ecologies. In: Proceedings of the 7th Nordic conference on Human-Computer Interaction: making sense through design. ACM, New York, pp 448–457CrossRefGoogle Scholar
  8. Cila N, Giaccardi E, Tynan O’Mahony F, Speed C, Caldwell M (2015) Thing-centered narratives: a study of object personas. Paper presented at Research Network for Design Anthropology Seminar 3: Collaborative Formation of Issues (January 2015), Aarhus, DenmarkGoogle Scholar
  9. Cila N, Smit I, Giaccardi E, Kröse B (2017) Products as agents: metaphors for designing the products of the IoT age. In: Proceedings of the 2017 CHI conference on human factors in computing systems. ACM, New York, pp 448–459CrossRefGoogle Scholar
  10. Crilly N, Moultrie J, Clarkson PJ (2009) Shaping things: intended consumer response and the other determinants of product form. Des Stud 30(3):224–254. Design, HelsinkiCrossRefGoogle Scholar
  11. Djajadiningrat T, Overbeeke K, Wensveen S (2002) But how, Donald, tell us how?: on the creation of meaning in interaction design through feedforward and inherent feedback. In: Proceedings of the 4th conference on designing interactive systems: processes, practices, methods, and techniques. ACM, New York, pp 285–291Google Scholar
  12. Dourish P (2004) Where the action is. MIT Press, Cambridge, MAGoogle Scholar
  13. Dourish P, Mazmanian M (2011) Media as material: information representations as material foundations for organizational practice. In: Proceedings of the third international symposium on process organization studies, pp 92–xxCrossRefGoogle Scholar
  14. Dove G, Halskov K, Forlizzi J, Zimmerman J (2017) UX design innovation: challenges for working with machine learning as a design material. In: Proceedings of the 2017 CHI conference on human factors in computing systems. ACM, New York, pp 278–288CrossRefGoogle Scholar
  15. Evans M, Sommerville S (2007) Seeing is believing: the challenge of product semantics in the curriculum. In: Shaping the future, proceedings of the international conference on engineering and product design education, 13–14 Sept 2007, Northumbria University, Newcastle Upon Tyne, UKGoogle Scholar
  16. Farooq U, Grudin JT (2016) Human-computer integration. Interactions 23(6):26–32CrossRefGoogle Scholar
  17. Ghajargar M (2017) Toward intelligent environments: supporting reflection with smart objects in the home. Interactions 24(4):60–62. ACM, New YorkCrossRefGoogle Scholar
  18. Giaccardi E (2015) Designing the connected everyday. Interactions 22(1):26–31CrossRefGoogle Scholar
  19. Giaccardi E, Cila N, Speed C, Caldwell M (2016) Thing ethnography: doing design research with non-humans. In: Proceedings of the 2016 ACM conference on designing interactive systems. ACM, New York, pp 377–387Google Scholar
  20. Gibson JJ (1979) The ecological approach to visual perception. Houghton Mifflin, BostonGoogle Scholar
  21. Google Home (2017) Last checked 14 Sept 2017
  22. Halse J, Brandt E, Clark B, Binder T (2010) Rehearsing the future. The Danish Design School Press, KøbenhavnGoogle Scholar
  23. Harrison S, Tatar D, Sengers P (2007) The three paradigms of HCI. In: Proceeding of CHI 2007. ACM, New YorkGoogle Scholar
  24. Heider F, Simmel M (1944) An experimental study of apparent behavior. Am J Psychol 57(2):243–259CrossRefGoogle Scholar
  25. Hoffman G, Ju W (2014) Designing robots with movement in mind. J Human-Robot Interact 3(1):89–122CrossRefGoogle Scholar
  26. Hoffman G, Kubat R, Breazeal C (2008) A hybrid control system for puppeteering a live robotic stage actor. In: The 17th IEEE international symposium on Robot and Human Interactive Communication, 2008. RO-MAN 2008. IEEE, pp 354–359Google Scholar
  27. Holmquist LE (2017) Intelligence on tap: artificial intelligence as a new design material. Interactions 24(4):28–33MathSciNetCrossRefGoogle Scholar
  28. WikiPedia Pixar’s Luxo ‘Junior’. Last checked 14 Sept 2017
  29. Jacucci G, Spagnolli A, Freeman J, Gamberini L (2014) Symbiotic interaction: a critical definition and comparison to other human-computer paradigms. In: International workshop on symbiotic interaction. Springer International Publishing, Cham, pp 3–20Google Scholar
  30. Janlert LE, Stolterman E (1997) The character of things. Des Stud 18(3):297–314CrossRefGoogle Scholar
  31. Janlert L-E, Stolterman E (2017) The meaning of interactivity—some proposals for definitions and measures. Human Comput Interact 32(3):103–138CrossRefGoogle Scholar
  32. Kaptelinin V, Nardi BA (2006) Acting with technology: activity theory and interaction design. MIT Press, Cambridge, MAGoogle Scholar
  33. Kortuem G, Kowsar F, Fitton D, Vasughi S (2010) Smart artifacts as building blocks for the internet of things. J Internet Comput IEEE 14(1):44–51CrossRefGoogle Scholar
  34. Krippendorff K (1989) Product semantics: a triangulation and four design theories. In: Väkevä S (ed) Product semantic ‘89. University of Industrial Arts, HelsinkiGoogle Scholar
  35. Krippendorff K (2006) The semantic turn: a new foundation for design. Taylor & Francis CRC, New YorkCrossRefGoogle Scholar
  36. Kuutti K, Bannon LJ (2014) The turn to practice in HCI: towards a research agenda. In: Proceedings of the 32nd annual ACM conference on Human factors in computing systems. ACM, New York, pp 3543–3552Google Scholar
  37. Laurel B (1997) Interface agents: metaphors with character. In: Friedman B (ed) Human values and the design of computer technology. Cambridge University Press, Cambridge, MA, pp 207–219Google Scholar
  38. Licklider JC (1960) Man-computer symbiosis. IRE trans Hum Factors Electron 1:4–11CrossRefGoogle Scholar
  39. Lim YK, Stolterman E, Jung H, Donaldson J (2007) Interaction gestalt and the design of aesthetic interactions. In: Proceedings of the 2007 conference on Designing pleasurable products and interfaces. ACM, New York, pp 239–254CrossRefGoogle Scholar
  40. Marenko B, van Allen P (2016) Animistic design: how to reimagine digital interaction between the human and the nonhuman. Digital Creativity 27(1):52–70CrossRefGoogle Scholar
  41. McKim RH (1980) Thinking visually: a strategy manual for problem solving. Wadsworth, BelmontGoogle Scholar
  42. Muller W (2001) Order and meaning in design. Lemma, UtrechtGoogle Scholar
  43. Niess J, Diefenbach S (2016) Communication styles of interactive tools for self-improvement. Psychol Well Being 6(1):3CrossRefGoogle Scholar
  44. Norman DA (1994) How might people interact with agents. Commun ACM 37(7):68–71CrossRefGoogle Scholar
  45. Papanek V (1971) Design for the Real World: human ecology and social change. Pantheon Books, New YorkGoogle Scholar
  46. Pasman G, Rozendaal M (2016) Exploring interaction styles through video. In: Proceedings of the 18th international conference on engineering and product design education (E&PDE16), Aalborg, Denmark, 8th–9th September 2016Google Scholar
  47. Petersen MG, Iversen OS, Krogh PG, Ludvigsen M (2004) Aesthetic Interaction: a pragmatist's aesthetics of interactive systems. In: Proceedings of the 5th conference on Designing interactive systems: processes, practices, methods, and techniques. ACM, New York, pp 269–276Google Scholar
  48. Pevsner N (1991) Pioneers of modern design: from William Morris to Walter Gropius. Penguin Books, LondonGoogle Scholar
  49. Raizman D (2004) A history of modern design: graphics and products since the industrial revolution. Laurence King Publishing, LondonGoogle Scholar
  50. Ramduny-Ellis D, Dix A, Evans M, Hare J, Gill S (2010) Physicality in design: an exploration. Des J 13(1):48–76Google Scholar
  51. Reeves B, Nass C (1996) How people treat computers, television, and new media like real people and places. CSLI Publications and Cambridge university press, Stanford/Cambridge, MA, pp 3–18Google Scholar
  52. Robles E, Wiberg M (2010) Texturing the material turn in interaction design. In: Proceedings of the fourth international conference on Tangible, embedded, and embodied interaction. ACM, New York, pp 137–144CrossRefGoogle Scholar
  53. Rosner DK (2012) The material practices of collaboration. In: Proceedings of the ACM 2012 conference on computer supported cooperative work. ACM, New York, pp 1155–1164Google Scholar
  54. Ross PR, Wensveen SA (2010) Designing behavior in interaction: using aesthetic experience as a mechanism for design. Int J Des 4(2):3–13Google Scholar
  55. Rozendaal MC (2016) Objects with intent: a new paradigm for interaction design. Interactions 23(3):62–65CrossRefGoogle Scholar
  56. Sabou M (2010) Smart artifacts: challenges for semantic web research, semantic web – interoperability. Usability Applicability 1:1–5Google Scholar
  57. Schön DA (1984) The reflective practitioner: how professionals think in action, vol 5126. Basic Books, New YorkGoogle Scholar
  58. Siegman AW, Feldstein S (eds) (2014) Nonverbal behavior and communication. In: Psychology press, OxfordGoogle Scholar
  59. Smets G, Overbeeke K, Gaver W (1994, April) Form-giving: expressing the nonobvious. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 79–84Google Scholar
  60. Taylor AS (2009) Machine intelligence. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 2109–2118Google Scholar
  61. Tholander J, Normark M, Rossitto C (2012) Understanding agency in interaction design materials. In: Proceedings of the SIGCHI conference on human factors in computing systems. ACM, New York, pp 2499–2508Google Scholar
  62. Vallgårda A (2014) Giving form to computational things: developing a practice of interaction design. Pers Ubiquit Comput 18(3):577–592CrossRefGoogle Scholar
  63. Vallgårda A, Redström J (2007) Computational composites. In: Proceedings of the SIGCHI conference on Human factors in computing systems. ACM, New York, pp 513–522CrossRefGoogle Scholar
  64. Verplank B (2009) Interaction design sketchbook: frameworks for designing interactive products and systems. Last access 14 Sept 2017
  65. Vihma S (1995) Products as representations: a semiotic and aesthetic study of design products. University of Arts & Design, HelsinkiGoogle Scholar
  66. Wiberg M (2014) Methodology for materiality: interaction design research through a material lens. Pers Ubiquit Comput 18(3):625–636CrossRefGoogle Scholar
  67. Wiberg M (2018) The materiality of interaction: notes on the materials of interaction design. MIT Press, Cambridge, MAGoogle Scholar
  68. Wiberg M, Robles E (2010) Computational compositions: aesthetics, materials, and interaction design. Int J Des 4(2):65–76Google Scholar
  69. Wiberg M, Ishii H, Dourish P, Vallgårda A, Kerridge T, Sundström P, Rosner D, Rolston M (2013) Materiality matters – experience materials. Interactions 20(2):54–57CrossRefGoogle Scholar
  70. Wooldridge M, Jennings NR (1995) Intelligent agents: theory and practice. Knowl Eng Rev 10(2):115–152CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Marco C. Rozendaal
    • 1
    Email author
  • Maliheh Ghajargar
    • 2
  • Gert Pasman
    • 1
  • Mikael Wiberg
    • 3
  1. 1.Delft University of TechnologyDelftThe Netherlands
  2. 2.Polytechnic University of TurinTurinItaly
  3. 3.Umeå UniversityUmeåSweden

Personalised recommendations