Skip to main content
SpringerLink
Log in

The Interaction Engine

  • Chapter
  • First Online:
Design Thinking Research

Part of the book series: Understanding Innovation ((UNDINNO))

Abstract

The Interaction Engine is a framework for prototyping interactive, connected devices based on widely available single-board Linux computers. With microcontrollers, networking, and modular open-source software, these modules enable interaction modalities such as audio, video, tangible, and digital interfaces to be embedded into forms that go beyond traditional computing. In this paper, we outline the hardware and software components that make up the general Interaction Engine framework and discuss its benefits for interaction designers. We provide an illustrative case study of the Interaction Engine in use. We ran workshops to introduce designers to the Interaction Engine framework and we describe the projects where they subsequently employed Interaction Engines to understand issues and opportunities presented by this model. In describing the framework and case studies, we intend to shift designer’s thinking of computer as product to computer as material to create new interactive devices.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 169.00
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 219.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 219.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

References

  • Acholonu, U. C. (2012). Techonology [sic] bricolage exploring the tension between convention and innovation when problem-solving with technology. Stanford University.

    Google Scholar 

  • Adafruit. (2016). Adafruit-occidentalis. Retrieved August 1, 2016, from https://github.com/adafruit/Adafruit-Occidentalis

  • Amazon. (2016a). Amazon dash button. Retrieved July 31, 2016, from https://www.amazon.com/Dash-Buttons/b?ie=UTF8&node=10667898011

  • Amazon. (2016b). Amazon echo. Retrieved July 30, 2016, from https://www.amazon.com/Amazon-Echo-Bluetooth-Speaker-with-WiFi-Alexa/dp/B00X4WHP5E

  • Aminzade, D., Hartmann, B., Doorley, S., et al. (2008). Interactive digital murals at San Jose Museum of Art.

    Google Scholar 

  • ARM. (2016). Cortex-M Series. Retrieved July 31, 2016, from http://www.arm.com/products/processors/cortex-m/index.php

  • Ballagas, R., Ringel, M., Stone, M., & Borchers, J. (2003). iStuff: A physical user interface toolkit for ubiquitous computing environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 537–544). New York: ACM.

    Google Scholar 

  • Banzi, M., & Shiloh, M. (2014). Make: Getting started with Arduino: The open source electronics prototyping platform. Sebastopol: Maker Media.

    Google Scholar 

  • Barragán, H. (2004). Wiring: Prototyping physical interaction design.

    Google Scholar 

  • Beigl, M., Gellersen, H.-W., & Schmidt, A. (2001). Mediacups: Experience with design and use of computer-augmented everyday artefacts. Computer Networks, 35, 401–409. doi:10.1016/S1389-1286(00)00180-8.

    Article  Google Scholar 

  • Berdahl, E., & Ju, W. (2011). Satellite CCRMA: A musical interaction and sound synthesis platform. In Proceedings of the International Conference on New Interfaces for Musical Expression (pp. 173–178).

    Google Scholar 

  • Borthwick, M. (2016). Building museum exhibits with open hardware: Raspberry Pi & Arduino.

    Google Scholar 

  • Bostock, M. D3.js—Data-driven documents. Retrieved July 26, 2016, from https://d3js.org/

  • Buechley, L., Eisenberg, M., Catchen, J., & Crockett, A. (2008) The LilyPad Arduino: Using computational textiles to investigate engagement, aesthetics, and diversity in computer science education. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 423–432). New York: ACM.

    Google Scholar 

  • Buechley, L., Elumeze, N., & Eisenberg, M. (2006). Electronic/computational textiles and children’s crafts. In Proceedings of the 2006 Conference on Interaction Design and Children (pp. 49–56). New York: ACM.

    Google Scholar 

  • Buechley, L., & Hill, B. M. (2010). LilyPad in the wild: How hardware’s long tail is supporting new engineering and design communities. In Proceedings of the 8th ACM Conference on Designing Interactive Systems (pp. 199–207). New York: ACM.

    Google Scholar 

  • Bush, V. (1989). As we may think. In Perspectives on the computer revolution (p. 49).

    Google Scholar 

  • Chan, L., Hsieh, C.-H., Chen, Y.-L., Yang, S., Huang, D.-Y., Liang, R.-H., et al. (2015) Cyclops: Wearable and single-piece full-body gesture input devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 3001–3009). New York: ACM.

    Google Scholar 

  • Charlesworth, T., Ford, H., Milton, L., Mortensson, T., Pedlingham, J., Knibbe, J., et al. (2015). TellTale: Adding a polygraph to everyday life. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 1693–1698). New York: ACM.

    Google Scholar 

  • Chung, E. S., Hong, J. I., Lin, J., Prabaker, M. K., & Landay, J. A. (2004). Development and evaluation of emerging design patterns for ubiquitous computing. In Proceedings of the 5th conference on designing interactive systems: processes, practices, methods, and techniques (pp. 233–242). New York: ACM.

    Google Scholar 

  • Derry, S. (1990). Flexible cognitive tools for problem solving instruction. In Annual meeting of the American Educational Research Association, Boston, MA.

    Google Scholar 

  • DiCola, T. (2015). Setting up WiFi with BeagleBone Black|Adafruit Learning System. Retrieved July 30, 2016, from https://learn.adafruit.com/setting-up-wifi-with-beaglebone-black/overview

  • Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwarz, D. L., & Klemmer, S. R. (2010). Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. ACM Transactions on Computer-Human Interaction TOCHI, 17, 18. doi:10.1145/1879831.1879836.

    Google Scholar 

  • Electric Imp, Inc. (2016). Electric Imp—Electric imp hardware. Retrieved July 31, 2016, from https://www.electricimp.com/platform/hardware/

  • Fitbit. (2016). Fitbit official site. Retrieved July 31, 2016, from https://www.fitbit.com/

  • Frei, P., Su, V., Mikhak, B., & Ishii, H. (2000). Curlybot: Designing a new class of computational toys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 129–136). New York: ACM.

    Google Scholar 

  • Gaver, W., Michael, M., Kerridge, T., Wilkie, A., Boucher, A., Ovalle, L., et al. (2015). Energy babble: Mixing environmentally-oriented internet content to engage community groups. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1115–1124). New York: ACM.

    Google Scholar 

  • Gibb, A. (2009). Bug labs: Hacks and apps. Linux J, 2009, 4.

    Google Scholar 

  • Gibb, A. M. (2010). New media art, design, and the Arduino microcontroller: A malleable tool. Pratt Institute.

    Google Scholar 

  • Greenberg, S., & Fitchett, C. (2001). Phidgets: Easy development of physical interfaces through physical widgets. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (pp. 209–218). New York: ACM.

    Google Scholar 

  • Hardkernal. (2016a). ODROID U3. Retrieved July 30, 2016, from http://www.hardkernel.com/main/products/prdt_info.php?g_code=g138745696275

  • Hardkernal. (2016b). ODROID C1. Retrieved July 30, 2016, from http://www.hardkernel.com/main/products/prdt_info.php?g_code=G141578608433

  • Harris-Cronin, S. (2008). High resolution video playback.

    Google Scholar 

  • Hartmann, B., Klemmer, S. R., Bernstein, M., Abdulla, L., Burr, B., Robinson-Mosher, A., et al. (2006). Reflective physical prototyping through integrated design, test, and analysis. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (pp. 299–308). New York: ACM.

    Google Scholar 

  • Hartmann, B., & Wright, P. K. (2013). Designing bespoke interactive devices. Computer, 46, 85–89. doi:10.1109/MC.2013.274.

    Article  Google Scholar 

  • Hennigh-Palermo, S. G. (2015). p5bots. Retrieved July 21, 2016, from https://github.com/sarahgp/p5bots

  • Huang, A. B. (2008). Chumby: An experiment in hackable pervasive computing. IEEE Pervasive Computing, 7, 55–62.

    Article  Google Scholar 

  • IBM. (2016) watson-developer-cloud/text-to-speech-nodejs. Retrieved July 26, 2016, fromhttps://github.com/watson-developer-cloud/text-to-speech-nodejs

  • Igoe, T. (2014). Lab: Serial communication with Node.js—ITP physical computing.

    Google Scholar 

  • Itseez. (2016). OpenCV|OpenCV. Retrieved July 26, 2016, from http://opencv.org/

  • Jibo, Inc. (2016). Jibo—The worlds first social robot. Retrieved July 31, 2016, from https://www.jibo.com/

  • Jonassen, D. H. (1994). Technology as cognitive tools: Learners as designers. ITForum Paper, 1, 67–80.

    Google Scholar 

  • Kaufmann, B., & Buechley, L. (2010). Amarino: A toolkit for the rapid prototyping of mobile ubiquitous computing. In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (pp. 291–298). New York: ACM.

    Google Scholar 

  • Kay, A. (1998). Alan Kay on messaging.

    Google Scholar 

  • Kubitza, T., Schmidt, A., Pohl, N., Petrelli, D., Dingler, T., & Dulake, N. (2013). Tools and methods for creating interactive artifacts. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (pp. 385–388). New York: ACM.

    Google Scholar 

  • Kuniavsky, M. (2010). Smart things: Ubiquitous computing user experience design. Amsterdam: Elsevier.

    Google Scholar 

  • Landin, H. (2005). Fragile and magical: Materiality of computational technology as design material. In Proceedings of the 4th Decennial Conference on Critical Computing: Between Sense and Sensibility (pp. 117–120). New York: ACM.

    Google Scholar 

  • Langer, M., & Alderman, J. (2016). Open hardware belongs in your museum. In Museums and the Web 2016. Los Angeles, CA.

    Google Scholar 

  • Lee, J. C., Avrahami, D., Hudson, S. E., Forlizzi, J., Dietz, P. H., & Leigh, D. (2004). The Calder Toolkit: wired and wireless components for rapidly prototyping interactive devices. In Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (pp. 167–175). New York: ACM.

    Google Scholar 

  • Levis, P., Madden, S., Polastre, J., Szewczyk, R., Whitehouse, K., Woo, A., et al. (2005). TinyOS: An operating system for sensor networks. In: W. Weber, J. M. Rabaey, E. Aarts (Eds.), Ambient intelligence (pp. 115–148). Berlin: Springer.

    Google Scholar 

  • Lin, F. X., Rahmati, A., & Zhong, L. (2010). Dandelion: A framework for transparently programming phone-centered wireless body sensor applications for health. In Wireless health 2010 (pp. 74–83). New York: ACM.

    Chapter  Google Scholar 

  • Marshall, M. T., Dulake, N., Petrelli, D., & Kockelkorn, H. (2015). From the deposit to the exhibit floor: An exploration on giving museum objects personality and social life. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 1917–1922). New York: ACM.

    Google Scholar 

  • Martelaro, N., Shiloh, M., & Ju, W. (2016). The interaction engine: Tools for prototyping connected devices. In Proceedings of the TEI ‘16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 762–765). New York: ACM.

    Google Scholar 

  • Marvell. (2016). Kinoma Create. Retrieved July 31, 2016, from http://kinoma.com/create/

  • mBed. (2016). Nordic nRF51822. Retrieved July 31, 2016, from https://developer.mbed.org/platforms/Nordic-nRF51822/

  • McCarthy, L. (2016). p5.js (Computer Program).

    Google Scholar 

  • McComb, G. (2003). Robot builder’s bonanza (3rd ed.). New York: McGraw-Hill.

    Google Scholar 

  • McGrath, W., Etemadi, M., Roy, S., & Hartmann, B. (2015). Fabryq: Using phones as gateways to prototype internet of things applications using web scripting. In Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (pp. 164–173). New York: ACM.

    Google Scholar 

  • Mellis, D., Banzi, M., Cuartielles, D., & Igoe, T. (2007). Arduino: An open electronic prototyping platform. In Proceedings of CHI.

    Google Scholar 

  • Mellis, D. A., & Buechley, L. (2014). Do-it-yourself cellphones: An investigation into the possibilities and limits of high-tech diy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1723–1732). New York: ACM.

    Google Scholar 

  • Mellis, D. A, Jacoby, S., Buechley, L., Perner-Wilson, H., & Qi, J. (2013). Microcontrollers as material: Crafting circuits with paper, conductive ink, electronic components, and an “untoolkit.” In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (pp. 83–90). New York: ACM.

    Google Scholar 

  • Merrill, D., Kalanithi, J., & Maes, P. (2007). Siftables: Towards sensor network user interfaces. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction (pp. 75–78). New York: ACM.

    Google Scholar 

  • Millner, A., & Baafi, E. (2011). Modkit: Blending and extending approachable platforms for creating computer programs and interactive objects. In Proceedings of the 10th International Conference on Interaction Design and Children (pp. 250–253). New York: ACM.

    Google Scholar 

  • Norman, D. A. (2013). The design of everyday things: Revised and expanded edition. New York: Basic books.

    Google Scholar 

  • O’Sullivan, D., & Igoe, T. (2004). Physical computing: Sensing and controlling the physical world with computers. Boston, MA: Course Technology Press.

    Google Scholar 

  • Output Arts. (2012). I have a message for you…

    Google Scholar 

  • Overholt, D., Møbius, N. “DZL”. (2013). Embedded audio without beeps: Synthesis and sound effects from cheap to steep. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (pp. 361–364). New York: ACM.

    Google Scholar 

  • Palazzetti, E. (2015). Getting Started with UDOO. Birmingham: Packt.

    Google Scholar 

  • Punch Through Design. (2016). LightBlue Bean. Retrieved July 31, 2016, from https://punchthrough.com/bean

  • Qi, J., & Buechley, L. (2010). Electronic popables: Exploring paper-based computing through an interactive pop-up book. In Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 121–128). New York: ACM.

    Chapter  Google Scholar 

  • Raghavan, P., Lad, A., & Neelakandan, S. (2005). Embedded linux system design and development. Boca Raton: CRC Press.

    Book  Google Scholar 

  • Redmer, A., & Stoeveken, T. (2016). MJPEG-STREAMER.

    Google Scholar 

  • Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., Kramer, K., et al. (1998). Digital manipulatives: New toys to think with. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 281–287). New York: ACM Press/Addison-Wesley.

    Google Scholar 

  • Riliskis, L., & Levis, P. (2014). Ravel a framework for embedded-gateway-cloud applications. In Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems (pp. 344–345). New York: ACM.

    Google Scholar 

  • Roman, M., Al-Muhtadi, J., Ziebart, B., Campbell, R., & Mickunas, M. D. (2003). System support for rapid ubiquitous computing application development and evaluation. In System Support for Ubiquitous Computing Workshop (UbiSys’ 03) in conjunction with UbiComp.

    Google Scholar 

  • Rozin, D. (1999). Wooden mirror.

    Google Scholar 

  • Russell, S. C. (2012). Pi and Arduino in action. The MagPi, 4–6.

    Google Scholar 

  • Sculley, J. (1987). Odyssey: Pepsi to apple, a journey of adventure, ideas, and the future. New York: Harper & Row.

    Google Scholar 

  • Sheppard, S. D. (1992). Mechanical dissection: An experience in how things work. In Proceedings of the Engineering Education: Curriculum Innovation & Integration, January 6–10.

    Google Scholar 

  • Sipitakiat, A., & Blikstein, P. (2013). Interaction design and physical computing in the era of miniature embedded computers. In Proceedings of the 12th International Conference on Interaction Design and Children (pp. 515–518). New York: ACM.

    Chapter  Google Scholar 

  • Spark Labs, Inc. (2016) Particle. In Particle. Retrieved July 31, 2016, from https://www.particle.io/prototype

  • Stankovic, J. A., Lee, I., Mok, A., & Rajkumar, R. (2005). Opportunities and obligations for physical computing systems. Computer, 38, 23–31. doi:10.1109/MC.2005.386.

    Article  Google Scholar 

  • Stiller, E. (2009). Teaching programming using bricolage. Journal of Computing Sciences in Colleges, 24, 35–42.

    Google Scholar 

  • Tahiroğlu, K., Svedström, T., & Wikström, V. (2015). NOISA: A novel intelligent system facilitating smart interaction. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 279–282). New York: ACM.

    Google Scholar 

  • Tan, S. L., & Nguyen, B. A. T. (2009). Survey and performance evaluation of real-time operating systems (RTOS) for small microcontrollers. IEEE Micro. doi: 10.1109/MM.2009.56.

  • Technical io. (2016). Tessel. Retrieved July 31, 2016, from https://tessel.io/

  • Texas Instruments. (2016). Simplelink SensorTag.

    Google Scholar 

  • The Workers. (2015). After dark (Artwork).

    Google Scholar 

  • Townsend, A. M. (2013). Smart cities: Big data, civic hackers, and the quest for a new utopia. New York: WW Norton & Company.

    Google Scholar 

  • Tullio, J., Dey, A. K., Chalecki, J., & Fogarty, J. (2007). How it works: A field study of non-technical users interacting with an intelligent system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 31–40). New York: ACM.

    Chapter  Google Scholar 

  • Vallgårda, A., & Redström, J. (2007). Computational composites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 513–522). New York: ACM.

    Chapter  Google Scholar 

  • Vallgårda, A., & Sokoler, T. A. Material strategy: Exploring material properties of computers. International Journal of Design. Retrieved August 2, 2016, from http://www.ijdesign.org/ojs/index.php/IJDesign/article/view/628/309

  • Van Every, S. (2015). p5.serialport (Computer Program). Retrieved July 21, 2016, from https://github.com/vanevery/p5.serialport

  • Vandevelde, C., Vanhoucke, M., & Saldien, J. (2015). Prototyping social interactions with DIY animatronic creatures. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 509–512). New York: ACM.

    Google Scholar 

  • Villar, N., Scott, J., Hodges, S., Hammil, K., & Miler, C. (2012). .NET gadgeteer: A platform for custom devices. In Pervasive computing (pp. 216–233). Berlin: Springer.

    Google Scholar 

  • Waldron, R. (2012). rwaldron/johnny-five (Computer Program). Retrieved July 21, 2016, from https://github.com/rwaldron/johnny-five

  • Weiser, M. (1999). The computer for the 21st century. SIGMOBILE Mobile Computing and Communications Review, 3, 3–11. doi:10.1145/329124.329126.

    Article  Google Scholar 

  • Willow Garage. (2011). Turtlebot. In Turtlebot. Retrieved January 4, 2017, from http://www.turtlebot.com

  • Wilson, S., Gurevich, M., Verplank, B., & Stang, P. (2003). Microcontrollers in music HCI instruction: Reflections on our switch to the Atmel AVR platform. In Proceedings of the 2003 Conference on New Interfaces for Musical Expression (pp. 24–29). Singapore: National University of Singapore.

    Google Scholar 

Download references

Acknowledgements

We would like to thank the following people for their help with this work: Michael Shiloh, Victor Chahuneau, Tom Igoe, Bjoern Hartmann, Rob Semmens, Dickson Chow, Phillip Dupree, Noam Zomerfeld, Carey Smith, Nathan Seidel, Bill Verplank, CCA & the Stanford Design Group.

Author information

Authors and Affiliations

  1. Center for Design Research, 424 Panama Mall, Stanford, CA, 94305-9035, USA

    Nikolas Martelaro, Wendy Ju & Mark Horowitz

Authors
  1. Nikolas Martelaro
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Wendy Ju
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Mark Horowitz
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nikolas Martelaro .

Editor information

Editors and Affiliations

  1. Hasso Plattner Institute for Software Systems Engineering , Potsdam, Germany

    Hasso Plattner

  2. Hasso Plattner Institute for Software Systems Engineering , Potsdam, Germany

    Christoph Meinel

  3. Stanford University , Stanford, California, USA

    Larry Leifer

Rights and permissions

Reprints and permissions

Copyright information

© 2018 Springer International Publishing AG

About this chapter

Cite this chapter

Martelaro, N., Ju, W., Horowitz, M. (2018). The Interaction Engine. In: Plattner, H., Meinel, C., Leifer, L. (eds) Design Thinking Research. Understanding Innovation. Springer, Cham. https://doi.org/10.1007/978-3-319-60967-6_8

Download citation

Publish with us

Policies and ethics

Search

Navigation