Abstract
The Interaction Engine is a framework for prototyping interactive, connected devices based on widely available single-board Linux computers. With microcontrollers, networking, and modular open-source software, these modules enable interaction modalities such as audio, video, tangible, and digital interfaces to be embedded into forms that go beyond traditional computing. In this paper, we outline the hardware and software components that make up the general Interaction Engine framework and discuss its benefits for interaction designers. We provide an illustrative case study of the Interaction Engine in use. We ran workshops to introduce designers to the Interaction Engine framework and we describe the projects where they subsequently employed Interaction Engines to understand issues and opportunities presented by this model. In describing the framework and case studies, we intend to shift designer’s thinking of computer as product to computer as material to create new interactive devices.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
References
Acholonu, U. C. (2012). Techonology [sic] bricolage exploring the tension between convention and innovation when problem-solving with technology. Stanford University.
Adafruit. (2016). Adafruit-occidentalis. Retrieved August 1, 2016, from https://github.com/adafruit/Adafruit-Occidentalis
Amazon. (2016a). Amazon dash button. Retrieved July 31, 2016, from https://www.amazon.com/Dash-Buttons/b?ie=UTF8&node=10667898011
Amazon. (2016b). Amazon echo. Retrieved July 30, 2016, from https://www.amazon.com/Amazon-Echo-Bluetooth-Speaker-with-WiFi-Alexa/dp/B00X4WHP5E
Aminzade, D., Hartmann, B., Doorley, S., et al. (2008). Interactive digital murals at San Jose Museum of Art.
ARM. (2016). Cortex-M Series. Retrieved July 31, 2016, from http://www.arm.com/products/processors/cortex-m/index.php
Ballagas, R., Ringel, M., Stone, M., & Borchers, J. (2003). iStuff: A physical user interface toolkit for ubiquitous computing environments. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 537–544). New York: ACM.
Banzi, M., & Shiloh, M. (2014). Make: Getting started with Arduino: The open source electronics prototyping platform. Sebastopol: Maker Media.
Barragán, H. (2004). Wiring: Prototyping physical interaction design.
Beigl, M., Gellersen, H.-W., & Schmidt, A. (2001). Mediacups: Experience with design and use of computer-augmented everyday artefacts. Computer Networks, 35, 401–409. doi:10.1016/S1389-1286(00)00180-8.
Berdahl, E., & Ju, W. (2011). Satellite CCRMA: A musical interaction and sound synthesis platform. In Proceedings of the International Conference on New Interfaces for Musical Expression (pp. 173–178).
Borthwick, M. (2016). Building museum exhibits with open hardware: Raspberry Pi & Arduino.
Bostock, M. D3.js—Data-driven documents. Retrieved July 26, 2016, from https://d3js.org/
Buechley, L., Eisenberg, M., Catchen, J., & Crockett, A. (2008) The LilyPad Arduino: Using computational textiles to investigate engagement, aesthetics, and diversity in computer science education. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 423–432). New York: ACM.
Buechley, L., Elumeze, N., & Eisenberg, M. (2006). Electronic/computational textiles and children’s crafts. In Proceedings of the 2006 Conference on Interaction Design and Children (pp. 49–56). New York: ACM.
Buechley, L., & Hill, B. M. (2010). LilyPad in the wild: How hardware’s long tail is supporting new engineering and design communities. In Proceedings of the 8th ACM Conference on Designing Interactive Systems (pp. 199–207). New York: ACM.
Bush, V. (1989). As we may think. In Perspectives on the computer revolution (p. 49).
Chan, L., Hsieh, C.-H., Chen, Y.-L., Yang, S., Huang, D.-Y., Liang, R.-H., et al. (2015) Cyclops: Wearable and single-piece full-body gesture input devices. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 3001–3009). New York: ACM.
Charlesworth, T., Ford, H., Milton, L., Mortensson, T., Pedlingham, J., Knibbe, J., et al. (2015). TellTale: Adding a polygraph to everyday life. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 1693–1698). New York: ACM.
Chung, E. S., Hong, J. I., Lin, J., Prabaker, M. K., & Landay, J. A. (2004). Development and evaluation of emerging design patterns for ubiquitous computing. In Proceedings of the 5th conference on designing interactive systems: processes, practices, methods, and techniques (pp. 233–242). New York: ACM.
Derry, S. (1990). Flexible cognitive tools for problem solving instruction. In Annual meeting of the American Educational Research Association, Boston, MA.
DiCola, T. (2015). Setting up WiFi with BeagleBone Black|Adafruit Learning System. Retrieved July 30, 2016, from https://learn.adafruit.com/setting-up-wifi-with-beaglebone-black/overview
Dow, S. P., Glassco, A., Kass, J., Schwarz, M., Schwarz, D. L., & Klemmer, S. R. (2010). Parallel prototyping leads to better design results, more divergence, and increased self-efficacy. ACM Transactions on Computer-Human Interaction TOCHI, 17, 18. doi:10.1145/1879831.1879836.
Electric Imp, Inc. (2016). Electric Imp—Electric imp hardware. Retrieved July 31, 2016, from https://www.electricimp.com/platform/hardware/
Fitbit. (2016). Fitbit official site. Retrieved July 31, 2016, from https://www.fitbit.com/
Frei, P., Su, V., Mikhak, B., & Ishii, H. (2000). Curlybot: Designing a new class of computational toys. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 129–136). New York: ACM.
Gaver, W., Michael, M., Kerridge, T., Wilkie, A., Boucher, A., Ovalle, L., et al. (2015). Energy babble: Mixing environmentally-oriented internet content to engage community groups. In Proceedings of the 33rd Annual ACM Conference on Human Factors in Computing Systems (pp. 1115–1124). New York: ACM.
Gibb, A. (2009). Bug labs: Hacks and apps. Linux J, 2009, 4.
Gibb, A. M. (2010). New media art, design, and the Arduino microcontroller: A malleable tool. Pratt Institute.
Greenberg, S., & Fitchett, C. (2001). Phidgets: Easy development of physical interfaces through physical widgets. In Proceedings of the 14th Annual ACM Symposium on User Interface Software and Technology (pp. 209–218). New York: ACM.
Hardkernal. (2016a). ODROID U3. Retrieved July 30, 2016, from http://www.hardkernel.com/main/products/prdt_info.php?g_code=g138745696275
Hardkernal. (2016b). ODROID C1. Retrieved July 30, 2016, from http://www.hardkernel.com/main/products/prdt_info.php?g_code=G141578608433
Harris-Cronin, S. (2008). High resolution video playback.
Hartmann, B., Klemmer, S. R., Bernstein, M., Abdulla, L., Burr, B., Robinson-Mosher, A., et al. (2006). Reflective physical prototyping through integrated design, test, and analysis. In Proceedings of the 19th Annual ACM Symposium on User Interface Software and Technology (pp. 299–308). New York: ACM.
Hartmann, B., & Wright, P. K. (2013). Designing bespoke interactive devices. Computer, 46, 85–89. doi:10.1109/MC.2013.274.
Hennigh-Palermo, S. G. (2015). p5bots. Retrieved July 21, 2016, from https://github.com/sarahgp/p5bots
Huang, A. B. (2008). Chumby: An experiment in hackable pervasive computing. IEEE Pervasive Computing, 7, 55–62.
IBM. (2016) watson-developer-cloud/text-to-speech-nodejs. Retrieved July 26, 2016, fromhttps://github.com/watson-developer-cloud/text-to-speech-nodejs
Igoe, T. (2014). Lab: Serial communication with Node.js—ITP physical computing.
Itseez. (2016). OpenCV|OpenCV. Retrieved July 26, 2016, from http://opencv.org/
Jibo, Inc. (2016). Jibo—The worlds first social robot. Retrieved July 31, 2016, from https://www.jibo.com/
Jonassen, D. H. (1994). Technology as cognitive tools: Learners as designers. ITForum Paper, 1, 67–80.
Kaufmann, B., & Buechley, L. (2010). Amarino: A toolkit for the rapid prototyping of mobile ubiquitous computing. In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (pp. 291–298). New York: ACM.
Kay, A. (1998). Alan Kay on messaging.
Kubitza, T., Schmidt, A., Pohl, N., Petrelli, D., Dingler, T., & Dulake, N. (2013). Tools and methods for creating interactive artifacts. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (pp. 385–388). New York: ACM.
Kuniavsky, M. (2010). Smart things: Ubiquitous computing user experience design. Amsterdam: Elsevier.
Landin, H. (2005). Fragile and magical: Materiality of computational technology as design material. In Proceedings of the 4th Decennial Conference on Critical Computing: Between Sense and Sensibility (pp. 117–120). New York: ACM.
Langer, M., & Alderman, J. (2016). Open hardware belongs in your museum. In Museums and the Web 2016. Los Angeles, CA.
Lee, J. C., Avrahami, D., Hudson, S. E., Forlizzi, J., Dietz, P. H., & Leigh, D. (2004). The Calder Toolkit: wired and wireless components for rapidly prototyping interactive devices. In Proceedings of the 5th Conference on Designing Interactive Systems: Processes, Practices, Methods, and Techniques (pp. 167–175). New York: ACM.
Levis, P., Madden, S., Polastre, J., Szewczyk, R., Whitehouse, K., Woo, A., et al. (2005). TinyOS: An operating system for sensor networks. In: W. Weber, J. M. Rabaey, E. Aarts (Eds.), Ambient intelligence (pp. 115–148). Berlin: Springer.
Lin, F. X., Rahmati, A., & Zhong, L. (2010). Dandelion: A framework for transparently programming phone-centered wireless body sensor applications for health. In Wireless health 2010 (pp. 74–83). New York: ACM.
Marshall, M. T., Dulake, N., Petrelli, D., & Kockelkorn, H. (2015). From the deposit to the exhibit floor: An exploration on giving museum objects personality and social life. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 1917–1922). New York: ACM.
Martelaro, N., Shiloh, M., & Ju, W. (2016). The interaction engine: Tools for prototyping connected devices. In Proceedings of the TEI ‘16: Tenth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 762–765). New York: ACM.
Marvell. (2016). Kinoma Create. Retrieved July 31, 2016, from http://kinoma.com/create/
mBed. (2016). Nordic nRF51822. Retrieved July 31, 2016, from https://developer.mbed.org/platforms/Nordic-nRF51822/
McCarthy, L. (2016). p5.js (Computer Program).
McComb, G. (2003). Robot builder’s bonanza (3rd ed.). New York: McGraw-Hill.
McGrath, W., Etemadi, M., Roy, S., & Hartmann, B. (2015). Fabryq: Using phones as gateways to prototype internet of things applications using web scripting. In Proceedings of the 7th ACM SIGCHI Symposium on Engineering Interactive Computing Systems (pp. 164–173). New York: ACM.
Mellis, D., Banzi, M., Cuartielles, D., & Igoe, T. (2007). Arduino: An open electronic prototyping platform. In Proceedings of CHI.
Mellis, D. A., & Buechley, L. (2014). Do-it-yourself cellphones: An investigation into the possibilities and limits of high-tech diy. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 1723–1732). New York: ACM.
Mellis, D. A, Jacoby, S., Buechley, L., Perner-Wilson, H., & Qi, J. (2013). Microcontrollers as material: Crafting circuits with paper, conductive ink, electronic components, and an “untoolkit.” In Proceedings of the 7th International Conference on Tangible, Embedded and Embodied Interaction (pp. 83–90). New York: ACM.
Merrill, D., Kalanithi, J., & Maes, P. (2007). Siftables: Towards sensor network user interfaces. In: Proceedings of the 1st International Conference on Tangible and Embedded Interaction (pp. 75–78). New York: ACM.
Millner, A., & Baafi, E. (2011). Modkit: Blending and extending approachable platforms for creating computer programs and interactive objects. In Proceedings of the 10th International Conference on Interaction Design and Children (pp. 250–253). New York: ACM.
Norman, D. A. (2013). The design of everyday things: Revised and expanded edition. New York: Basic books.
O’Sullivan, D., & Igoe, T. (2004). Physical computing: Sensing and controlling the physical world with computers. Boston, MA: Course Technology Press.
Output Arts. (2012). I have a message for you…
Overholt, D., Møbius, N. “DZL”. (2013). Embedded audio without beeps: Synthesis and sound effects from cheap to steep. In Proceedings of the 8th International Conference on Tangible, Embedded and Embodied Interaction (pp. 361–364). New York: ACM.
Palazzetti, E. (2015). Getting Started with UDOO. Birmingham: Packt.
Punch Through Design. (2016). LightBlue Bean. Retrieved July 31, 2016, from https://punchthrough.com/bean
Qi, J., & Buechley, L. (2010). Electronic popables: Exploring paper-based computing through an interactive pop-up book. In Proceedings of the Fourth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 121–128). New York: ACM.
Raghavan, P., Lad, A., & Neelakandan, S. (2005). Embedded linux system design and development. Boca Raton: CRC Press.
Redmer, A., & Stoeveken, T. (2016). MJPEG-STREAMER.
Resnick, M., Martin, F., Berg, R., Borovoy, R., Colella, V., Kramer, K., et al. (1998). Digital manipulatives: New toys to think with. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 281–287). New York: ACM Press/Addison-Wesley.
Riliskis, L., & Levis, P. (2014). Ravel a framework for embedded-gateway-cloud applications. In Proceedings of the 12th ACM Conference on Embedded Network Sensor Systems (pp. 344–345). New York: ACM.
Roman, M., Al-Muhtadi, J., Ziebart, B., Campbell, R., & Mickunas, M. D. (2003). System support for rapid ubiquitous computing application development and evaluation. In System Support for Ubiquitous Computing Workshop (UbiSys’ 03) in conjunction with UbiComp.
Rozin, D. (1999). Wooden mirror.
Russell, S. C. (2012). Pi and Arduino in action. The MagPi, 4–6.
Sculley, J. (1987). Odyssey: Pepsi to apple, a journey of adventure, ideas, and the future. New York: Harper & Row.
Sheppard, S. D. (1992). Mechanical dissection: An experience in how things work. In Proceedings of the Engineering Education: Curriculum Innovation & Integration, January 6–10.
Sipitakiat, A., & Blikstein, P. (2013). Interaction design and physical computing in the era of miniature embedded computers. In Proceedings of the 12th International Conference on Interaction Design and Children (pp. 515–518). New York: ACM.
Spark Labs, Inc. (2016) Particle. In Particle. Retrieved July 31, 2016, from https://www.particle.io/prototype
Stankovic, J. A., Lee, I., Mok, A., & Rajkumar, R. (2005). Opportunities and obligations for physical computing systems. Computer, 38, 23–31. doi:10.1109/MC.2005.386.
Stiller, E. (2009). Teaching programming using bricolage. Journal of Computing Sciences in Colleges, 24, 35–42.
Tahiroğlu, K., Svedström, T., & Wikström, V. (2015). NOISA: A novel intelligent system facilitating smart interaction. In Proceedings of the 33rd Annual ACM Conference Extended Abstracts on Human Factors in Computing Systems (pp. 279–282). New York: ACM.
Tan, S. L., & Nguyen, B. A. T. (2009). Survey and performance evaluation of real-time operating systems (RTOS) for small microcontrollers. IEEE Micro. doi: 10.1109/MM.2009.56.
Technical io. (2016). Tessel. Retrieved July 31, 2016, from https://tessel.io/
Texas Instruments. (2016). Simplelink SensorTag.
The Workers. (2015). After dark (Artwork).
Townsend, A. M. (2013). Smart cities: Big data, civic hackers, and the quest for a new utopia. New York: WW Norton & Company.
Tullio, J., Dey, A. K., Chalecki, J., & Fogarty, J. (2007). How it works: A field study of non-technical users interacting with an intelligent system. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 31–40). New York: ACM.
Vallgårda, A., & Redström, J. (2007). Computational composites. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 513–522). New York: ACM.
Vallgårda, A., & Sokoler, T. A. Material strategy: Exploring material properties of computers. International Journal of Design. Retrieved August 2, 2016, from http://www.ijdesign.org/ojs/index.php/IJDesign/article/view/628/309
Van Every, S. (2015). p5.serialport (Computer Program). Retrieved July 21, 2016, from https://github.com/vanevery/p5.serialport
Vandevelde, C., Vanhoucke, M., & Saldien, J. (2015). Prototyping social interactions with DIY animatronic creatures. In Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (pp. 509–512). New York: ACM.
Villar, N., Scott, J., Hodges, S., Hammil, K., & Miler, C. (2012). .NET gadgeteer: A platform for custom devices. In Pervasive computing (pp. 216–233). Berlin: Springer.
Waldron, R. (2012). rwaldron/johnny-five (Computer Program). Retrieved July 21, 2016, from https://github.com/rwaldron/johnny-five
Weiser, M. (1999). The computer for the 21st century. SIGMOBILE Mobile Computing and Communications Review, 3, 3–11. doi:10.1145/329124.329126.
Willow Garage. (2011). Turtlebot. In Turtlebot. Retrieved January 4, 2017, from http://www.turtlebot.com
Wilson, S., Gurevich, M., Verplank, B., & Stang, P. (2003). Microcontrollers in music HCI instruction: Reflections on our switch to the Atmel AVR platform. In Proceedings of the 2003 Conference on New Interfaces for Musical Expression (pp. 24–29). Singapore: National University of Singapore.
Acknowledgements
We would like to thank the following people for their help with this work: Michael Shiloh, Victor Chahuneau, Tom Igoe, Bjoern Hartmann, Rob Semmens, Dickson Chow, Phillip Dupree, Noam Zomerfeld, Carey Smith, Nathan Seidel, Bill Verplank, CCA & the Stanford Design Group.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2018 Springer International Publishing AG
About this chapter
Cite this chapter
Martelaro, N., Ju, W., Horowitz, M. (2018). The Interaction Engine. In: Plattner, H., Meinel, C., Leifer, L. (eds) Design Thinking Research. Understanding Innovation. Springer, Cham. https://doi.org/10.1007/978-3-319-60967-6_8
Download citation
DOI: https://doi.org/10.1007/978-3-319-60967-6_8
Published:
Publisher Name: Springer, Cham
Print ISBN: 978-3-319-60966-9
Online ISBN: 978-3-319-60967-6
eBook Packages: Business and ManagementBusiness and Management (R0)