Advertisement

MilkyWay: A Toolbox for Prototyping Collaborative Mobile-Based Interaction Techniques

  • Mandy KorzetzEmail author
  • Romina Kühn
  • Karl Kegel
  • Leon Georgi
  • Franz-Wilhelm Schumann
  • Thomas Schlegel
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 11573)

Abstract

Beside traditional multitouch input, mobile devices provide various possibilities to interact in a physical, device-based manner due to their built-in hardware. Applying such interaction techniques allows for sharing content easily, e.g. by literally pouring content from one device into another, or accessing device functions quickly, e.g. by facing down the device to mute incoming calls. So-called mobile-based interaction techniques are characterized by movements and concrete positions in real spaces. Even though such interactions may provide many advantages in everyday life, they have limited visibility in interaction design due to the complexity of sensor processing. Hence, mobile-based interactions are often integrated, if any, at late design stages. To support testing interactive ideas in early design stages, we propose MilkyWay, a toolbox for prototyping collocated collaborative mobile-based interaction techniques. MilkyWay includes an API and a mobile application. It enables easily building up mobile interactive spaces between multiple collocated devices as well as prototyping interactions based on device sensors by a programming-by-demonstration approach. Appropriate sensors are selected and combined automatically to increase tool support. We demonstrate our approach using a proof of concept implementation of a collaborative Business Model Canvas (BMC) application.

Keywords

Prototyping Mobile devices Device-based interaction Collocated interaction Collaboration Interaction design 

Notes

Acknowledgments

The European Social Fund (ESF) and the German Federal State of Saxony have funded this work within the project CyPhyMan (100268299). We would also like to thank Lukas Büschel for his work and support.

References

  1. 1.
    Ajaj, R., Jacquemin, C., Vernier, F.: RVDT: a design space for multiple input devices, multiple views and multiple display surfaces combination. In: Proceedings of the International Conference on Multimodal Interfaces (ICMI-MLMI 2009), pp. 269–276 (2009).  https://doi.org/10.1145/1647314.1647372
  2. 2.
    Ashbrook, D., Starner, T.: MAGIC: a motion gesture design tool. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI 2010), pp. 2159–2168. ACM (2010).  https://doi.org/10.1145/1753326.1753653
  3. 3.
    Ballagas, R., Memon, F., Reiners, R., Borchers, J.: iStuff mobile: rapidly prototyping new mobile phone interfaces for ubiquitous computing. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2007), pp. 1107–1116 (2007).  https://doi.org/10.1145/1240624.1240793
  4. 4.
    Ballagas, R., Ringel, M., Stone, M., Borchers, J.: iStuff: a physical user interface toolkit for ubiquitous computing environments. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2003), pp. 537–544 (2003).  https://doi.org/10.1039/F29767200988CrossRefGoogle Scholar
  5. 5.
    Baytaş, M.A., Yemez, Y., Özcan, O.: User interface paradigms for visually authoring mid-air gestures: a survey and a provocation. In: 1st International Workshop on Engineering Gestures for Multimodal Interfaces (EGMI 2014), pp. 8–14 (2014). http://ceur-ws.org/Vol-1190/
  6. 6.
    Geiger, C., Fritze, R., Lehmann, A., Stöcklein, J.: HYUI: a visual framework for prototyping hybrid user interfaces. In: Proceedings of the 2nd International Conference on Tangible and Embedded Interaction (TEI 2008), pp. 63–70 (2008).  https://doi.org/10.1145/1347390.1347406
  7. 7.
    Gutwin, C., Greenberg, S.: Design for individuals, design for groups: tradeoffs between power and workspace awareness. In: Proceedings of the 1998 ACM Conference on Computer Supported Cooperative Work (CSCW 1998), pp. 207–216 (1998).  https://doi.org/10.1145/289444.289495
  8. 8.
    Hartmann, B., Abdulla, L., Mittal, M., Klemmer, S.R.: Authoring sensor-based interactions by demonstration with direct manipulation and pattern recognition. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2007), pp. 145–154 (2007).  https://doi.org/10.1145/1240624.1240646
  9. 9.
    Jokela, T., Chong, M.K., Lucero, A., Gellersen, H.W.: Connecting devices for collaborative interactions. Interactions 22(4), 1–5 (2015).  https://doi.org/10.1145/2776887CrossRefGoogle Scholar
  10. 10.
    Keller, C., Kühn, R., Engelbrecht, A., Korzetz, M., Schlegel, T.: A prototyping and evaluation framework for interactive ubiquitous systems. In: Proceedings of the First International Conference on Distributed, Ambient, and Pervasive Interactions, pp. 215–224 (2013).  https://doi.org/10.1007/978-3-642-39351-8_24CrossRefGoogle Scholar
  11. 11.
    Kim, J.W., Kim, H.J., Nam, T.J.: M.gesture: an acceleration-based gesture authoring system on multiple handheld and wearable devices. In: Proceedings of the 2016 CHI Conference on Human Factors in Computing Systems (CHI 2016), pp. 2307–2318. ACM (2016).  https://doi.org/10.1145/2858036.2858358
  12. 12.
    Kim, J.W., Nam, T.J.: EventHurdle: supporting designers’ exploratory interaction prototyping with gesture-based sensors. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2013), pp. 267–276 (2013).  https://doi.org/10.1145/2470654.2470691
  13. 13.
    Klemmer, S.R., Li, J., Lin, J., Landay, J.A.: Papier-Mâché: toolkit support for tangible input. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI 2004), Vienna, pp. 399–406 (2004).  https://doi.org/10.1016/j.jjcc.2014.08.007CrossRefGoogle Scholar
  14. 14.
    Klompmaker, F., Schrage, K., Reimann, C.: INDiE: a framework for human computer interaction in distributed environments. In: Proceedings of the 6th International Conference on Mobile Technology, Application & Systems (Mobility 2009), pp. 26–29 (2009).  https://doi.org/10.1145/1710035.1710061
  15. 15.
    Korzetz, M., Kühn, R., Heisig, P., Schlegel, T.: Natural collocated interactions for merging results with mobile devices. In: Proceedings of the 18th International Conference on Human-Computer Interaction with Mobile Devices and Services Adjunct (MobileHCI 2016), pp. 746–752. ACM (2016).  https://doi.org/10.1145/2957265.2961839
  16. 16.
    Korzetz, M., Kühn, R., Schlegel, T.: Turn it, pour it, twist it: a model for designing mobile device-based interactions. In: Proceedings of the 5th International Conference on Human-Computer Interaction and User Experience in Indonesia (CHIuXiD 2019). ACM (2019).  https://doi.org/10.1145/3328243.3328246
  17. 17.
    Kühn, R., Korzetz, M., Büschel, L., Korger, C., Manja, P., Schlegel, T.: Natural voting interactions for collaborative work with mobile devices. In: Proceedings of the 2016 CHI Conference Extended Abstracts on Human Factors in Computing Systems (CHI EA 2016), pp. 2570–2575. ACM (2016).  https://doi.org/10.1145/2851581.2892300
  18. 18.
    Kühn, R., Korzetz, M., Büschel, L., Schumann, F.W., Schlegel, T.: Device-based interactions for anonymous voting and rating with mobile devices in collaborative scenarios. In: Proceedings of the 15th International Conference on Mobile and Ubiquitous Multimedia (MUM 2016), pp. 315–317. ACM (2016).  https://doi.org/10.1145/3012709.3016067
  19. 19.
    Kühn, R., Schlegel, T.: Mixed-focus collaboration activities for designing mobile interactions. In: Proceedings of the 20th International Conference on Human-Computer Interaction with Mobile Devices and Services (MobileHCI 2018), pp. 71–77. ACM (2018).  https://doi.org/10.1145/3236112.3236122
  20. 20.
    Ledo, D., Houben, S., Vermeulen, J., Marquardt, N., Oehlberg, L., Greenberg, S.: Evaluation strategies for HCI toolkit research. In: Proceedings of the 2018 Conference on Human Factors in Computing Systems (CHI 2018), p. 17 (2018).  https://doi.org/10.1145/3173574.3173610
  21. 21.
    Leigh, S.W., Schoessler, P., Heibeck, F., Maes, P., Ishii, H.: THAW: Tangible interaction with see-through augmentation for Smartphones on computer screens. In: Proceedings of the Ninth International Conference on Tangible, Embedded, and Embodied Interaction (TEI 2014), pp. 89–96. ACM (2015).  https://doi.org/10.1145/2677199.2680584
  22. 22.
    Lucero, A., Holopainen, J., Jokela, T.: Pass-them-around: collaborative use of mobile phones for photo sharing. In: Proceedings of the 2011 Annual Conference on Human Factors in Computing Systems (CHI 2011), pp. 1787–1796 (2011).  https://doi.org/10.1145/1978942.1979201
  23. 23.
    Lucero, A., Keränen, J., Jokela, T.: Social and spatial interactions: shared co-located mobile phone use. In: CHI 2010 Extended Abstracts on Human Factors in Computing Systems (CHI EA 2010), pp. 3223–3228. ACM (2010)Google Scholar
  24. 24.
    Lucero, A., Keränen, J., Korhonen, H.: Collaborative use of mobile phones for brainstorming. In: Proceedings of the 12th International Conference on Human Computer Interaction with Mobile Devices and Services (MobileHCI 2010), pp. 337–340. ACM (2010).  https://doi.org/10.1145/1851600.1851659
  25. 25.
    Nielsen, M., Störring, M., Moeslund, T.B., Granum, E.: A procedure for developing intuitive and ergonomic gesture interfaces for HCI. In: Camurri, A., Volpe, G. (eds.) GW 2003. LNCS (LNAI), vol. 2915, pp. 409–420. Springer, Heidelberg (2004).  https://doi.org/10.1007/978-3-540-24598-8_38CrossRefGoogle Scholar
  26. 26.
    Rico, J., Brewster, S.: Usable gestures for mobile interfaces: evaluating social acceptability. In: Proceedings of the 28th International Conference on Human Factors in Computing Systems (CHI 2010), p. 887. ACM (2010).  https://doi.org/10.1145/1753326.1753458

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  • Mandy Korzetz
    • 1
    Email author
  • Romina Kühn
    • 1
  • Karl Kegel
    • 1
  • Leon Georgi
    • 1
  • Franz-Wilhelm Schumann
    • 1
  • Thomas Schlegel
    • 2
  1. 1.Software Technology GroupTU DresdenDresdenGermany
  2. 2.Institute of Ubiquitous Mobility SystemsKarlsruhe University of Applied ScienceKarlsruheGermany

Personalised recommendations