Skip to main content
SpringerLink
Log in

Real-time coding method and tool for artefact-centric interaction analysis in co-design situations assisted by augmented reality

  • Original Paper
  • Published:
International Journal on Interactive Design and Manufacturing (IJIDeM) Aims and scope Submit manuscript

Abstract

This paper presents and discusses a fast and efficient method to study artefact-centric interactions in co-design sessions. The method is particularly useful for exploring the introduction of augmented reality (AR) environments since AR application representations combine both digital content and physical content, transforming the way users interact with the design object. Although protocol analysis is extensively used in cognitive studies of design, it is a time-consuming and cumbersome method and hence unsuitable for extensive analysis in industrial environments. Our real-time coding method makes it possible to perform “on-the-fly” coding of physical interactions in co-design sessions. Focusing on quantifying interaction occurrences, our results are consistent with those obtained with post-session coding. Internal validity was assessed using relevant statistical tests. Based on the data collected in co-design sessions we show how aggregate results, especially timelines and interaction densities, can be displayed rapidly at the end of each session. This research paves the way for a more general implementation of real-time coding of collaborative work sessions in industrial situations.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2
Fig. 3

[38]

Fig. 4
Fig. 5
Fig. 6
Fig. 7
Fig. 8
Fig. 9
Fig. 10
Fig. 11
Fig. 12
Fig. 13
Fig. 14
Fig. 15

Similar content being viewed by others

Notes

  1. These companies are part of the SPARK consortium. Artefice specialises in brand design while Stimulo designs innovative products.

References

  1. Caridà, A., Melia, M., Colurcio, M.: Business model design and value co-creation: Looking for a new pattern. In: Russo-Spena, T., Mele, C., Nuutinen, M. (eds.) Innovating in Practice. Springer, Cham (2017)

    Google Scholar 

  2. Ardito, C., Buono, P., Costabile, F., Lanzilotti, R., Piccinno, A.: End users as co-designers of their own tools and products. J. Vis. Lang. Comput. 23(2), 78–90 (2012)

    Article  Google Scholar 

  3. Steiner, I. D.: Group Process and Productivity. New York, NY: Academic Press (1972) Cited by Kvan, T.: Collaborative design: what is it? Automation in Construction, vol. 9, pp. 409–415 (2000)

  4. Verlinden, J., Horváth, I., Nam, T.-J.: Recording augmented reality experiences to capture design reviews. Int. J Interact. Des. Manuf. (IJIDeM) 3(3), 189–200 (2009)

    Article  Google Scholar 

  5. Maurya, S., Arai, K., Moriya, K., Arrighi, P.A., Mougenot, C.: A mixed reality tool for end-users participation in early creative design tasks. Int. J. Interact. Des. Manuf. 13(1), 163–182 (2019)

    Article  Google Scholar 

  6. Schifferstein, H.N.J., Desmet, P.M.A.: Tools facilitating multi-sensory product design. Des. J. 11(2), 137–158 (2008)

    Google Scholar 

  7. Raskar, R., Welch, G., Fuchs, H.: Spatially augmented reality. In: Proceeding of the First IEEE Workshop on Augmented Reality; 1998 November 1; San Francisco, CA; pp. 63–72 (1998)

  8. Park, M.K., Lim, K.J., Seo, M.K., Jung, S.J., Lee, K.H.: Spatial augmented reality for product appearance design evaluation. J. Comput. Des. Eng. 2(1), 38–46 (2015)

    Google Scholar 

  9. Hannah, R., Joshi, S., Summers, J.D.: A user study of interpretability of engineering design representations. J. Eng. Des. 23, 443–468 (2012)

    Article  Google Scholar 

  10. Ericsson, K.A., Simon, H.A.: Protocol Analysis: Verbal Reports as Data. MIT Press, Cambridge (1993)

    Book  Google Scholar 

  11. Cross, N., Christiaans, H., Dorst, K. (eds.): Analysing Design Activity. Wiley, Chichester (1996)

    Google Scholar 

  12. Mondada, L.: Video Recording as the Reflexive Preservation and Configuration of Phenomenal Features for Analysis. In: Knoblauch, H., Raab, J., Soeffner, H.-G., Schnettler, B. (eds.) Video Analysis. Lang, Bern (2006)

    Google Scholar 

  13. Eastman, C.M.: Cognitive processes and ill-defined problems: a case study from design. In: Proceedings of the First Joint International Conference on Artificial Intelligence. Bedford, MA: MITRE (1969)

  14. Visser, W.: Use of episodic knowledge and information in design problem solving. Design Studies, Elsevier, 1995, Analysing Design Activity, 16(2), 171–187 (1995)

  15. Lloyd, P.: Can concurrent verbalization reveal design cognition? Des. Stud. 16, 237–259 (1995)

    Article  Google Scholar 

  16. Jaspers, M.W.M., Steen, T., Bos, C., Geenen, M.: The think aloud method: a guide to user interface design. Int. J. Med. Inform. 73(11–12), 781–795 (2004)

    Article  Google Scholar 

  17. Hay, L., Duffy, A. H. B., McTeague C., Pidgeon L. M., Vuletic T., Grealy, M.: A systematic review of protocol studies on conceptual design cognition: design as search and exploration. Des. Sci. 3 (2017)

  18. Gero, J.: Design prototypes: a knowledge representation schema for design. Artif. Intell. Mag. 11(4), 26 (1990)

    Google Scholar 

  19. Purcell, A.T., Gero, J.S., Edwards, H., Matka, E.: Design fixation and intelligent design assistants. In: Gero, J.S., Sudweeks, F. (eds.) Artificial Intelligence in Design ‘94, pp. 483–496. Kluwer, Dordrecht (1994)

    Chapter  Google Scholar 

  20. Amhed, S., Hansen, C.T.: A decision making model for engineering designers. In: Sahin, T.M.M. (ed) Computer based design, EDC2002, Cambridge, pp. 217–227 (2002)

  21. Girod, M., Elliot, A.C., Wright, I.C., Burns, N.D.: Decision making in conceptual engineering design: an empirical investigation. Proc. Inst. Mech. Eng. Part B J. Eng. Manuf. 217(9), 1215–1228 (2003)

    Article  Google Scholar 

  22. Tang, J.C., Leifer, L.J.: A framework for understanding the workspace activity of design teams. In: Proceedings of ACM Conference on Computer-supported cooperative work: ACM (1988)

  23. Radcliffe, D.F., Lee, T.Y.: Design methods used by undergraduate engineering students. Des. Stud. 10(4), 199–207 (1989)

    Article  Google Scholar 

  24. Olson, G.M., Olson, J.S.: Small group design meetings: an analysis of collaboration. Human–Comput. Interact. 7, 347–374 (1992)

    Article  Google Scholar 

  25. Cross, N., Cross, A.C.: Observations of teamwork and social processes in design. Des. Stud. 16, 145–170 (1995)

    Google Scholar 

  26. Wiltschnig, S., Christensen, B.T., Ball, L.J.: Collaborative problem-solution co-evolution in creative design. Des. Stud. 34(2013), 515–545 (2013)

    Article  Google Scholar 

  27. Waldron, M.B., Waldron, K.J.: Methods of studying mechanical design. In: Waldron, M.B., Waldron, K.J. (eds.) Mechanical Design: Theory and Methodology, pp. 21–34. Springer, New York (1996)

    Chapter  MATH  Google Scholar 

  28. Jiang, H. & Yen, C.-C.: Protocol analysis in design research: a review. In: Rigor and Relevance in Design: IASDR 2009, Seoul, Korea October 18–22, Seoul, pp. 147–156. International Association of Societies of Design Research (2009)

  29. Perry, M.: Coordinating joint design work: the role of communication and artefacts. Des. Stud. 19(3), 273–288 (1998)

    Article  MathSciNet  Google Scholar 

  30. Visser, W.: Function and Form of Gestures in a Collaborative Design Meeting. In: Kopp, S., Wachsmuth, I. (eds.) Gesture in Embodied Communication and Human–Computer Interaction, vol. 5934, pp. 61–72. Springer, Berlin (2010)

    Chapter  Google Scholar 

  31. Härkki, T., Seitamaa-Hakkarainen, P., Hakkarainen, K.: Hands on design: comparing the use of sketching and gesturing in collaborative designing. J. Des. Res. 16(1), 24–46 (2018)

    Google Scholar 

  32. Eris, O., Martelaro, N.: Badke-Schaube, P: a comparative analysis of multimodal communication during design sketching in co-located and distributed environments. Des. Stud. 35, 559–592 (2014)

    Article  Google Scholar 

  33. McNeill, D.: Hand and Mind: What Gestures Reveal About Thought. University of Chicago Press, Chicago (1992)

    Google Scholar 

  34. Kang, S., Tversky, B., Black, J.B.: Coordinating gesture, word, and diagram: explanations for experts and novices. Spat. Cognit. Comput. 15, 1–26 (2015)

    Article  Google Scholar 

  35. Jeantet, A.: Les objets intermédiaires dans les processus de conception des produits. Sociologie du travail 3, 291–316 (1998)

    Article  Google Scholar 

  36. Vinck, D.: Taking intermediary objects and equipping work into account in the study of engineering practices. Eng. Stud. 3(1), 25–44 (2011)

    Article  Google Scholar 

  37. Boujut, J.F., Blanco, E.: Intermediary objects as a means to foster co-operation in engineering design. Comput. Support. Coop. Work 12, 205–219 (2003)

    Article  Google Scholar 

  38. Becattini, N., Masclet, C., Ben-Guefrache, F., Prudhomme, G., Cascini, G., Dekoninck, E.: Characterisation of a co-creative design session through the analysis of multi-modal interactions. In: Proceeding of the International Conference on Engineering Design ICED2017, pp. 479–488 (2017)

  39. Ahmed, S., Wallace, K.M., Blessing, L.T.: Understanding the differences between how novice and experienced designers approach design tasks. Res. Eng. Des. 14(1), 1–11 (2003)

    Article  Google Scholar 

  40. Dorst, K.: Analysing design activity: new directions in protocol analysis. Des. Stud. 16(2), 139–142 (1995)

    Article  Google Scholar 

  41. Shapiro, S.S., Wilk, M.B.: An analysis of variance test for normality (complete samples). Biometrika 52, 591–611 (1965)

    Article  MathSciNet  MATH  Google Scholar 

Download references

Acknowledgements

The work reported in this paper was completed as part of the SPARK project, which received funding from the European Union’s Horizon 2020 research and innovation programme under grant agreement No. 688417. This paper only reflects the authors’ views. The European Commission is not responsible for any use that may be made of the information it contains.

Author information

Authors and Affiliations

  1. Univ. Grenoble Alpes, CNRS, Grenoble INP, G-SCOP, 38000, Grenoble, France

    C. Masclet, M. Poulin & J. F. Boujut

  2. Department of Mechanical Engineering, Politecnico di Milano, Milan, Italy

    N. Becattini

Authors
  1. C. Masclet
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. M. Poulin
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. J. F. Boujut
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. N. Becattini
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to J. F. Boujut.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Masclet, C., Poulin, M., Boujut, J.F. et al. Real-time coding method and tool for artefact-centric interaction analysis in co-design situations assisted by augmented reality. Int J Interact Des Manuf 14, 1141–1157 (2020). https://doi.org/10.1007/s12008-020-00683-8

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s12008-020-00683-8

Keywords

Search

Navigation