Semiotic Engineering: A Cohering Theory to Connect EUD with HCI, CMC and More

Chapter

Abstract

Theories have an important role to play in research areas whose application faces rapid technological changes. They can provide longer-term intellectual references that shape deeper investigations and contribute to consolidate the identity of such research areas. A recent survey of EUD-related work published between 2004 and 2013 suggests that our field is remarkably techno-centered and could increase its scientific impact by diversifying some of its research approaches and practices. In this paper we show concrete examples of how Semiotic Engineering, originally a semiotic theory of human-computer interaction, can provide a unified theoretical framing for various EUD-related topics of investigation. Our contribution to the collection of chapters in this book is to demonstrate this particular theory’s potential as a catalyst of new kinds of transdisciplinary debate, as well as a source of inspiration for new breeds of technological developments.

Keywords

Semiotic Engineering computer-mediated social communication programming as self-expression EUD theory 

Notes

Acknowledgements

I thank CNPq, FAPERJ and The AMD Foundation for financial support to the research reported in this paper. I am also grateful to Alex Repenning, AgentSheets Inc. and all the participants, researchers and partner schools involved in SGD-Br. I am additionally thankful to Ernest Edmonds and Alan Blackwell for comments, suggestions and interesting discussions about some of the ideas exposed in this paper. Finally, I am greatly obliged to the anonymous reviewers whose insightful comments have helped me improve this chapter significantly.

References

  1. Andersen, P. B. (1997). A theory of computer semiotics: semiotic approaches to construction and assessment of computer systems. Cambridge: Cambridge University Press.Google Scholar
  2. Andersen, P. B. (1985). Semiotics and informatics: computers as media Proceedings of the Conference on Information Technology for Information Use. Copenhagen: The Royal School of Librarianship.Google Scholar
  3. Andersen, P. B., Holmqvist, B., Jensen, J. F. (1993). The computer as medium. Cambridge: Cambridge University Press.Google Scholar
  4. Arias, E. G., Eden, H., Fischer, G. (2015). The envisionment and discovery collaboratory (EDC): explorations in human-centered informatics with tabletop computing environments. San Rafael, California: Morgan & Claypool Publishers.Google Scholar
  5. Barbosa, S. D. J., & de Souza, C. S. (2001). Extending software through metaphors and metonymies. Knowledge-Based Systems, 14(1), 15–27.CrossRefGoogle Scholar
  6. Blackwell, A. (2006). Psychological issues in end-user programming. In H. Liberman, F. Parternò, V. Wulf (Eds.). End user development (pp. 9–30). Dordrecht, Netherlands: Springer.CrossRefGoogle Scholar
  7. Blackwell, A., & Green, T. (2003). Notational systems–the cognitive dimensions of notations framework. In J. Carroll (Ed.), HCI models, theories, and frameworks: toward an interdisciplinary science (pp. 103–133). San Francisco, Calif: Morgan Kaufmann.Google Scholar
  8. Blandford, A., Furniss, D., Makri, S. (2016). “Qualitative HCI research: going behind the scenes.” Synthesis Lectures on Human-Centered Informatics, 9(1), 1–115.CrossRefGoogle Scholar
  9. Boden, M. A., & Edmonds, E. A. (2009). What is generative art? Digital Creativity, 20, 21–46.CrossRefGoogle Scholar
  10. Burke, Q., & Kafai, Y. B. (2010). Programming & storytelling: opportunities for learning about coding & composition. In Proceedings of the 9th International Conference on Interaction Design and Children (IDC ’10) (pp. 348–351). New York, NY, USA: ACM. doi: 10.1145/1810543.1810611.Google Scholar
  11. Burke, Q., & Kafai, Y. B. (2012). The writers’ workshop for youth programmers: digital storytelling with scratch in middle school classrooms. In Proceedings of the 43rd ACM technical symposium on Computer Science Education (SIGCSE ’12) (pp. 433–438). New York, NY, USA: ACM. doi: 10.1145/2157136.2157264.Google Scholar
  12. Bush, V. (1990). Science: the endless frontier: a report to the president on a program for postwar scientific research. Washington, DC: National Science Foundation. 192 p.Google Scholar
  13. Calvo, R. A., & Peters, D. (2013). The irony and re-interpretation of our quantified self. In H. Shen, R. Smith, J. Paay, P. Calder & T. Wyeld (Eds.), Proceedings of the 25th Australian computer-human interaction conference: augmentation, application, innovation, collaboration (OzCHI ’13) (pp. 367–370). New York, NY, USA: ACM. doi: 10.1145/2541016.2541070.Google Scholar
  14. de Souza, C., Barbosa, S., da Silva, S. (2001). Semiotic engineering principles for evaluating end-user programming environments. Interacting with Computers, 13, 467–495.CrossRefGoogle Scholar
  15. de Souza, C. S. (1993). The semiotic engineering of user interface languages. International Journal of Man-Machine Studies, Academic Press, 39, 753–773.CrossRefGoogle Scholar
  16. de Souza, C. S. (2005a). The semiotic engineering of human-computer interaction. Cambridge, MA: The MIT Press.Google Scholar
  17. de Souza, C. S. (2005b). Semiotic engineering: bringing designers and users together at interaction time. Interacting with Computers, 17(3), 317–341.CrossRefGoogle Scholar
  18. de Souza, C. S. (2013). Semiotic perspectives on interactive languages for life on the screen. Journal of Visual Languages & Computing, 24(3), 218–221.CrossRefGoogle Scholar
  19. de Souza, C. S., & Barbosa, S. D. J. (2006). A semiotic framing for end-user development. In H. Lieberman, F. Paternò, V. Wulf (Eds.). End user development (pp. 401–426). Netherlands: Springer.CrossRefGoogle Scholar
  20. de Souza, C. S., Cerqueira, R. F. G., Afonso, L. M., Brandão, R. M. R., Ferreira, J. S. J. (2016). Software developers as users: semiotic investigations in human-centered software development. Cham: Springer International Publishing.CrossRefGoogle Scholar
  21. de Souza, C. S., Garcia, A. C. B., Slaviero, C., Pinto, H., Repenning, A. (2011). End-user development: third international symposium, IS-EUD 2011, Torre Canne (BR), Italy, June 7–10, 2011. Proceedings (pp. 155–170). Berlin, Heidelberg: Springer Berlin Heidelberg.CrossRefGoogle Scholar
  22. de Souza, C. S., & Leitão, C. F. (2009). Semiotic engineering methods for scientific research in HCI. Synthesis Lectures on Human-Centered Informatics, 2(1), 1–122.CrossRefGoogle Scholar
  23. de Souza, C. S., Salgado, L. C., Leitão, C. F., Serra, M. M. (2014). Cultural appropriation of computational thinking acquisition research: seeding fields of diversity. In Proceedings of the 2014 conference on Innovation & technology in computer science education (ITiCSE ’14) (pp. 117–122). New York, NY, USA: ACM. doi: 10.1145/2591708.2591729.Google Scholar
  24. diSessa, A. A., & Cobb, P. (2004). Ontological innovation and the role of theory in design experiments. The Journal of the Learning Sciences, 13(1), 77–103.CrossRefGoogle Scholar
  25. Edmonds, E. A. (2007) Reflections on the Nature of Interaction CoDesign, 3, 139–143.Google Scholar
  26. Ferreira, J. J., de Souza, C. S., de Castro Salgado, L. C., Slaviero, C., Leitão, C. F., de F. Moreira F. (2012), Combining cognitive, semiotic and discourse analysis to explore the power of notations in visual programming. In 2012 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC)., September, 2012 (pp. 101–108). Los Alamitos, CA: IEEE Computer Society.Google Scholar
  27. Fischer, G., & Giaccardi, E. (2006). Meta-design: a framework for the future of end-user development. End user development (pp. 427–457). Netherlands: Springer.CrossRefGoogle Scholar
  28. Franchi, S., & Güzeldere, G. (2005). Machinations of the mind: cybernetics and artificial intelligence from automata to cyborgs. In S. Franchi, & G. Güzeldere (Eds.). Mechanical bodies, computational minds (pp. 15–149). Cambridge, MA: The MIT Press.Google Scholar
  29. Gerd Waloszek. (2005). Book review: the semiotic engineering of human-computer interaction. https://experience.sap.com/archived/review_semiotic_eng/ Accessed 24 May 2016.
  30. Green, T. (2006). Aims, achievements, agenda—where CDs stand now. Journal of Visual Languages & Computing, 17(4), 288–291.CrossRefGoogle Scholar
  31. Green, T. R. G., & Petre, M. (1996). Usability analysis of visual programming environments: a ‘cognitive dimensions’ framework. Journal of Visual Languages & Computing, 7(2), 131–174.CrossRefGoogle Scholar
  32. Greenwood, D. J., & Levin, M. (2007). Introduction to action research. Social research for social change (2nd Edition) Thousand Oaks, CA: Sage Publications, Inc.CrossRefGoogle Scholar
  33. Hundhausen, C. D. (2005). Using end user visualization environments to mediate conversations: a ‘communicative dimensions’ framework. Journal of Visual Languages and Computing, 16(3), 153–185.CrossRefGoogle Scholar
  34. Intrator, C. (2009). Using web scripts to improve accessibility. MSc dissertation. Departamento de Informática. PUC-Rio. 103 p. http://www2.dbd.puc-rio.br/pergamum/tesesabertas/0711270_09_pretextual.pdf.
  35. Intrator, C., & de Souza, C. S. (2008). Using Web scripts to improve accessibility. In IHC 2008 - VIII Simpósio Brasileiro de Fatores Humanos em Sistemas Computacionais, 2008, Porto Alegre. Proceedings of the VIII Brazilian Symposium on Human Factors in Computing Systems (v. 378. pp. 292–295). New York: ACM.Google Scholar
  36. Jones, S. P., Blackwell, A., Burnett, M. (2003). A user-centred approach to functions in Excel. In Proceedings of the eighth ACM SIGPLAN international conference on Functional programming (ICFP ’03) (pp. 165–176). New York: ACM.Google Scholar
  37. Kammersggard, J. (1988). Four different perspectives on human–computer interaction. International Journal of Man-Machine Studies, 28, 343–362.CrossRefGoogle Scholar
  38. Kelleher, C., & Pausch, R. (2007). Using storytelling to motivate programming. Communications of the ACM, 50, 58–64.CrossRefGoogle Scholar
  39. Kuutti, K. (1996). Activity theory as a potential framework for human-computer interaction research. In Context and consciousness: activity theory and human-computer interaction (pp. 17–44). Cambridge, Mass.: MIT Press.Google Scholar
  40. Leshed, G., Haber, E. M., Matthews, T., Lau, T. (2008). CoScripter: automating & sharing how-to knowledge in the enterprise. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (CHI ’08) (pp. 1719–1728). New York, NY, USA: ACM. doi: 10.1145/1357054.1357323.Google Scholar
  41. Lieberman, H., Paternò, F., Wulf, V. (Eds.) (2006). End user development. Netherlands, Dordrecht: Springer.Google Scholar
  42. Liu, K. (2000). Semiotics in information systems engineering. Cambridge: Cambridge University Press.Google Scholar
  43. Maceli, M., & Atwood, M. E. (2011). From human crafters to human factors to human actors and back again: bridging the design time–use time divide. In International symposium on end user development (pp. 76–91). Berlin Heidelberg: Springer.Google Scholar
  44. Maceli, M., & Atwood, M. E. (2013). “Human crafters” once again: supporting users as designers in continuous co-design. In International symposium on end user development (pp. 9–24). Berlin Heidelberg: Springer.Google Scholar
  45. Mehra, A., Grundy J., Hosking, J. (2005). A generic approach to supporting diagram differencing and merging for collaborative design. In Proceedings of the 20th IEEE/ACM international conference on automated software engineering (ASE ’05) (pp. 204–213). New York: ACM.Google Scholar
  46. Monteiro, I. T., da Silva Alves, A., de Souza, C. S. (2013). Using mediated communication to teach vocational concepts to deaf users. In C. Stephanidis, & M. Antona (Eds.). Universal access in human-computer interaction. applications and services for quality of life: 7th international conference, UAHCI 2013, held as part of HCI international 2013, Las Vegas, NV, USA, July 21–26, 2013, Proceedings, Part III (pp. 213–222). Berlin Heidelberg: Springer.CrossRefGoogle Scholar
  47. Monteiro, I. T., & de Souza, C. S. (2012). The representation of self in mediated interaction with computers. In Proceedings of the 11th Brazilian Symposium on Human Factors in Computing Systems (IHC ’12) (pp. 219–228). Porto Alegre, Brazil: Brazilian Computer Society.Google Scholar
  48. Monteiro, I. T., de Souza, C. S., Tolmasquim, E. T. (2015). My program, my world: insights from 1st-person reflective programming in EUD education. In P. Díaz, V. Pipek, C. Ardito, C. Jensen, I. Aedo, A. Boden (Eds.), End-user development: 5th international symposium, IS-EUD 2015, Madrid, Spain, May 26–29, 2015. Proceedings (pp. 76–91). Cham: Springer.Google Scholar
  49. Monteiro, I. T., de Castro Salgado, L. C., Mota, M. P., Sampaio, A. L., de Souza, C. S. (June 2017). Signifying software engineering to computational thinking learners with AgentSheets and PoliFacets. Journal of Visual Languages & Computing, 40, 91–112. doi: 10.1016/j.jvlc.2017.01.005.CrossRefGoogle Scholar
  50. Monteiro, I. T., Tolmasquim, E. T., & de Souza, C. S. (2013). Going back and forth in metacommunication threads. In Proceedings of the 12th Brazilian symposium on human factors in computing systems (IHC ’13). Brazilian computer society, Porto Alegre, Brazil (pp. 102–111).Google Scholar
  51. Mota, M. P., Faria, L. S., de Souza, C. S. (2012). Documentation comes to life in computational thinking acquisition with agentsheets. In Proceedings of the 11th Brazilian Symposium on Human Factors in Computing Systems (IHC ’12) (pp. 151–160). Porto Alegre, Brazil: Brazilian Computer Society.Google Scholar
  52. Mota, M. P., Monteiro, I. T., Ferreira, J. J., Slaviero, C., de Souza, C. S. (2013). On signifying the complexity of inter-agent relations in AgentSheets games and simulations. In Proceedings of the 31st ACM international conference on design of communication (pp. 133–142). New York: ACM.Google Scholar
  53. Nadin, M. (1988). Interface design and evaluation – semiotic implications. In H. R. Hartson, & D. Hix (Ed.), Advances in human-computer interaction ablex (v. 2, pp. 45–100).Google Scholar
  54. Nadin, M. (2011). Computation, information, meaning. Anticipation and Games International Journal of Applied Research on Information Technology and Computing, 2, 50–76.Google Scholar
  55. Nake, F. (2005). Computer art: a personal recollection. In Proceedings of the 5th conference on Creativity & cognition (C&C ’05) (pp. 54–62). New York, NY, USA: ACM. doi: 10.1145/1056224.1056234.CrossRefGoogle Scholar
  56. Nake, F., & Grabowski, S. (2001). Human–computer interaction viewed as pseudo-communication. Knowledge-Based Systems, 14, 441–447.CrossRefGoogle Scholar
  57. Norman, D. A. (1986). Cognitive engineering. In D. Norman, & S. Draper (Eds.), User centered systems design (pp. 31–629). Hillsdale, N.J.: L. Erlbaum Associates.Google Scholar
  58. Norman, D. A. (2009). THE WAY I SEE IT: Systems thinking: a product is more than the product. Interactions, 16, 52–54.CrossRefGoogle Scholar
  59. Oberquelle, H., Kupka, I., Maass, S. (1983). A view of human—machine communication and co-operation. International Journal of Man-Machine Studies, 19, 309–333.CrossRefGoogle Scholar
  60. Peirce, C. S. (1992). The Essential Peirce. Selected Philosophical Writings Volume 1 (1987–1893) edited by Nathan Houser and Christian Kloesel. Bloomington IN: Indiana University Press.Google Scholar
  61. Peirce, C. S. (1998). The Essential Peirce. Selected Philosophical Writings Volume 2 (1893–1913) edited by Nathan Houser and Christian Kloesel. Bloomington IN: Indiana University Press.Google Scholar
  62. Repenning, A. (2011). Making programming more conversational. 2011 IEEE Symposium on Visual Languages and Human-Centric Computing (VL/HCC) (pp. 191–194). Los Alamitos, CA: IEEE Computer Society.CrossRefGoogle Scholar
  63. Repenning, A., & Ioannidou, A. (2004). Agent-based end-user development. Communications of the ACM, 47(9), 43–46.CrossRefGoogle Scholar
  64. Repenning, A., & Ioannidou, A. (2006). What makes end-user development tick? 13 design guidelines. In H. Lieberman, F. Paternò, V. Wulf, (Eds.). End user development (pp. 51–85). Netherlands: Springer.CrossRefGoogle Scholar
  65. Repenning, A., Webb, D., Ioannidou, A. (2010). Scalable game design and the development of a checklist for getting computational thinking into public schools. In Proceedings of the 41st ACM technical symposium on Computer science education (SIGCSE ’10) (pp. 265–269). New York, NY, USA: ACM. doi: 10.1145/1734263.1734357.CrossRefGoogle Scholar
  66. Rogers, Y. (2012). HCI theory: classical, modern, and contemporary. Synthesis Lectures on Human-Centered Informatics 5.2, 1–129.Google Scholar
  67. Salgado, L. C. C., Leitão, C. F., de Souza, C. (2012). A journey through cultures: metaphors for guiding the design of cross-cultural interactive systems. London; New York: Springer.Google Scholar
  68. Searle, J. R. (1985). Expression and meaning: studies in the theory of speech acts. Cambridge: Cambridge University Press.Google Scholar
  69. Simon, H. A. (1996). The sciences of the artificial. 3rd Edition Cambridge, MA: The MIT Press.Google Scholar
  70. Stokes, D. E. (2011). Pasteur’s quadrant: basic science and technological innovation. Washington, DC: Brookings Institution Press.Google Scholar
  71. Tanaka-Ishii, K. (2010). Semiotics of programming. Cambridge: Cambridge University Press.Google Scholar
  72. Tetteroo, D., & Markopoulos. (2015). A review of research methods in end user development. In P. Díaz, V. Pipek, C. Ardito, C. Jensen, I. Aedo, A. Boden (Eds.), End-user development: 5th international symposium, IS-EUD 2015, Madrid, Spain, May 26–29, 2015 (pp. 58–75). Cham: Springer.Google Scholar
  73. Turkle, S. (2005). The second self: computers and the human spirit. Twentieth Anniversary Edition. Cambridge, MA: The MIT Press.Google Scholar
  74. Venable, J. (2006). The role of theory and theorising in design science research. In S. Chatterjee & A. Hevner (Eds), Proceedings of the First International Conference on Design Science in Information Systems and Technology (DESRIST 2006) (pp. 1–18). Claremont, CA: Claremont Graduate University.Google Scholar
  75. Winograd, T. (1997). The design of interaction. In P. Denning (Ed.). Beyond calculation: the next fifty years of computing (pp. 149–161). New York: Springer.CrossRefGoogle Scholar
  76. Winograd, T. (2006). Designing a new foundation for design. Communications of the ACM, 49, 71–74.CrossRefGoogle Scholar
  77. Winograd, T., & Flores, F. (1986). Understanding computers and cognition: a new foundation for design. Boston, Mass.: Addison-Wesley.Google Scholar
  78. Wolz, U., Stone, M., Pearson, K., Pulimood, S. M., Switzer, M. (2011). Computational thinking and expository writing in the middle school. Transactions on Computing Education, 11, 9:1–9:22.CrossRefMATHGoogle Scholar

Copyright information

© Springer International Publishing AG 2017

Authors and Affiliations

  1. 1.Semiotic Engineering Research Group (SERG), Departamento de Informática, PUC-RioRio de JaneiroBrazil

Personalised recommendations