Skip to main content

Analyzing the usability of the WYRED Platform with undergraduate students to improve its features

Universal Access in the Information Society volume 18, pages 455–468 (2019)Cite this article

Abstract

The WYRED ecosystem is a technological ecosystem developed as part of WYRED (netWorked Youth Research for Empowerment in the Digital society), a European Project funded by the Horizon 2020 program. The main aim of the project is to provide a framework for research in which children and young people can express and explore their perspectives and interests concerning digital society. The WYRED ecosystem supports this framework from a technological point of view. The WYRED Platform is one of the main software components of this complex technological solution; it is focused on supporting the social dialogues that take place between children, young people and stakeholders. The ecosystem, and in particular the Platform, are already developed, but it is vital to ensure the acceptance by the final users, the children and young people mainly. This work presents the usability test carried out to evolve the Platform through the System Usability Scale. This usability test allows the identification of the weaknesses of the Platform regarding its characteristics, also allowing the corresponding improvement of the WYRED Platform, and it will serve as a reference for further usability testing.

This is a preview of subscription content, access via your institution.

Access options

Buy single article

Instant access to the full article PDF.

USD 39.95

Price includes VAT (USA)
Tax calculation will be finalised during checkout.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7

References

  1. García-Peñalvo, F.J., Rodríguez-Conde, M.J., Seoane-Pardo, A.M., Conde-González, M.Á., Zangrando, V., García-Holgado, A.: GRIAL (GRupo de investigación en InterAcción y eLearning), USAL. IE Comun. Rev. Iberoam. Inform. Educ. 15, 85–94 (2012)

    Google Scholar 

  2. GRIAL Group: GRIAL Research Group Scientific Production Report (2011–2017). Version 2.0. GRIAL Research Group, University of Salamanca, Salamanca (2018). https://doi.org/10.5281/zenodo.1217097

    Google Scholar 

  3. Rodríguez Conde, M.J., García-Holgado, A., Zangrando, A., García-Peñalvo, F.J.: Delphi study to identify the young people priorities about digital society. In: García-Peñalvo F.J. (eds.) Proceedings of the 6th International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM 2018) (Salamanca, Spain, 24–26 Oct 2018). ACM International Conference Proceeding Series (ICPS). ACM, New York (2018). https://doi.org/10.1145/3284179.3284183

  4. García-Peñalvo, F.J., Kearney, N.A.: Networked youth research for empowerment in digital society: the WYRED project. In: García-Peñalvo F.J. (eds.) Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’16) (Salamanca, Spain, 2–4 Nov 2016). ACM International Conference Proceeding Series (ICPS), pp. 3–9. ACM, New York, NY, USA (2016). https://doi.org/10.1145/3012430.3012489

  5. García-Peñalvo, F.J.: The WYRED project: a technological platform for a generative research and dialogue about youth perspectives and interests in digital society. J. Inf. Technol. Res. 9(4), vi–x (2016)

    Google Scholar 

  6. García-Peñalvo, F.J., García-Holgado, A.: WYRED, a platform to give young people the voice on the influence of technology in today’s society. A citizen science approach. In: Villalba-Condori K.O., García-Peñalvo F.J., Lavonen J., Zapata-Ros M. (eds.) Proceedings of the II Congreso Internacional de Tendencias e Innovación Educativa—CITIE 2018 (Arequipa, Perú, 26–30 Nov 2018), pp. 128–141. CEUR-WS.org, Aachen (2019)

  7. Fidalgo-Blanco, A., Sein-Echaluce, M.L., García-Peñalvo, F.J.: Epistemological and ontological spirals: from individual experience in educational innovation to the organisational knowledge in the university sector. Program Electron. Libr. Inf. Syst. 49(3), 266–288 (2015). https://doi.org/10.1108/prog-06-2014-0033

    Google Scholar 

  8. Fidalgo-Blanco, A., Sein-Echaluce, M.L., García-Peñalvo, F.J.: Knowledge spirals in higher education teaching innovation. Int. J. Knowl. Manag. 10(4), 16–37 (2014). https://doi.org/10.4018/ijkm.2014100102

    Article  Google Scholar 

  9. García-Holgado, A., García-Peñalvo, F.J.: Architectural pattern to improve the definition and implementation of eLearning ecosystems. Sci. Comput. Program. 129, 20–34 (2016). https://doi.org/10.1016/j.scico.2016.03.010

    Article  Google Scholar 

  10. Alspaugh, T.A., Asuncion, H.U., Scacchi, W.: The role of software licenses in open architecture ecosystems. In: Proceedings of the First International Workshop on Software Ecosystems 2009, pp. 4–18. CEUR-WS (2009)

  11. Chang, E., West, M.: Digital ecosystems a next generation of the collaborative environment. In: Kotsis G., Taniar D., Pardede E., Ibrahim I.K. (eds.) Proceedings of iiWAS’2006—The Eighth International Conference on Information Integration and Web-based Applications Services, 4–6 Dec 2006, Yogyakarta, Indonesia, pp. 3–24. Austrian Computer Society (2006)

  12. Chen, W., Chang, E.: Exploring a digital ecosystem conceptual model and its simulation prototype. In: Proceedings of IEEE International Symposium on Industrial Electronics, 2007 (ISIE 2007), pp. 2933–2938. IEEE (2007). https://doi.org/10.1109/isie.2007.4375080

  13. Laanpere, M.: Digital learning ecosystems: rethinking virtual learning environments in the age of social media. Paper Presented at the IFIP-OST’12: Open and Social Technologies for Networked Learning, Taillinn, Estonia (2012)

  14. Pata, K.: Meta-design framework for open learning ecosystems. Paper Presented at the Mash-UP Personal Learning Environments (MUP/PLE 2011), London, UK (2011)

  15. Mens, T., Claes, M., Grosjean, P., Serebrenik, A.: Studying evolving software ecosystems based on ecological models. In: Mens T., Serebrenik A., Cleve A. (eds.) Evolving Software Systems, pp. 297–326. Springer, Berlin (2014). https://doi.org/10.1007/978-3-642-45398-4_10

  16. Lungu, M.F.: Towards reverse engineering software ecosystems. In: 2008 IEEE International Conference on Software Maintenance, ICSM 2008 (Beijing, China, 28 September–4 Oct 2008), pp. 428–431. IEEE (2008). https://doi.org/10.1109/icsm.2008.4658096

  17. Lungu, M.F.: Reverse Engineering Software Ecosystems. University of Lugano, Lugano (2009)

    Google Scholar 

  18. Nachira, F.: Towards a Network of Digital Business Ecosystems Fostering the Local Development. European Commission, Brussels (2002)

    Google Scholar 

  19. García-Holgado, A., García-Peñalvo, F.J.: The evolution of the technological ecosystems: an architectural proposal to enhancing learning processes. In: Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality (TEEM’13) (Salamanca, Spain, 14–15 Nov 2013), pp 565–571. ACM, New York (2013). https://doi.org/10.1145/2536536.2536623

  20. García-Holgado, A., García-Peñalvo, F.J.: Architectural pattern for the definition of eLearning ecosystems based on Open Source developments. In: Sierra-Rodríguez J.L., Dodero-Beardo J.M., Burgos D. (eds.) Proceedings of 2014 International Symposium on Computers in Education (SIIE) (Logroño, La Rioja, Spain, 12–14 Nov 2014). Institute of Electrical and Electronics Engineers, pp. 93–98. IEEE Catalog Number CFP1486T-ART (2014). https://doi.org/10.1109/siie.2014.7017711

  21. García-Peñalvo, F.J., García-Holgado, A. (eds.): Open Source Solutions for Knowledge Management and Technological Ecosystems. Advances in Knowledge Acquisition, Transfer, and Management (AKATM) Book Series. IGI Global, Hershey (2017)

    Google Scholar 

  22. García-Holgado, A., García-Peñalvo, F.J.: Preliminary validation of the metamodel for developing learning ecosystems. In: Dodero J.M., Ibarra Sáiz M.S., Ruiz Rube I. (eds.) Proceedings of the 5th International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM 2017) (Cádiz, Spain, 18–20 Oct 2017). ACM International Conference Proceeding Series (ICPS). ACM, New York (2017). https://doi.org/10.1145/3144826.3145439

  23. García-Holgado, A., García-Peñalvo, F.J.: Human interaction in learning ecosystems based in Open Source solutions. In: Learning and Collaboration Technologies. Design, Development and Technological Innovation. 5th International Conference, LCT 2018. Held as Part of HCI International 2018, Las Vegas, NV, USA, 15–20 July 2018, vol. 10924. Springer International Publishing (2018). https://doi.org/10.1007/978-3-319-91743-6_17

  24. Rubio Royo, E., Cranfield McKay, S., Nelson-Santana, J.C., Delgado Rodríguez, R.N., Ocon-Carreras, A.A.: Web knowledge turbine as a proposal for personal and professional self-organisation in complex times: application to higher education. J. Inf. Technol. Res. 11(1), 70–90 (2018). https://doi.org/10.4018/JITR.2018010105

    Article  Google Scholar 

  25. García-Holgado, A., García-Peñalvo, F.J.: A metamodel proposal for developing learning ecosystems. In: Zaphiris P., Ioannou A. (eds.) Learning and Collaboration Technologies. Novel Learning Ecosystems. 4th International Conference, LCT 2017. Held as Part of HCI International 2017, Vancouver, BC, Canada, 9–14 July 2017. Proceedings, Part I, vol. 10295, pp. 100–109. Springer International Publishing (2017). https://doi.org/10.1007/978-3-319-58509-3_10

  26. García-Peñalvo, F.J., Durán-Escudero, J.: Interaction design principles in WYRED platform. In: Zaphiris P., Ioannou A. (eds.) Learning and Collaboration Technologies. Technology in Education. 4th International Conference, LCT 2017. Held as Part of HCI International 2017, Vancouver, BC, Canada, 9–14 July 2017. Proceedings, Part II, pp. 371–381. Springer International Publishing (2017). https://doi.org/10.1007/978-3-319-58515-4_29

  27. Mening, R.: WordPress vs Joomla vs Drupal + CMS “comparison chart” (2014). http://websitesetup.org/cms-comparison-wordpress-vs-joomla-drupal/. Accessed 17 July 2019

  28. Rackspace: CMS Comparison: Drupal, Joomla and Wordpress (2013). https://support.rackspace.com/how-to/cms-comparison-drupal-joomla-and-wordpress/. Accessed 17 July 2019

  29. WYRED Consortium (2018) Platform v2. WP3_D3.3. V1.2. https://doi.org/10.5281/zenodo.1248813

  30. Ministerio de Educación Cultura y Deporte: Datos y cifras del sistema universitario español. Curso 2015/2016. Ministerio de Educación, Cultura y Deporte, Madrid, Spain (2016)

  31. Chin, J.P., Diehl, V.A., Norman, K.L.: Development of an instrument measuring user satisfaction of the human–computer interface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 213–218. ACM (1988)

  32. Kirakowski, J., Corbett, M.: SUMI: the software usability measurement inventory. Br. J. Edu. Technol. 24(3), 210–212 (1993)

    Article  Google Scholar 

  33. Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 7(1), 57–78 (1995)

    Article  Google Scholar 

  34. Sauro, J., Lewis, J.R.: Quantifying the User Experience: Practical Statistics for User Research. Morgan Kaufmann, Burlington (2016)

    Google Scholar 

  35. Tullis, T.S., Stetson, J.N.: A comparison of questionnaires for assessing website usability. In: Usability Professional Association Conference, pp. 1–12 (2004)

  36. ISO 9241-11: Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs), vol. 45, p. 9. The International Organization for Standardization, Geneva (1998)

    Google Scholar 

  37. Brooke, J.: SUS: a retrospective. J. Usabil. Stud. 8(2), 29–40 (2013)

    Google Scholar 

  38. Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the system usability scale. Int. J. Hum. Comput. Interact. 24(6), 574–594 (2008)

    Article  Google Scholar 

  39. Brooke, J.: SUS-A quick and dirty usability scale. Usabil. Eval. Ind. 189(194), 4–7 (1996)

    Google Scholar 

  40. Finstad, K.: The system usability scale and non-native english speakers. J. Usabil. Stud. 1(4), 185–188 (2006)

    Google Scholar 

  41. McKinney, W.: Pandas: a foundational Python library for data analysis and statistics. Paper presented at the PyHPC 2011, Workshop on Python for High Performance and Scientific Computing. Seattle, WA, USA (2011)

  42. Bangor, A., Kortum, P., Miller, J.: Determining what individual SUS scores mean: adding an adjective rating scale. J. Usabil. Stud. 4(3), 114–123 (2009)

    Google Scholar 

  43. Sauro, J.: A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices. Measuring Usability LLC, Denver (2011)

    Google Scholar 

  44. Vázquez-Ingelmo, A.: Code repository that supports the analysis of the usability test (through the System Usability Scale) applied to the WYRED Platform (2018). https://github.com/AndVazquez/wyred-usability-analysis. Accessed 17 July 2019

  45. Nielsen, J., Loranger, H.: Prioritizing Web Usability. Pearson Education, London (2006)

    Google Scholar 

  46. Kortum, P., Sorber, M.: Measuring the usability of mobile applications for phones and tablets. Int. J. Hum. Comput. Interact. 31(8), 518–529 (2015)

    Article  Google Scholar 

  47. Lewis, J.R., Sauro, J.: The Factor Structure of the System Usability Scale. Human Centered Design, pp. 94–103. Springer, Berlin (2009)

    Google Scholar 

  48. Borsci, S., Federici, S., Bacci, S., Gnaldi, M., Bartolucci, F.: Assessing user satisfaction in the era of user experience: comparison of the SUS, UMUX, and UMUX-LITE as a function of product experience. Int. J. Hum. Comput. Interact. 31(8), 484–495 (2015)

    Article  Google Scholar 

  49. Vázquez-Ingelmo, A.: Code repository that supports the analysis in the paper “usability test of WYRED platform” (2018). http://doi.org/10.5281/zenodo.1164124

  50. Yamane, T: Statistics: An Introductory Analysis. Harper & Row, NY (1967)

  51. Sauro, J.: SUSTisfied? Little-known System Usability Scale facts. User Exp. Mag. 10(3) (2011). http://uxpamagazine.org/sustified/?lang=es

  52. Boone, H.N., Boone, D.A.: Analyzing likert data. J. Ext. 50(2), 1–5 (2012)

    Google Scholar 

  53. McLellan, S., Muddimer, A., Peres, S.C.: The effect of experience on System Usability Scale ratings. J. Usabil. Stud. 7(2), 56–67 (2012)

    Google Scholar 

Download references

Acknowledgements

This research work has been carried out within the University of Salamanca Ph.D. program on Education in the Knowledge Society scope (http://knowledgesociety.usal.es) and was supported by the Spanish Ministry of Education, Culture and Sport under a FPU fellowship (FPU014/04783). With the support of the EU Horizon 2020 program, “Europe in a changing world—inclusive, innovative and reflective Societies (HORIZON 2020: REV-INEQUAL-10-2016: Multi-stakeholder platform for enhancing youth digital opportunities)” Call, Project WYRED (netWorked Youth Research for Empowerment in the Digital society) (Grant Agreement No 727066). The sole responsibility for the content of this webpage lies with the authors. It does not necessarily reflect the opinion of the European Union. The European Commission is not responsible for any use that may be made of the information contained therein. This work has been partially funded also by the Spanish Government Ministry of Economy and Competitiveness throughout the DEFINES Project (Ref. TIN2016- 80172-R).

Author information

Authors and Affiliations

  1. GRIAL Research Group, Department of Computer Science, Research Institute for Educational Sciences, University of Salamanca, Paseo de Canalejas 169, 37008, Salamanca, Spain

    Francisco J. García-Peñalvo, Andrea Vázquez-Ingelmo, Alicia García-Holgado & Antonio M. Seoane-Pardo

Authors
  1. Francisco J. García-Peñalvo
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Andrea Vázquez-Ingelmo
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Alicia García-Holgado
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Antonio M. Seoane-Pardo
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Alicia García-Holgado.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

García-Peñalvo, F.J., Vázquez-Ingelmo, A., García-Holgado, A. et al. Analyzing the usability of the WYRED Platform with undergraduate students to improve its features. Univ Access Inf Soc 18, 455–468 (2019). https://doi.org/10.1007/s10209-019-00672-z

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10209-019-00672-z

Keywords

  • Human interaction
  • Usability analysis
  • Digital society
  • Citizen science
  • Communication networks
  • Technological ecosystems