Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Analyzing the usability of the WYRED Platform with undergraduate students to improve its features


The WYRED ecosystem is a technological ecosystem developed as part of WYRED (netWorked Youth Research for Empowerment in the Digital society), a European Project funded by the Horizon 2020 program. The main aim of the project is to provide a framework for research in which children and young people can express and explore their perspectives and interests concerning digital society. The WYRED ecosystem supports this framework from a technological point of view. The WYRED Platform is one of the main software components of this complex technological solution; it is focused on supporting the social dialogues that take place between children, young people and stakeholders. The ecosystem, and in particular the Platform, are already developed, but it is vital to ensure the acceptance by the final users, the children and young people mainly. This work presents the usability test carried out to evolve the Platform through the System Usability Scale. This usability test allows the identification of the weaknesses of the Platform regarding its characteristics, also allowing the corresponding improvement of the WYRED Platform, and it will serve as a reference for further usability testing.

This is a preview of subscription content, log in to check access.

Fig. 1
Fig. 2
Fig. 3
Fig. 4
Fig. 5
Fig. 6
Fig. 7


  1. 1.

    García-Peñalvo, F.J., Rodríguez-Conde, M.J., Seoane-Pardo, A.M., Conde-González, M.Á., Zangrando, V., García-Holgado, A.: GRIAL (GRupo de investigación en InterAcción y eLearning), USAL. IE Comun. Rev. Iberoam. Inform. Educ. 15, 85–94 (2012)

  2. 2.

    GRIAL Group: GRIAL Research Group Scientific Production Report (2011–2017). Version 2.0. GRIAL Research Group, University of Salamanca, Salamanca (2018). https://doi.org/10.5281/zenodo.1217097

  3. 3.

    Rodríguez Conde, M.J., García-Holgado, A., Zangrando, A., García-Peñalvo, F.J.: Delphi study to identify the young people priorities about digital society. In: García-Peñalvo F.J. (eds.) Proceedings of the 6th International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM 2018) (Salamanca, Spain, 24–26 Oct 2018). ACM International Conference Proceeding Series (ICPS). ACM, New York (2018). https://doi.org/10.1145/3284179.3284183

  4. 4.

    García-Peñalvo, F.J., Kearney, N.A.: Networked youth research for empowerment in digital society: the WYRED project. In: García-Peñalvo F.J. (eds.) Proceedings of the Fourth International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM’16) (Salamanca, Spain, 2–4 Nov 2016). ACM International Conference Proceeding Series (ICPS), pp. 3–9. ACM, New York, NY, USA (2016). https://doi.org/10.1145/3012430.3012489

  5. 5.

    García-Peñalvo, F.J.: The WYRED project: a technological platform for a generative research and dialogue about youth perspectives and interests in digital society. J. Inf. Technol. Res. 9(4), vi–x (2016)

  6. 6.

    García-Peñalvo, F.J., García-Holgado, A.: WYRED, a platform to give young people the voice on the influence of technology in today’s society. A citizen science approach. In: Villalba-Condori K.O., García-Peñalvo F.J., Lavonen J., Zapata-Ros M. (eds.) Proceedings of the II Congreso Internacional de Tendencias e Innovación Educativa—CITIE 2018 (Arequipa, Perú, 26–30 Nov 2018), pp. 128–141. CEUR-WS.org, Aachen (2019)

  7. 7.

    Fidalgo-Blanco, A., Sein-Echaluce, M.L., García-Peñalvo, F.J.: Epistemological and ontological spirals: from individual experience in educational innovation to the organisational knowledge in the university sector. Program Electron. Libr. Inf. Syst. 49(3), 266–288 (2015). https://doi.org/10.1108/prog-06-2014-0033

  8. 8.

    Fidalgo-Blanco, A., Sein-Echaluce, M.L., García-Peñalvo, F.J.: Knowledge spirals in higher education teaching innovation. Int. J. Knowl. Manag. 10(4), 16–37 (2014). https://doi.org/10.4018/ijkm.2014100102

  9. 9.

    García-Holgado, A., García-Peñalvo, F.J.: Architectural pattern to improve the definition and implementation of eLearning ecosystems. Sci. Comput. Program. 129, 20–34 (2016). https://doi.org/10.1016/j.scico.2016.03.010

  10. 10.

    Alspaugh, T.A., Asuncion, H.U., Scacchi, W.: The role of software licenses in open architecture ecosystems. In: Proceedings of the First International Workshop on Software Ecosystems 2009, pp. 4–18. CEUR-WS (2009)

  11. 11.

    Chang, E., West, M.: Digital ecosystems a next generation of the collaborative environment. In: Kotsis G., Taniar D., Pardede E., Ibrahim I.K. (eds.) Proceedings of iiWAS’2006—The Eighth International Conference on Information Integration and Web-based Applications Services, 4–6 Dec 2006, Yogyakarta, Indonesia, pp. 3–24. Austrian Computer Society (2006)

  12. 12.

    Chen, W., Chang, E.: Exploring a digital ecosystem conceptual model and its simulation prototype. In: Proceedings of IEEE International Symposium on Industrial Electronics, 2007 (ISIE 2007), pp. 2933–2938. IEEE (2007). https://doi.org/10.1109/isie.2007.4375080

  13. 13.

    Laanpere, M.: Digital learning ecosystems: rethinking virtual learning environments in the age of social media. Paper Presented at the IFIP-OST’12: Open and Social Technologies for Networked Learning, Taillinn, Estonia (2012)

  14. 14.

    Pata, K.: Meta-design framework for open learning ecosystems. Paper Presented at the Mash-UP Personal Learning Environments (MUP/PLE 2011), London, UK (2011)

  15. 15.

    Mens, T., Claes, M., Grosjean, P., Serebrenik, A.: Studying evolving software ecosystems based on ecological models. In: Mens T., Serebrenik A., Cleve A. (eds.) Evolving Software Systems, pp. 297–326. Springer, Berlin (2014). https://doi.org/10.1007/978-3-642-45398-4_10

  16. 16.

    Lungu, M.F.: Towards reverse engineering software ecosystems. In: 2008 IEEE International Conference on Software Maintenance, ICSM 2008 (Beijing, China, 28 September–4 Oct 2008), pp. 428–431. IEEE (2008). https://doi.org/10.1109/icsm.2008.4658096

  17. 17.

    Lungu, M.F.: Reverse Engineering Software Ecosystems. University of Lugano, Lugano (2009)

  18. 18.

    Nachira, F.: Towards a Network of Digital Business Ecosystems Fostering the Local Development. European Commission, Brussels (2002)

  19. 19.

    García-Holgado, A., García-Peñalvo, F.J.: The evolution of the technological ecosystems: an architectural proposal to enhancing learning processes. In: Proceedings of the First International Conference on Technological Ecosystem for Enhancing Multiculturality (TEEM’13) (Salamanca, Spain, 14–15 Nov 2013), pp 565–571. ACM, New York (2013). https://doi.org/10.1145/2536536.2536623

  20. 20.

    García-Holgado, A., García-Peñalvo, F.J.: Architectural pattern for the definition of eLearning ecosystems based on Open Source developments. In: Sierra-Rodríguez J.L., Dodero-Beardo J.M., Burgos D. (eds.) Proceedings of 2014 International Symposium on Computers in Education (SIIE) (Logroño, La Rioja, Spain, 12–14 Nov 2014). Institute of Electrical and Electronics Engineers, pp. 93–98. IEEE Catalog Number CFP1486T-ART (2014). https://doi.org/10.1109/siie.2014.7017711

  21. 21.

    García-Peñalvo, F.J., García-Holgado, A. (eds.): Open Source Solutions for Knowledge Management and Technological Ecosystems. Advances in Knowledge Acquisition, Transfer, and Management (AKATM) Book Series. IGI Global, Hershey (2017)

  22. 22.

    García-Holgado, A., García-Peñalvo, F.J.: Preliminary validation of the metamodel for developing learning ecosystems. In: Dodero J.M., Ibarra Sáiz M.S., Ruiz Rube I. (eds.) Proceedings of the 5th International Conference on Technological Ecosystems for Enhancing Multiculturality (TEEM 2017) (Cádiz, Spain, 18–20 Oct 2017). ACM International Conference Proceeding Series (ICPS). ACM, New York (2017). https://doi.org/10.1145/3144826.3145439

  23. 23.

    García-Holgado, A., García-Peñalvo, F.J.: Human interaction in learning ecosystems based in Open Source solutions. In: Learning and Collaboration Technologies. Design, Development and Technological Innovation. 5th International Conference, LCT 2018. Held as Part of HCI International 2018, Las Vegas, NV, USA, 15–20 July 2018, vol. 10924. Springer International Publishing (2018). https://doi.org/10.1007/978-3-319-91743-6_17

  24. 24.

    Rubio Royo, E., Cranfield McKay, S., Nelson-Santana, J.C., Delgado Rodríguez, R.N., Ocon-Carreras, A.A.: Web knowledge turbine as a proposal for personal and professional self-organisation in complex times: application to higher education. J. Inf. Technol. Res. 11(1), 70–90 (2018). https://doi.org/10.4018/JITR.2018010105

  25. 25.

    García-Holgado, A., García-Peñalvo, F.J.: A metamodel proposal for developing learning ecosystems. In: Zaphiris P., Ioannou A. (eds.) Learning and Collaboration Technologies. Novel Learning Ecosystems. 4th International Conference, LCT 2017. Held as Part of HCI International 2017, Vancouver, BC, Canada, 9–14 July 2017. Proceedings, Part I, vol. 10295, pp. 100–109. Springer International Publishing (2017). https://doi.org/10.1007/978-3-319-58509-3_10

  26. 26.

    García-Peñalvo, F.J., Durán-Escudero, J.: Interaction design principles in WYRED platform. In: Zaphiris P., Ioannou A. (eds.) Learning and Collaboration Technologies. Technology in Education. 4th International Conference, LCT 2017. Held as Part of HCI International 2017, Vancouver, BC, Canada, 9–14 July 2017. Proceedings, Part II, pp. 371–381. Springer International Publishing (2017). https://doi.org/10.1007/978-3-319-58515-4_29

  27. 27.

    Mening, R.: WordPress vs Joomla vs Drupal + CMS “comparison chart” (2014). http://websitesetup.org/cms-comparison-wordpress-vs-joomla-drupal/. Accessed 17 July 2019

  28. 28.

    Rackspace: CMS Comparison: Drupal, Joomla and Wordpress (2013). https://support.rackspace.com/how-to/cms-comparison-drupal-joomla-and-wordpress/. Accessed 17 July 2019

  29. 29.

    WYRED Consortium (2018) Platform v2. WP3_D3.3. V1.2. https://doi.org/10.5281/zenodo.1248813

  30. 30.

    Ministerio de Educación Cultura y Deporte: Datos y cifras del sistema universitario español. Curso 2015/2016. Ministerio de Educación, Cultura y Deporte, Madrid, Spain (2016)

  31. 31.

    Chin, J.P., Diehl, V.A., Norman, K.L.: Development of an instrument measuring user satisfaction of the human–computer interface. In: Proceedings of the SIGCHI Conference on Human Factors in Computing Systems, pp. 213–218. ACM (1988)

  32. 32.

    Kirakowski, J., Corbett, M.: SUMI: the software usability measurement inventory. Br. J. Edu. Technol. 24(3), 210–212 (1993)

  33. 33.

    Lewis, J.R.: IBM computer usability satisfaction questionnaires: psychometric evaluation and instructions for use. Int. J. Hum. Comput. Interact. 7(1), 57–78 (1995)

  34. 34.

    Sauro, J., Lewis, J.R.: Quantifying the User Experience: Practical Statistics for User Research. Morgan Kaufmann, Burlington (2016)

  35. 35.

    Tullis, T.S., Stetson, J.N.: A comparison of questionnaires for assessing website usability. In: Usability Professional Association Conference, pp. 1–12 (2004)

  36. 36.

    ISO 9241-11: Ergonomic Requirements for Office Work with Visual Display Terminals (VDTs), vol. 45, p. 9. The International Organization for Standardization, Geneva (1998)

  37. 37.

    Brooke, J.: SUS: a retrospective. J. Usabil. Stud. 8(2), 29–40 (2013)

  38. 38.

    Bangor, A., Kortum, P.T., Miller, J.T.: An empirical evaluation of the system usability scale. Int. J. Hum. Comput. Interact. 24(6), 574–594 (2008)

  39. 39.

    Brooke, J.: SUS-A quick and dirty usability scale. Usabil. Eval. Ind. 189(194), 4–7 (1996)

  40. 40.

    Finstad, K.: The system usability scale and non-native english speakers. J. Usabil. Stud. 1(4), 185–188 (2006)

  41. 41.

    McKinney, W.: Pandas: a foundational Python library for data analysis and statistics. Paper presented at the PyHPC 2011, Workshop on Python for High Performance and Scientific Computing. Seattle, WA, USA (2011)

  42. 42.

    Bangor, A., Kortum, P., Miller, J.: Determining what individual SUS scores mean: adding an adjective rating scale. J. Usabil. Stud. 4(3), 114–123 (2009)

  43. 43.

    Sauro, J.: A Practical Guide to the System Usability Scale: Background, Benchmarks & Best Practices. Measuring Usability LLC, Denver (2011)

  44. 44.

    Vázquez-Ingelmo, A.: Code repository that supports the analysis of the usability test (through the System Usability Scale) applied to the WYRED Platform (2018). https://github.com/AndVazquez/wyred-usability-analysis. Accessed 17 July 2019

  45. 45.

    Nielsen, J., Loranger, H.: Prioritizing Web Usability. Pearson Education, London (2006)

  46. 46.

    Kortum, P., Sorber, M.: Measuring the usability of mobile applications for phones and tablets. Int. J. Hum. Comput. Interact. 31(8), 518–529 (2015)

  47. 47.

    Lewis, J.R., Sauro, J.: The Factor Structure of the System Usability Scale. Human Centered Design, pp. 94–103. Springer, Berlin (2009)

  48. 48.

    Borsci, S., Federici, S., Bacci, S., Gnaldi, M., Bartolucci, F.: Assessing user satisfaction in the era of user experience: comparison of the SUS, UMUX, and UMUX-LITE as a function of product experience. Int. J. Hum. Comput. Interact. 31(8), 484–495 (2015)

  49. 49.

    Vázquez-Ingelmo, A.: Code repository that supports the analysis in the paper “usability test of WYRED platform” (2018). http://doi.org/10.5281/zenodo.1164124

  50. 50.

    Yamane, T: Statistics: An Introductory Analysis. Harper & Row, NY (1967)

  51. 51.

    Sauro, J.: SUSTisfied? Little-known System Usability Scale facts. User Exp. Mag. 10(3) (2011). http://uxpamagazine.org/sustified/?lang=es

  52. 52.

    Boone, H.N., Boone, D.A.: Analyzing likert data. J. Ext. 50(2), 1–5 (2012)

  53. 53.

    McLellan, S., Muddimer, A., Peres, S.C.: The effect of experience on System Usability Scale ratings. J. Usabil. Stud. 7(2), 56–67 (2012)

Download references


This research work has been carried out within the University of Salamanca Ph.D. program on Education in the Knowledge Society scope (http://knowledgesociety.usal.es) and was supported by the Spanish Ministry of Education, Culture and Sport under a FPU fellowship (FPU014/04783). With the support of the EU Horizon 2020 program, “Europe in a changing world—inclusive, innovative and reflective Societies (HORIZON 2020: REV-INEQUAL-10-2016: Multi-stakeholder platform for enhancing youth digital opportunities)” Call, Project WYRED (netWorked Youth Research for Empowerment in the Digital society) (Grant Agreement No 727066). The sole responsibility for the content of this webpage lies with the authors. It does not necessarily reflect the opinion of the European Union. The European Commission is not responsible for any use that may be made of the information contained therein. This work has been partially funded also by the Spanish Government Ministry of Economy and Competitiveness throughout the DEFINES Project (Ref. TIN2016- 80172-R).

Author information

Correspondence to Alicia García-Holgado.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

García-Peñalvo, F.J., Vázquez-Ingelmo, A., García-Holgado, A. et al. Analyzing the usability of the WYRED Platform with undergraduate students to improve its features. Univ Access Inf Soc 18, 455–468 (2019). https://doi.org/10.1007/s10209-019-00672-z

Download citation


  • Human interaction
  • Usability analysis
  • Digital society
  • Citizen science
  • Communication networks
  • Technological ecosystems