Digital assessment in higher education

Promoting universal usability through requirements specification and universal design quality (UD-Q) reviews
  • Miriam Eileen Nes Begnum
  • Rikke Julie Foss-Pedersen
Long Paper


Statistics show there is a clear relationship between higher education and employment in Norway, especially for people with disabilities. The use of digital assessment solutions is increasing in Norwegian higher education. The overall goal of this study is therefore to highlight the potential for improvement of current practices related to universal design, both for providers of digital assessment solutions and for higher education institutions. Based on a case study of practices in Norwegian higher education sector, this article reviews existing requirements for ensuring universal design in digital assessment solutions, prototypes an approach to evaluating universal design quality (UD-Q) of two major Norwegian digital assessment solutions and investigates the compliance between providers’ self-assessments from interviews and UD-Q evaluation scores. The article presents two contributions: (1) an improved set of requirements for universal usability when procuring digital assessments solutions and (2) UD-Q, a stepwise feature analysis-based expert inspection method for evaluating the UD-Q of digital assessment solutions.


Universal design eLearning Digital assessment Higher education Requirement specification Expert evaluation Universal usability 



A warm thank you to UNINETT, Inspera, WISEflow (Uniwise) and all our HE institution participants.


  1. 1.
    The Norwegian Directorate for Children, Youth and Family Affairs, Utdanning. _og_utdanning/. (16 Sept 2015)
  2. 2.
    Lid, I.M.: Universell utforming - Verdigrunnlag, kunnskap og praksis. Cappelen Damm Akademisk, Oslo (2013)Google Scholar
  3. 3.
    The Nordic Institute for Studies in Innovation, Research and Education (NIFU): Institusjoner i universitets- og høgskolesektoren. (27 Oct 2016)
  4. 4.
    Statistics Norway (SSB): Studenter ved universiteter og høgskoler, 1. oktober (2015). (27 Oct 2016)
  5. 5.
    Norwegian Centre for Research Data (NSD): Database for Statistikk om Høgre Utdanning, Gjennomsnittsalder tilsatte. (27 Oct 2016)
  6. 6.
  7. 7.
    TNS Gallup, Universell, Prosjektrapport, Felles norsk lœringsmiljøundersøkelse (LMU) (2012). (27 Oct 2016)
  8. 8.
    Ministry of Children and Equality (BLD): Lov om forbud mot diskriminering på grunn av nedsatt funksjonsevne (diskriminerings- og tilgjengelighetsloven), LOV-2013-06-21-61, Lovdata (2013). (28 Oct 2016)
  9. 9.
    Syse, A., et. al.: NOU 2005:8, Likeverd og tilgjengelighet- Rettslig vern mot diskriminering på grunnlag av nedsatt funksjonsevne. Bedret tilgjengelighet for alle. Ministry of Justice and Public Security. (28 Oct 2016)
  10. 10.
    Ministry of Education and Research: Lov om universiteter og høyskoler (universitets- og høyskoleloven), LOV-2005-04-01-15. (28 Oct 2016)
  11. 11.
    Agency for Public Management and eGovernment (Difi), Utdanningssektorens plikt til universell utforming av IKT, report 2015:20. ISSN 1890-6583. (28 Oct 2016)
  12. 12.
    Ministry of Education and Research: Proposisjoner til Stortinget. Ot.prp. nr. 40 (2001-2002), Om lov om endringer i lov 12. mai 1995 nr. 22 om universiteter og høgskoler og lov 2. juli 1999 nr. 64 om helsepersonell. (28 Oct 2016)
  13. 13., Høringsnotat om felles likestillings- og diskrimineringslov. (13 Nov 2015)
  14. 14.
    Ministry of Local Government and Modernisation (KMD): Forskrift om universell utforming av informasjons- og kommunikasjonsteknologiske (IKT)-løsninger, FOR-2013-06-21-732, Lovdata (2013).
  15. 15., Høring - forslag til felles likestillings- og diskriminerslov. (19 Nov 2015)
  16. 16.
    Zhuhadar, L., Carson, B., Daday, J., Nasraoui, O.: A universal design infrastructure for multimodal presentation of materials in STEM programs: universal design. In: Proceedings of the 24th International Conference on World Wide Web, Florence, Italy, pp. 569–574 (2015)Google Scholar
  17. 17.
    Klironomos, I., Antona, M., Basdekis, I., Stephanidis, C.: White paper: promoting design for all and e-accessibility in Europe. Univ. Access Inf. Soc. 5(1), 105–119 (2006)CrossRefGoogle Scholar
  18. 18.
    Bocconi, S., Dini, S., Ferlino, L., Martinoli, C., Ott, M.: ICT educational tools and visually impaired students: different answers to different accessibility needs. In: Universal Access in Human-Computer Interaction, Applications and Services, Lecture Notes in Computer Science vol. 4556, pp. 491–500 (2007)Google Scholar
  19. 19.
    Rose, D.H., Strangman, N.: Universal design for learning: meeting the challenge of individual learning differences through a neurocognitive perspective. Univ. Access Inf. Soc. 5(4), 381–391 (2007)CrossRefGoogle Scholar
  20. 20.
    Buzzi, M., Buzzi, M., Leporini, B.: Accessing e-learning systems via screen reader: an example, Lecture Notes in Computer Science, 5613 Human-Computer Interaction. Interacting in Various Application Domains, pp. 21–30 (2009)Google Scholar
  21. 21.
    Bocconi, S., Ott, M.: ICT and universal access to education: towards a culture of accessibility. Inf. Syst. E-learn. Knowl. Manag. Res. Commun. Comput. Inf. Sci. 278, 330–337 (2013)CrossRefGoogle Scholar
  22. 22.
    Squires, D., Preece, J.: Predicting quality in educational software: evaluating for learning, usability and the synergy between them. Interact. Comput. 11(5), 467–483 (1999)CrossRefGoogle Scholar
  23. 23.
    Petrie, H., Bevan, N.: The Evaluation of Accessibility, Usability and User Experience, The Universal Access Handbook. CRC Press, Boca Raton (2009)Google Scholar
  24. 24.
    Benyon, D.: Designing Interactive Systems—A Comprehensive Guide to HCI, UX and Interaction Design, vol. 219, 3rd edn. Pearson, London (2014)Google Scholar
  25. 25.
    Nielsen, J.: Usability Engineering, pp. 155–156. Academic Press, London (1993)MATHGoogle Scholar
  26. 26.
    The Norwegian Directorate for Children, Youth and Family Affairs, Universell utforming A-B-C. (14 Sept 2015)
  27. 27.
    Lazar, J., Feng, J.H., Hochheiser, H.: Research Methods in Human-Computer Interaction, vol. 150. Wiley, Chichester (2010)Google Scholar
  28. 28.
    Granić, A., Ćukušić, M.: Universal design within the context of e-learning, universal access in human-computer interaction. In: Applications and Services, Lecture Notes in Computer Science vol. 4556, pp. 617–626 (2007)Google Scholar
  29. 29.
    Granić, A., Adams, R.: User sensitive research in e-learning: exploring the role of individual user characteristics. Univ. Access Inf. Soc. 10(3), 307–318 (2010)CrossRefGoogle Scholar
  30. 30.
    Pullin, G., Newell, A.: Focussing on extra-ordinary users. In: Universal Access in Human Computer Interaction, Coping with Diversity, Lecture Notes in Computer Science, vol. 4554, pp. 253–262 (2007)Google Scholar
  31. 31.
    Ardito, C., Costabile, M.F., De Marsico, M., Lanzilotti, R., Levialdi, S., Plantamura, P., Roselli, T., Rossano, V., Tersigni, M.: Towards guidelines for usability of e-learning applications. In: Lecture Notes in Computer Science, 3196 User-Centered Interaction Paradigms for Universal Access in the Information Society, pp. 185–202 (2004)Google Scholar
  32. 32.
    Ardito, C., Costabile, M.F., De Marsico, M., Lanzilotti, R., Levialdi, S., Roselli, T., Rossano, V.: An approach to usability evaluation of e-learning applications. Univ. Access Inf. Soc. 4(3), 270–283 (2006)CrossRefGoogle Scholar
  33. 33.
    Yesilada, Y., Brajnik, G., Harper, S.: How much does expertise matter? A barrier walkthrough study with experts and non-experts. In: Proceedings of the 11th international ACM SIGACCESS conference on Computers and accessibility, Pittsburgh, Pennsylvania, USA (2009)Google Scholar
  34. 34.
    Tanaka, E.H., Da Rocha, H.V.: Evaluation of web accessibility tools. In: Proceedings of the 10th Brazilian Symposium on Human Factors in Computing Systems and the 5th Latin American Conference on Human-Computer Interaction, Porto de Galinhas, Pernambuco, Brazil, pp. 272–279 (2011)Google Scholar
  35. 35.
    Kitchenham, B.A.: DESMET: A method for evaluating Software Engineering methods and tools. Technical Report, University of Keele (1996)Google Scholar
  36. 36.
    Kitchenham, B.A.: Evaluating software engineering methods and tool part 1: the evaluation context and evaluation methods. SIGSOFT Softw. Eng. Notes 21(1), 11–14 (1996)CrossRefGoogle Scholar
  37. 37.
    Tollefsen, M.: Web og Universell Utforming, 63. Universitetsforlaget, Oslo (2013)Google Scholar
  38. 38.
    Agency for Public Management and eGovernment (Difi), WCAG 2.0 standard. (29 Sept 2015)
  39. 39.
    Universell, Sju prinsipper for universell utforming. (05 Nov 2015)
  40. 40.
    Leedy, P.D., Ormrod, J.E.: Practical Research Planning and Design, 10th edn. Pearson Education Limited, Essex (2014)Google Scholar
  41. 41.
    Thagaard, T.: Systematikk og innlevelse: en innføring i kvalitativ metode, 4th edn. Fagbokforlaget, Bergen (2013)Google Scholar
  42. 42.
    Creswell, J.W., Miller, D.L.: Determining validity in qualitative inquiry. Theory Pract. 39(3), 124–130 (2000)CrossRefGoogle Scholar
  43. 43.
    Nes, M.E.S.: Appraising and Evaluating the Use of DAISY: For Print Disabled Students in Norwegian Primary- and Secondary Education. University of Oslo, Oslo (2007)Google Scholar
  44. 44.
    Nes, M.E.S., Ribu, K., Tollefsen, M.: DAISY—Universally designed? Prototyping an approach to measuring universal design. In: Proceedings ICCHP, Linz, Austria, Lecture Notes in Computer Science, Computers Helping People with Special Needs, vol. 5105, pp. 268–275 (2008)Google Scholar
  45. 45.
    WebAIM web accessibility in mind, Screen Reader User Survey #6 Results. (1 Nov 2016)
  46. 46.
    UNINETT: Breakthrough in Norway: paperless exams. (27 Oct 2016)
  47. 47.
    UNINETT: Digital assessment leap: Norwegian research and higher education sector concludes on three ICT systems. (27 Oct 2016)
  48. 48.
    UNINETT: Kravspesifikasjon for digital eksamen. (10 Nov 2016)
  49. 49.
    WebAIM web accessibility in mind, Screen Reader User Survey #5 Results. (1 Nov 2016)
  50. 50.
    Guenaga, M.L., Burger, D., Olivier, J.: Accessibility for e-Learning Environments. In: Proceedings ICCHP, Paris, Lecture Notes in Computer Science, Computers Helping People with Special Needs, vol. 3118, pp. 157–163 (2004)Google Scholar
  51. 51.
    Hansen, E.G., Mislevyb, R.J., Steinbergc, L.S., Leed, M.J., Forere, D.C.: Accessibility of tests for individuals with disabilities within a validity framework. System 33(1), 107–133 (2005)CrossRefGoogle Scholar
  52. 52.
    Eidgenössische Technische Hochschule (ETH) Zurich, Educational Development and Technology (LET), Safe Exam Browser, About—Overview, Concept. (1 Nov 2016)
  53. 53.
    Eidgenössische Technische Hochschule (ETH) Zurich, Educational Development and Technology (LET), Safe Exam Browser, Windows - User Manual, Safe Exam Browser 2.1.3 for Windows. (1 Nov 2016)
  54. 54.
    Inspera, 1: Støttes nettleseren og operativsystemet ditt? (1 Nov 2016)
  55. 55.
    Ministry of Education and Research: Forskrift om egenbetaling ved universiteter og høyskoler, FOR-2005-12-15-1506. (1 Nov 2016)

Copyright information

© Springer-Verlag Berlin Heidelberg 2017

Authors and Affiliations

  • Miriam Eileen Nes Begnum
    • 1
    • 2
  • Rikke Julie Foss-Pedersen
    • 3
  1. 1.Department of Design, Faculty of Architecture and DesignNTNU Norwegian University of Science and TechnologyGjøvikNorway
  2. 2.Department of Computer Science, Faculty of Information Technology and Electrical EngineeringNTNU Norwegian University of Science and TechnologyGjøvikNorway
  3. 3.UiO University of Oslo, USIT University Center for Information TechnologyOsloNorway

Personalised recommendations