Skip to main content
SpringerLink
Log in

Evaluating e-learning accessibility by automated and student-centered methods

  • Development Article
  • Published:
Educational Technology Research and Development Aims and scope Submit manuscript

Abstract

The use of learning technologies is becoming ubiquitous in higher education. As a result, there is a pressing need to develop methods to evaluate their accessibility to ensure that students do not encounter barriers to accessibility while engaging in e-learning. In this study, sample online units were evaluated for accessibility by automated tools and by student participants (in sessions moderated and unmoderated by researchers), and the data from these different methods of e-learning accessibility evaluation were compared. Nearly all students were observed encountering one or more barriers to accessibility while completing the online units, though the automated tools did not predict these barriers and instead predicted potential barriers that were not relevant to the study participants. These data underscore the need to carry out student-centered accessibility evaluation in addition to relying on automated tools and accessibility guideline conformance as measures of accessibility. Students preferred to participate in unmoderated sessions, and the data from the unmoderated sessions were comparable to that from the more traditional moderated sessions. Additional work is needed to further explore methods of student-centered evaluation, including different variations of unmoderated sessions.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Fig. 1

Similar content being viewed by others

References

  • Americans with Disabilities Act. (1990, 2008). ADA. Retrieved from http://www.ada.gov/pubs/ada.htm.

  • Andreason, M. S., Villemann Nielsen, H., Ormholt Schroeder, S., & Stage, J. (2007). What happened to remote usability testing? An empirical study of three methods. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1405–1414). New York, NY: ACM.

  • AODA Integrated Accessibility Standards. (2011). Retrieved from the Service Canada website http://www.e-laws.gov.on.ca/html/source/regs/english/2011/elaws_src_regs_r11191_e.htm#BK15.

  • Asuncion, J. V., Fichten, C. S., Ferraro, V., Chwojka, C., Barile, M., Nguyen, M. N., et al. (2010). Multiple perspectives on the accessibility of e-learning in Canadian colleges and universities. Assistive Technology, 22(4), 187–199.

    Article  Google Scholar 

  • Baravalle, A., & Lanfranchi, V. (2003). Remote web usability testing. Behavior Research Methods, Instruments, & Computers, 35(3), 364–368.

    Article  Google Scholar 

  • Bland, M. (2000). Multiple significance tests and the Bonferroni correction. An introduction to medical statistics (3rd ed.). New York, NY: Oxford Medical Publications.

    Google Scholar 

  • Bohman, P. R., & Anderson, S. (2005). A conceptual framework for accessibility tools to benefit users with cognitive disabilities. Paper presented at the W4A, Chiba, Japan.

  • Bolt, N., & Tulathimutte, T. (2010). Remote research. Brooklyn, NY: Rosenfeld Media.

    Google Scholar 

  • Boren, T., & Ramey, J. (2000). Thinking aloud: Reconciling theory and practice. IEEE Transactions on Professional Communications, 43(3), 261–278.

    Article  Google Scholar 

  • Carnevale, D. (2005). Lawsuit charges online university does not accommodate learning-disabled students. The Chronicle of Higher Education, 51(49), 33.

    Google Scholar 

  • Cassidy, S. (2004). Learning styles: An overview of theories, models, and measures. Educational Psychology, 24(4), 419–444.

    Article  Google Scholar 

  • Chen, Y. T. (2014). A study to explore the effects of self-regulated learning environment for hearing-impaired students. Journal of Computer Assisted learning, 30(2), 97–109.

    Article  Google Scholar 

  • Clark, J. (2006, May 23). To hell with the WCAG2. A List Apart, 217. Retrieved from http://www.alistapart.com/articles/tohellwithwcag2.

  • Denton, W., & Coysh, S. J. (2011). Usability testing of VuFind at an academic library. Library Hi Tech, 29(2), 301–319.

    Article  Google Scholar 

  • Dermody, K., & Majekodunmi, N. (2010). Online databases and the research experience for university students with print disabilities. Library Hi Tech, 29(1), 149–160.

    Article  Google Scholar 

  • Ellcessor, E. (2010). Bridging disability divides: A critical history of web content accessibility through 2001. Information, Communication & Society, 13(3), 289–308.

    Article  Google Scholar 

  • Evett, L., & Brown, D. (2005). Text formats and web design for visually impaired and dyslexic readers—clear text for all. Interacting with Computers, 17(4), 453–472.

    Article  Google Scholar 

  • Fichten, C. S., Ferraro, V., Asuncion, J. V., Chwojka, C., Barile, M., Nguyen, M. N., et al. (2009). Disabilities and e-learning problems and solutions: An exploratory study. Educational Technology & Society, 12(4), 241–256.

    Google Scholar 

  • Foley, A. (2011). Exploring the design, development and use of websites through accessibility and usability studies. Journal of Educational Multimedia and Hypermedia, 20(4), 361–385.

    Google Scholar 

  • Friedman, M. G., & Bryen, D. N. (2007). Web accessibility design recommendations for people with cognitive disabilities. Technology and Disability, 19(4), 205–212.

    Google Scholar 

  • Gerrard, C. (2007). Virtual learning environments: Enhancing the learning experience for students with disabilities. Campus-Wide Information Systems, 24(3), 199–206.

    Article  Google Scholar 

  • Goggin, G., & Newell, C. (2003). Digital disability: The social construction of disability in new media. New York, NY: Rowman & Littlefield.

    Google Scholar 

  • Goodall, B. (2008, January 3). Judge finds university didn’t discriminate online. Retrieved from http://chronicle.com/blogs/wiredcampus/judge-finds-university-didnt-discriminate-online/3582.

  • Grabinger, S. (2010). A framework for supporting postsecondary learners with psychiatric disabilities in online environments. Electronic Journal of e-Learning, 8(2), 101–110.

    Google Scholar 

  • Habib, L., Berget, G., Sandnes, F. E., Sanderson, N., Kahn, P., Fagernes, S., et al. (2012). Dyslexic students in higher education and virtual learning environments: an exploratory study. Journal of Computer Assisted learning, 28(6), 574–584.

    Article  Google Scholar 

  • Hartson, H. R., Castillo, J. C., Kelso, J., & Neale, W. (1996). Remote evaluation: The network as an extension of the usability laboratory. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 228–235). New York, NY: ACM.

  • Henry, S. L. (2007). Just ask: Integrating accessibility throughout design. Retrieved from http://www.uiaccess.com/JustAsk/index.html.

  • Her Majesty’s Stationary Office. (2001). The Special Educational Needs and Disability Act 2001, London. Retrieved from http://www.legislation.gov.uk/ukpga/2001/10/contents.

  • Her Majesty’s Stationary Office. (2010). Equality Act 2010, London. Retrieved from http://www.legislation.gov.uk/ukpga/2010/15/section/92.

  • Houck-Whitaker, J. (2005). Remote testing versus lab testing. Retrieved from http://www.boltpeters.com/articles/versus.html.

  • Inside Higher Ed. (2014, February 17). Portland State settles complaint by deaf student. Retrieved from http://www.insidehighered.com/quicktakes/2014/02/17/portland-state-settles-complaint-deaf-student#sthash.YQTpkfRK.dpbs.

  • Joint Department of Justice and Department of Education. (2011). Frequently asked questions about the June 29, 2010, Dear Colleagues letter. Retrieved from http://www2.ed.gov/about/offices/list/ocr/docs/dcl-ebook-faq-201105.html.

  • Jung, S., Herlocker, J. L., Webster, J., Mellinger, M., & Frumkin, J. (2008). LibraryFind: System design and usability testing of academic metasearch system. Journal of the American Society for Information Science and Technology, 59(3), 375–389.

    Article  Google Scholar 

  • Kapsi, M., Vlachogiannis, E., Darzentas, J., & Spyrou, T. (2009a). A preliminary feedback for the WCAG 2.0-WCAG 1.0 Vs WCAG 2.0 evaluation study. Paper presented at the 2nd annual conference on pervasive technologies related to assistive environments (PETRA), Confu, Greece.

  • Kapsi, M., Vlachogiannis, E., Darzentas, J., & Spyrou, T. (2009b). The usability of web accessibility guidelines: An approach for evaluation. In Proceedings of the 5th international conference on universal access in human computer interaction (pp. 716–724). Berlin: Springer.

  • Kelly, B., Phipps, L., & Swift, E. (2004). Developing a holistic approach for e-learning accessibility. Canadian Journal of Learning and Technology, 30(3), 20.

    Google Scholar 

  • Kennedy, H., Evans, S., & Thomas, S. (2011). Can the web be made accessible for people with intellectual disabilities? The Information Society, 27(1), 29–39.

    Article  Google Scholar 

  • Kumar, K., & Owston, R. (2012). Learnings from post-secondary students on e-learning accessibility and accessibility evaluation methods. In Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education 2012 (pp. 663–669). Chesapeake, VA: AACE.

  • Laff, M., & Rissenberg, M. (2007). Cognitive ability measures for accessible web content. Paper presented at the 4th international conference on universal access in human computer interaction (HCI), Beijing, China.

  • Marks, D. (1997). Models of disability. Disability and Rehabilitation, 19(3), 85–91.

    Article  Google Scholar 

  • McCarthy, J. E., & Swierenga, S. J. (2010). What we know about dyslexia and web accessibility: A research review. Universal Access in the Information Society, 9(2), 147–152.

    Article  Google Scholar 

  • McFadden, E., Hager, D. R., Elie, C. J., & Blackwell, J. M. (2002). Remote usability evaluation: Overview and case studies. International Journal of Human-Computer Interaction, 14(3&4), 489–502.

    Article  Google Scholar 

  • Narasimhan, N. (Ed.). (2012). Web accessibility policy making: An international perspective. New York: Hemkunt Publishers Ltd.

    Google Scholar 

  • National Federation of the Blind. (2012, March 6). Florida State resolves litigation with students [Press release]. Retrieved from https://nfb.org/node/931.

  • National Federation of the Blind. (2014, January 13). Blind student files discrimination suit against Miami University [Press release]. Retrieved from https://nfb.org/blind-student-files-discrimination-suit-against-miami-university.

  • Nicolle, C., & Paulson, D. (2004). Making the Internet accessible by people with cognitive and communication impairments. Universal Access in the Information Society, 3(1), 48–56.

    Article  Google Scholar 

  • Nielsen, J. (1993). Usability engineering. Boston, MA: AP Professional.

    Google Scholar 

  • Oliver, M. (1996). Understanding disability: From theory to practice. Houndmills: Macmillan.

    Book  Google Scholar 

  • Olmsted-Hawala, E. L., Murphy, E. D., Hawala, S., & Ashenfelter, K. T. (2010a). Think-aloud protocols: A comparison of three think-aloud protocols for use in testing data-dissemination web sites for usability. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 2381–2390). New York, NY: ACM.

  • Olmsted-Hawala, E. L., Murphy, E. D., Hawala, S., & Ashenfelter, K. T. (2010b). Think-aloud protocols: Analyzing three different think-aloud protocols with counts of verbalized frustrations in a usability study of an information-rich Web site. In Proceedings of the SIGCHI Conference on Human Factors in Computing Systems (pp. 60–66). New York, NY: ACM.

  • Petrie, H., Hamilton, F., King, N., & Pavan, P. (2006). Remote usability evaluations with disabled people. In Proceedings of the SIGCHI conference on human factors in computing systems (pp. 1133–1141). New York, NY: ACM.

  • Power, C., Petrie, H., & Mitchell, R. (2009). A framework for remote user evaluation of accessibility and usability of websites. In C. Stephanidis (Ed.), Universal access in human computer interaction. Addressing diversity (Vol. 5614, pp. 594–601)., Lecture Notes in Computer Science Berlin, Germany: Springer-Verlag.

    Chapter  Google Scholar 

  • Power, C., Petrie, H., Sakharov, V., & Swallow, D. (2010). Virtual learning environments: Another barrier to blended and e-learning. In Proceedings of the 12th international conference on computers helping people with special needs: part I (pp. 519–526). Berlin: Springer.

  • Pretorius, M., & van Biljon, J. (2010). Learning management systems: ICT skills, usability and learnability. Interactive Technology and Smart Education, 7(1), 30–43.

    Article  Google Scholar 

  • Rehabilitation Act. (1998). Rehabilitation Act of 1973. Retrieved from http://www2.ed.gov/policy/speced/reg/narrative.html.

  • Ribera, M., Porras, M., Boldu, M., Termens, M., Sule, A., & Paris, P. (2009). Web Content Accessibility Guidelines 2.0: A further step towards accessible digital information. Program: Electronic Library and Information Systems, 43(4), 392–406.

    Article  Google Scholar 

  • Roberts, J. B., Crittenden, L. A., & Crittenden, J. C. (2011). Students with disabilities and online learning: A cross-institutional study of perceived satisfaction with accessibility compliance and services. Internet and Higher Education, 14(4), 242–250.

    Article  Google Scholar 

  • Rubin, J., & Chisnell, D. (2008). Handbook of usability testing. Indianapolis, IN: Wiley.

    Google Scholar 

  • Seale, J. K. (2003). The challenge of researching accessibility practices within Higher Education: An exploration of “shared enterprises” or “political games”. Paper presented at the International Education Research Conference AARE and NZARE.

  • Seeman, L. (2006). Re: Formal objection to WCAG 2.0. [Electronic mailing list message]. Retrieved from http://lists.w3.org/Archives/Public/w3c-wai-gl/2006AprJun/0368.html.

  • Selvaraj, P. (2004). Comparative study of synchronous remote and traditional in-lab usability evaluation methods. Unpublished Unpublished Master’s thesis, Virginia Polytechnic Institute and State University.

  • Sloan, D., & Kelly, B. (2008). Reflections on the development of a holistic approach to web accessibility. Paper presented at the 2008 accessible design in the digital world conference (ADDW), York.

  • Thatcher, J., Burks, M. R., Heilmann, C., Henry, S. L., Kirkpatrick, A., Lauke, P. H., et al. (2006). Web accessibility: Web standards and regulatory compliance. Retrieved from http://library.books24x7.com.ezproxy.library.yorku.ca/assetviewer.aspx?bookid=14675&chunkid=335181448&rowid=1352&noteMenuToggle=0&leftMenuState=1.

  • Thompson, T., Comden, D., Ferguson, S., Burgstahler, S., & Moore, E. (2013). Seeking predictors of web accessibility in U.S. higher education institutions. Information Technology and Disabilities, 13(1), 18.

    Google Scholar 

  • U.S. Department of Justice. (2010a). Letter of Resolution, D.J. No. 202-48-213 Princeton University. Retrieved from http://www.ada.gov/princeton.htm.

  • U.S. Department of Justice. (2010b). Settlement agreement between the United States of America, the National Federation of the Blind, Inc. (“NFB”), and the American Council of the Blind (“ACB”), and the Arizona Board of Regents (“ABOR”), for and on behalf of Arizona State University (“ASU”). Retrieved from http://www.ada.gov/arizona_state_university.htm.

  • U.S. Department of Justice. (2013, July 23). Justice department settles with Louisiana Tech University over inaccessible course materials [Press release]. Retrieved from http://www.justice.gov/opa/pr/2013/July/13-crt-831.html.

  • Ustun, T. B., Chatterji, S., Bickenbach, J., Kostanjsek, N., & Schneider, M. (2003). The international classification for functioning, disability and health: A new tool for understanding disability and health. Disability and Rehabilitation, 25(11–12), 565–571.

    Article  Google Scholar 

  • Vanderheiden, G. (2000). Fundamental principles and priority setting for universal usability. In Proceedings of the 2000 conference on universal usability (pp. 32–38). New York, NY: ACM.

  • Vigo, M., & Brajnik, G. (2011). Automatic web accessibility metrics: Where we are and where we can go. Interacting with Computers, 23(2), 137–155.

    Article  Google Scholar 

  • W3C. (2008). Web Content Accessibility Guidelines 2.0: W3C Recommendation 11 December 2008. Retrieved from http://www.w3.org/TR/WCAG20/.

  • Woodfine, B. P., Nunes, M. B., & Wright, D. J. (2008). Text-based synchronous e-learning and dyslexia: Not necessarily the perfect match! Computers & Education, 50(3), 703–717.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Manitoba, Extended Education, 166 Extended Education Complex, Winnipeg, MB, R3T 2N2, Canada

    Kari L. Kumar

  2. Faculty of Education, Winters College, York University, 4700 Keele Street, Toronto, ON, M3J 1P3, Canada

    Ron Owston

Authors
  1. Kari L. Kumar
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ron Owston
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kari L. Kumar.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Kumar, K.L., Owston, R. Evaluating e-learning accessibility by automated and student-centered methods. Education Tech Research Dev 64, 263–283 (2016). https://doi.org/10.1007/s11423-015-9413-6

Download citation

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11423-015-9413-6

Keywords

Search

Navigation