The first result of our (SLR) is that only 31 studies of 746 met our selection criteria. The bibliographic details of the 31 primary studies analyzed in this SLR are presented in “Appendix 1.” In what follows, the trends of primary studies are presented, followed by the main findings.
Trends of primary studies
Of all primary studies, 75% were published in scientific journals, while 25% were presented in high impact conferences. Figure 5 depicts the distribution of studies by year of publication. As can be seen, during the past few years, there appears to be a slow but growing interest in this topic. The peak year in terms of the number of papers was 2018 in which seven papers were published. The period 2016 to 2018 seems to be a period during which the local legislations established compliance with regulations about accessibility. Research about this accessibility metadata in 2019 is still sparse. On the other hand, the primary studies consisted of 24 journal articles and seven conference papers.
Table 1 shows the number of publications by type of research according to . Most of the primary studies (71%, 20) were “Proposal of solution” (12) and “Evaluation research” (10), followed by “Personal experience paper” (6), and “Validation research” (3).
Finally, Table 2 shows a summary of primary studies grouped by research questions that were answered after full text reading. Therefore, the findings were grouped into four categories in order to answer the research questions.
RQ1: To what extent do standards and specifications in e-learning include accessibility metadata?
To answer this question, it is necessary to analyze relevant previous studies on accessibility and metadata. In this way, it is possible to understand how standards and specifications in e-learning include accessibility metadata. Figure 6 shows a timeline of metadata and accessibility that allows to better understand what standards and specifications include accessibility metadata. IMS Global and ISO are the organizations that lead this subject [30, 46]. From 2003 to 2004, guidelines to support the accessibility requirements appeared [47, 36]. Thus, information profiles for IMS LIP students and its information packet for ACCLIP students were published. In 2004, a more extensive proposal called AfA 1.0 with access metadata for all (ACCMD) was proposed.
In 2008, ISO published the ISO/IEC 24751 standard. In particular, the second and the third part of it are related to this review. The second part includes metadata for preferences and user needs (PNP) [12, 48], and the third part includes the description of the resources (DRD) [49, 50]. However, those initiatives were still complex to apply in practice. In 2009, AfA 2.0 was published to provide technical information that allows the implementation of accessibility metadata. There were few efforts and initiatives of applicability of metadata that support the rules created to date. Then, in 2011, Google, Yahoo and Bing agreed to use a Schema.org specification as a common vocabulary . By 2012, AfA 3.0 was proposed. This effort was made to reduce data model and organization of AfA 2.0 and to include other metadata [33, 34, 37, 51, 52]. Moreover, refinement concepts such as the selection of more than one value per metadata and the possibility of multiple adaptation requests were included. However, it is in 2014 that the accessibility metadata project (AMP) achieved a subset of accessibility metadata for search and discovery. AMP was carried out in collaboration with IMS Global along with the Learning Resource Metadata Initiative (LRMI) and the Gates Foundation. Finally, a metadata proposal called A11y was generated within Schema.org . In , researchers summarize that the metadata of greater use and related to accessibility would be accessibilityControl (input methods with which the resource can be controlled), accessibilityFeature (accessibility features present in the resource), and accessMode (sensory access form- human perceptive). The above-mentioned metadata and its characteristics can be found on the Schema website .
RQ2: Could accessibility and adaptability in e-learning be evaluated through metadata?
Contributions in regulations that guide the process of implementing metadata in accessibility have been valuable. Rodriguez-Ascaso and González Boticario  pointed out the importance of ISO/IEC 24751 in describing the student’s accessibility needs, as well as the digital resource in the guidelines of their research. Moreover,  and  justify their research with AfA 3.0 while [49, 51] base their respective proposals on the WCAG standard to suggest success criteria related to learning objects. Navarrete and Luján-Mora  based the evaluation of the accessibility gap in Open Educational Resources on WCAG 2.0 (identified as ISO/IEC 40500), ISO 9241-151: Guide on the user interfaces of the World Wide Web, and AfA 3.0. The different methodologies used in these studies reveal efforts to understand and apply the standards established by ISO and IMS. Adaptability, self-learning, self-organization [52, 56], personalization  are considered relevant to establish metadata that strengthen the search for the best educational resource for each student. In this sense, the EU4ALL project (IST-FP6-034778) developed a framework to address the needs of accessible lifelong learning at Higher Education level . In fact, this project was the main reference for building the new IMS Access for All v3.0, i.e., it contributed to the development of the Accessibility Metadata Standards. An Educational Metadata Profile is proposed by  to characterize digital educational resources based on IEEE LOM, concluding that it is rich enough to effectively describe both the educational and technical aspects of an educational resource. The effective use of metadata demonstrates its importance in case studies such as screen readers . In addition,  points out that having a repository of learning objects with full metadata could support the adoption of accessibility metadata systems. The generation of LOM editors facilitates the implementation and storage of metadata in the resource, with LOMPad being a freely, and widely user editor. However, accessibility metadata are not yet considered in that editor. The quality of data and metadata is still a rare practice of implementation since it involves subjective, multi-dimensional and context-dependent concepts . Quantitative research on the use of accessibility metadata and limited knowledge in its implementation are addressed in the research carried out by . They conclude that the most used are those that respond to general properties in terms of the pedagogical proposal. Regarding accessibility metadata itself, a single educational domain (www.bookshare.org) was found. Such domain includes accessibility features and the license of resources as a whole . Therefore, it can be concluded that it is possible to evaluate the accessibility and adaptability in e-learning by the information of the metadata, which would provide valuable information of accessible resources for learners. Table 4 shows a summary of specifications and standards that consider the accessibility in the evaluation (Table 3).
RQ3: Does accessibility metadata have any positive impact on the preferences and needs of a student with disabilities?
Research that specifies a real sample of students with disabilities and their endorsement in proposed methodologies that provide solutions to the accessibility problem is still limited. Rodriguez-Ascaso and González Boticario  proposed a user-centered design methodology that allows the identification, through scenarios, of found problems and existing challenges. This research is based on the needs identified in a process of collecting requirements of users with and without disabilities in higher education institutions within the project EU4ALL. On the other hand,  identifies virtual scenarios of social learning, designs recommendations aimed at meaningful education. It also considers the generality for reuse in other scenarios with similar characteristics. Batanero et al.  conducted a study to establish the mandatory accessibility metadata that should be included in a learning object, considering the preferences and needs of the student (AfA PNP):
AccessMode, HasAdaptation, ControlFlexibility, DisplayTransformability, Hazard, AtInteroperable, ApiInteroperable, IsAdaptationOf, IsPartialAdaptationOf, IsFullAdaptationOf, AdaptationType, AccessModeAdapted, AdaptationDetail, AdaptationMediaType, LanguageOfAdaptation, EducationalComplexityOfAdaptation, EucationaLevelOfAdaptation.
This study is based on the interaction of underground workers, with a program that provides them with information about the type of fault or zone map. The scenario in this case is low visibility and/or high noise so they are provided with adapted Learning Object (LO), having the possibility of choosing the profile, considering sign language, animated diagrams or Braille device. Rodriguez-Ascaso et al.  emphasizes the needs of Accessible Lifelong Learning. It details the interaction of a team that includes five stakeholders: (1) teachers trained in design of accessible material and its monitoring, (2) students expressing their accessibility needs, (3) disability officers assessing needs, (4) transformation officers working on the adaptation of materials and (5) librarians to support the labeling and handling of learning materials in electronic repositories. Lin et al.  establishes the need for segmentation and annotation strategies in e-learning domains through metadata. Annotation is the basis for the accessibility and reuse of resources to search and detection of micro-learning. Koutkias et al.  proposes a structure that addresses student preferences and generates an environment to interact framed in universal design and inclusion. Pal et al.  identifies a subset of educational metadata, from IEEE LOM, for video-based e-learning materials considering that appropriate choices can be made based on the student’s learning requirements, preferences, and pedagogy choice. Navarrete and Luján-Mora  proposed the identification of the user through login and defining a disability profile along with accessibility options for the search. They also established the need for a custom adaptive interface design considering the language. The study carried out by  was based on students with functional diversity. The authors concluded that a simple design allows an efficient implementation of the adapted learning platform and easy portability to other platforms for learning and storage of LO. Kearns  identifies the problems of online courses with screen readers and recommends solutions with the effective use of metadata for a better understanding of the course material. Vizoso  refers to the ESVIAL Project Guide as a model of transformation and proposal of good practices in accessibility, based on the participation and collaborative construction of adapted educational resources and the needs of the student. Batanero et al. [30, 52] propose the implementation of an adaptation in the Moodle platform. The participants were 10 blind, 10 deaf and 3 deaf-blind students whose age ranged between 26 and 50 years. The study describes adaptations to students in Moodle based on their preferences and the incorporation of accessible resources with the possibility of reuse. Besides, the authors emphasize that audio descriptions of the secondary elements should be carefully limited to avoid problems with other disabilities. Sanchez-Gordon and Luján-Mora  established the need for further research in the specification of online and off-line accessibility requirements. As an example, the authors described specific cases of Chemistry learning requirements, their experimentation, reports, and discussion forums. In response to RQ3, it can be said that there is empirical evidence related to the impact of metadata on the preferences and needs of students with disabilities. However, it is necessary to consider the continuous and active attention to the needs of functional diversity that may arise in the educational context. In this sense, the implementation of accessibility metadata would be a fundamental contribution to the generation of repositories, as well as its constant feedback on the experience of the student with disabilities and the various scenarios that exist in the educational process.
RQ4: What are the challenges and opportunities that have been addressed in this area of research?
In general, some research points out the lack of tools and systems in virtual learning environments for the application of accessibility metadata. In this topic,  established the lack of reviews related to the issue of accessibility by the different educational platforms. However, it is worth noting the existence of literature on the analysis of limited scope on some courses. Batanero et al.  recommended the application of standards based on the general analysis of web accessibility and the level of compliance with the WCAG standard. The understanding of the different standards in the area of accessibility and adaptability leads to future studies focused on developing evaluation mechanisms, automatic tools, and methodological proposals in order to contribute to the development of accessibility in e-learning. It is important to measure the impact on the user experience  considering the use of assistive technology . Another great challenge is to provide students with advanced and personalized services to efficiently manage and disseminate educational material . The difficulty of effectively interpreting the content of educational resources that facilitates personalization constitutes a research challenge. Lin et al. . It is worth building systems with intuitive adaptations that facilitate the delivery and selection of educational resources for students with disabilities [30, 46]. In addition, the generation of instructional design methodologies is important because it supports the management and dissemination of educational material according to specific needs . The development of tools that strengthen the applicability of accessibility metadata is also required. There exist frameworks based on semantic rules that facilitate the self-personalization of assistive technologies . However, other mechanisms are necessary for the recommendation of learning objects, e.g., self-organization strategies  and self-control . Recommender systems are also needed to refine the modeling of user profiles in order to establish accessibility requirements for courses and develop a holistic approach , or based on the learning profiles of other students with similar needs or preferences .
The establishment of good accessibility practices in a teaching-learning process is time consuming, since it is necessary to establish pre- and post-comparisons. Rodriguez-Ascaso et al.  present projects of 48 months to address the needs of Accessible Lifelong Learning, as the EU4ALL project that finally lasted 54 months. Additionally, the complexity to design adapted applications is determined due to the wide range of characteristics in the functional diversity of students and the lack of reliable specific data . Considering that metadata come from multiple and heterogeneous sources, metadata are compiled with different approaches and used in different contexts . However, the ability to discover resources that do not adjust to a common standard is difficult [37, 67]. Pal et al.  determines the need to establish a generic model that leads to the use of a universal ontology for all educational domains, which could categorize all the metadata of different topics and subjects. However, there is a lack of a methodological framework for the implementation of an accessible virtual educational project. Therefore, it is necessary to incorporate metrics based on accessibility indicators that facilitate the evaluation of the results obtained in the different phases . The involvement of teachers in the use and creation of educational resources, their dynamic characteristics and organizational structures are also necessary . The sustainability and scalability of an educational resource depend on the socialization of the optimal use of repositories and reuse of resources according to the needs of the teaching-learning process. The quality of a resource, including its metadata and accessibility information, represents a broad topic that needs to be integrated . Although some parameters to measure quality are in competition with each other, both peer review tools and user evaluation tools are methods to evaluate the quality of resources. Thus,  suggests three dimensions of analysis: priority, possibility of achieving (responsibility) and sustainability. Although the use of HTML5 is expected to increase in some areas including education , the implementation of accessibility metadata in educational resources is still incipient. According to , a mapping with eight dimensions of research on the issue of accessibility and MOOCs is proposed, establishing important guidelines in future research. In addition, more research is needed on automatic procedures and policy measures to support and monitor learning activities that involve a massive number of students , also considering other areas of knowledge such as exact sciences and required adaptations . Future search processes could be facilitated through metadata editors capable of interpreting and reading files generated by various tools . For example,  presents a conceptual framework for the automated generation of metadata that highlights the importance of adaptive e-learning process based on the learning activity profile. It is also necessary to have automatic support tools that facilitate the use of accessibility metadata  in order to detect drawbacks and successes and propose alternatives in order to increase accessible educational resources in various repositories. In response to RQ4, it can be said that the field of accessibility is extensive, so the techniques and associated standards need to be homogenized and socialized to boost their use among e-learning resource developers. The use of metadata needs to be extended to accessible educational resources, in such a way that the learning curve can be reduced. Additionally, a common language in the development of accessible digital resources with easy implementation and search on the web should be developed.
This systematic review presented some limitations during the process of answering the research questions. The selection of the research keywords and exclusion criteria may limit relevant searches, as may the language. The systematic review sought to identify a theoretical context, so it was based on databases of scientific literature. Gray literature is not included. The omission of articles may also respond to the selected period time 2012–2019. The study was carried out since 2012 because big corporations as Google, Yahoo and Bing started to use schema.org in 2011. Since then, it called the attention of practitioners and researchers. Bias was reduced by choosing a set of databases that cover the main disciplinary fields in which accessibility and metadata can be addressed (computer science and education). To minimize bias, a systematic process was defined to perform the data extraction. In fact, an optimum level of reliability (86.1%) among researchers was determined by calculating a coefficient of krippendorff’s alpha. Another limitation is the exclusion of documents written in another language (than English or Spanish) and having less than 4 pages, so it is possible that current research status in other countries is missing. Even so, the systematic review process is considered to offer a good overview of the metadata and accessibility research status, identifying the relationships between the evaluation of e-learning environments and accessibility metadata.