Quality Assurance in Online, Open, and Distance Education

Abstract

Over the past few decades, online, open, and distance education (ODE) has enjoyed phenomenal growth across different regions, and with the spread of the COVID-19 virus, its use has adopted more quickly and widely at all levels of education in both developed and developing countries. There has also been a surge in trans-institutional online courses and programs. Despite this surge in widespread practice of ODE, an image problem that perceives ODE as a second-rate education, indicated by Daniel (Quality assurance and accreditation in distance education and e-learning: Models, policies, and research. Routledge, New York, 2011) a decade ago, still exists, meaning that there is a need for even stronger measures to ensure that ODE is as good as in-person education and that quality assurance (QA) systems are in place as it enters the mainstream of education.

Applying QA to ODE processes and outcomes is a relatively new phenomenon. Drawing upon previous literature on quality and QA in ODE, including institutional case studies, this chapter examines various definitions of quality and QA in ODE practice, difficulties, and challenges presented by QA in ODE and the role of stakeholders in QA in ODE. It then discusses how different ODE institutions judge and assure the quality of their courses, programs, and services and what values and assumptions underpin such QA approaches. It finally draws conclusions about what still needs to be considered and actioned.

Introduction: Defining Quality and Quality Assurance in ODE

Harvey and Green (1993) view quality as excellence (something exceptional and distinctive), as perfection (something perfect or consistent), as fitness for purpose (something fulfilling needs or requirements), as value for money (something worth investing), and as transformative (something empowering or enhancing). These five disparate ways of explaining quality in higher education definitely help conceptualize quality as being five interrelated ideas in ODE, but at the same time they make it difficult to univocally and explicitly define the concept of quality in relation to ODE.

Quality, moreover, in ODE may be judged in different ways depending on who defines it (Latchem, 2016). Students, governments, or employers, for example, who pay for ODE may define quality of ODE in terms of value for money, while educators might tend to focus on excellence and consistency, and researchers may highlight transformative aspects of ODE. Likewise, QA may be understood quite differently by different stakeholders as Jung and Latchem (2007) argued. Governments may be interested in the socioeconomic benefits and public accountability of ODE (Koul, 2006), while ODE institutions seek assurance in quality across planning and management, design, development and delivery of course and course materials, learner support, and assessment and completion/graduation rates (Jung, 2005). Researchers may highlight the depth and extent of learning (Ehlers, 2004) and the development of self-directed lifelong learning skills as an important aspect of QA in ODE (Paul, 1990). Students are more concerned with well-designed ODE courses and materials, support, and logistics (Daniel, Kanwar, & Uvalić-Trumbić, 2008), the applicability or relevance of assignments (Conrad, 2002), clear instructions especially related to grading policy and feedback (Song, Singleton, Hill, & Koh, 2004), and ensuring their competitiveness in the workplace (Kihwelo, 2013).

While quality is perceived as a value to be pursued for the enhancement of higher education, not everyone concurs that QA is a good thing. Seyfried and Pohlenz (2018) argue that internal and external reviews within a QA system are not always reliable and valid in evaluating the quality of teaching and learning and thus may not be a strong basis for important management decisions concerning quality enhancement. Beerkens (2018) claims that the majority of QA policies have not focused on student learning and thus it is still unknown whether all QA efforts and reforms have resulted in better learning performances of graduates.

Considering the complexity involved in defining quality and QA in ODE, Jung and Latchem (2007, 2011) categorize the varying definitions of quality in five ways:

• Quality as conforming to the standards set for conventional education. Here, the same criteria and standards are used to judge quality in ODE and conventional campus-based institutions’ teaching and learning, management, resources, and outcomes. This approach, however, may fail to take into account unique aspects of ODE (Phipps & Merisotis, 1999; Stella & Gnanam, 2004), but as ODE is fast becoming a main delivery mode in conventional education, the difference between conventional campus-based education and ODE in higher education may not be as big as it used to be.

• Quality as fitness for purpose. Here, quality is judged by the extent to which ODE programs or institutions fulfill their predetermined purposes. But the problem is that this approach may fail to address conflicting purposes of ODE as defined by the various stakeholders discussed earlier.

• Quality as meeting customers’ needs. Derived from business sectors, this approach values customer satisfaction. Some ODE institutions have adopted ISO9001 and implement course satisfaction surveys to assess the quality of their courses and services (Aisyah, Samsiyah, Wulandari, & Juliana, 2019). One problem with this approach is that it may fail to address varying needs of different customer groups.

• Quality as continuous improvement. Here the focus is on quality enhancement, following the cycle of input, implementation, output, and back to input. Institutional research to support and improve quality for ODE is highly valued, but the problem is that findings from QA research do not necessarily guarantee the improvement of practice.

• Quality as compliance with national/regional/international standards and requirements. Here, ODE institutions seek accreditation or recognition from highly regarded regional, international, or transnational agencies, often to eliminate ODE’s “second-class education” image within their society. Problems with this approach include possible conflicts with national priorities and extensive reporting requirements during the evaluation process.

The above shows that quality in ODE is a relative, complex, multifaceted, and culture-related issue, as is QA (Schindler, Puls-Elvidge, Welzant, & Crawford, 2015; Zuhairi, Raymundo, & Mir, 2020). In defining quality in ODE, some (e.g., Stella & Gnanam, 2004) claim that the quality of ODE cannot be judged using conventional quality concepts as ODE is structurally different in many ways; while others (e.g., Huertas et al., 2018; Jung, 2011; Ossiannilsson, Williams, Camilleri, & Brown, 2015) argue that while some universal principles of quality can apply to both traditional face-to-face education and ODE, there are unique features to ODE that should also be noted in a different way, such as technology-based asynchronous interactions, openness in admission and learning paths, and flexibility in teaching and learning. Compared with traditional education, recent ODE relies to a greater extent on students’ self-directed learning and technology competencies as well as their engagement in interaction and collaboration, making it difficult to define and judge the quality of ODE (Beaudoin, Kurtz, Jung, Suzuki, & Grabowski, 2013; Ferreras-Garcia, Ribas, Sales-Zaguirre, & López, 2021; Jung, 2011).

These difficulties and differing views in defining quality in ODE have contributed to the discussions and development of various QA models at international, regional, national, and institutional levels. In analyzing the QA models applied in ODE worldwide, Ossiannilsson et al. (2015, p. 5) concluded that while each QA model serves specific purposes in a given time and context, it defines QA in ODE as:

• Multifaceted. The QA model includes “a multiplicity of measures for quality” and often considers “strategy, policy, infrastructure, processes, outputs and more so as to come to a well-rounded view of holistic quality.”

• Dynamic. The QA model integrates flexibility in its system in order “to accommodate for rapid changes in technology, as well as social norms.”

• Mainstreamed. The QA model is “intended to trickle down throughout the institution and be used as a tool for reflective practice by individual members of staff in their daily work.”

• Representative. The QA model seeks to “balance the perspectives and demands of various interested stakeholders, including students, staff, enterprise, government, and society at large.”

• Multifunctional. The QA model aims to “serve a triple function of instilling a quality culture within an institution, providing a roadmap for future improvement, as well as serving as a label of quality for outside perspectives” (Ossiannilsson et al., 2015, p. 5).

Recognizing the complexity of defining and judging the quality in ODE, Kihwelo (2013, p. 4) argues that definitions of both quality and QA should be open to change and evolution as information in ODE, challenges faced, and understanding of those challenges in the context of ODE are constantly emerging and changing. Keeping this argument in mind, attention must be turned to various models and guidelines for judging and assuring quality in ODE.

Judging and Assuring Quality in ODE

Accreditation and QA Frameworks at National, Regional, and International Levels

As Latchem (2016, p. 10) pointed out, in most countries, ODE is subject to laws, regulations, and practices imposed by national, regional, and/or international QA and accreditation agencies together with other professional and academic organizations.

At the national level, accreditation of an ODE institution or program, after internal and external reviews into the quality of the institution or program, is granted by one or more national quality and accreditation agencies for higher education and hence recognition or license is offered to the said institution. For example, in the UK, the Quality Assurance Agency oversees QA and accreditation of all higher education institutions, including ODE institutions. But in some countries, QA and accreditation for ODE and conventional education are conducted by different agencies or units; an example of this is in India, where conventional universities are accredited by the University Grants Commission (UGC), while ODE institutions are assessed and accredited by a separate unit: the Distance Education Bureau (DEB) of UGC.

Accrediting and QA agencies have developed standards, performance indicators, and procedures to guide higher education institutions during the internal and external review processes and also to guide continuous improvement. Examples of national quality and accreditation agencies that have developed accreditation and QA standards or guidelines for ODE include:

• 1) The UK’s Open and Distance Learning Quality Council which assesses providers in order to enhance quality in ODE and to protect the interests of distance learners and the Quality Assurance Agency which provides various resources on the quality of online learning including Online Delivery & Student Experience (https://www.qaa.ac.uk/en/news-events/support-and-guidance-covid-19/online-delivery-student-experience) and Getting Your Teaching Online (https://www.qaa.ac.uk/scotland/en/focus-on/technology-enhanced-learning/getting-your-teaching-online)

• 2) The US Commissions on Higher Education which published Best Practices for Electronically Offered Degree and Certificate Programs (http://www.c3l.uni-oldenburg.de/cde/found/wiche2.pdf); the US Distance Education Accrediting Commission, recognized by the US Department of Education and the Council for Higher Education Accreditation as a national accrediting agency for ODE, which offers the DEAC Accreditation Handbook (https://www.deac.org/UploadedDocuments/Handbook/DEAC_Accreditation_Handbook.pdf); and the US Quality Matters (QM) which offers various standards including Higher ED Rubric Standards (https://www.qualitymatters.org/qa-resources/rubric-standards) for online and blended courses

• 3) India’s Distance Education Bureau of UGC which publishes Recognition of Open and Distance Learning (ODL) Institutions (https://www.ugc.ac.in/deb/pdf/RecognitionODLInstitutionsHandbook2009.pdf)

• 4) The Malaysian Qualifications Agency which publishes Code of Practice for Programme Accreditation: Open and Distance Learning (https://www2.mqa.gov.my/qad/garispanduan/COPIA/2019/Final%20COPPA-ODL%202nd%20edition%204.12.19.pdf)

• 5) Australia’s Tertiary Education Quality and Standards Agency which provides Quality Assurance of Online Learning: Discussion Paper (https://www.teqsa.gov.au/sites/default/files/quality-assurance-online-learning-discussion-paper_0.pdf?v=1575861233) and Toolkit (https://www.teqsa.gov.au/sites/default/files/quality-assurance-online-learning-toolkit_0.pdf?v=1575861567) and Online Learning Good Practice (https://www.teqsa.gov.au/online-learning-good-practice) during COVID-19

• 6) South Korea’s Ministry of Education and Korea Education and Research Information Service (KERIS) which publish QA regulations and guidelines including Standards for Evaluation and Accreditation of Cyber Universities (available only in Korean) and A Manual for the Management of Academic Affairs in Cyber Universities (available only in Korean)

In some countries across Africa, QA and accreditation systems in higher education are relatively new or do not yet exist at the national level. In such cases, regional agencies offer QA guidelines and training opportunities to support the institutions in such countries.

At the regional level, various agencies and organizations offer QA guidelines and resources for ODE institutions and programs in their region, including (1) the European Association for Distance Learning (EADL) which requires its members to follow EADL’s Quality Standards (https://www.eadl.org/quality-standards/); (2) the African Council for Distance Education (ACDE) which promotes research, policy development, and quality in ODE across African region; (3) the Harmonisation, Accreditation and Quality Assurance in African Higher Education (HAQAA) Initiative with support from the European Union which has created African Standards and Guidelines for Quality Assurance in Higher Education (ASG-QA) (https://haqaa2.obsglob.org/wp-content/uploads/2020/06/ASG-QA_Manual_en_09.FINALE-with-License-1.pdf) and included a set of guidelines for ODE; (4) the Latin American and Caribbean Institute for Quality in Distance Higher Education (CALED) which develops guidelines and instruments for QA in ODE and promotes quality culture in ODE throughout the Latin America region; and (5) the Asian Association of Open Universities which provides Quality Assurance Framework (https://www.aaou.org/quality-assurance-framework/) for ODE institutions in Asia.

At the international level, guidelines on quality in transnational ODE are offered by such agencies as (1) UNESCO/OECD’s Guidelines on Quality Provision in Cross-Border Higher Education (http://www.oecd.org/education/skills-beyond-school/35779480.pdf), (2) International Council for Open and Distance Education (ICDE)‘s Quality Resources (https://www.icde.org/quality-education), and (3) the UK Quality Assurance Agency which provides UK Quality Code for Higher Education Part B: Assuring and Enhancing Academic Quality Chapter B10 – Managing Higher Education Provision with Others (https://www.qaa.ac.uk/docs/qaa/quality-code/chapter-b10_-managing-higher-education-provision-with-others.pdf?sfvrsn=8c02f781_8).

From a look at QA frameworks at national, regional, and international levels, it is clear that globalization and the advancement of technology and transnational ODE have led to similar QA standards and procedures in ODE across the board. However, as Jung (2005) and Jung and Latchem (2007) indicate, while many ODE institutions share some similar QA features, they operate QA in rather different ways depending in part on their resources, size, organizational structure, and culture of quality. Now, let’s delve into QA systems operated in different ODE institutions and programs.

Institutional QA Systems

In order to meet the challenges and demands relating to public funding, social accountability, and the satisfaction of various stakeholders, not to mention the competitiveness of the education offered, higher educational institutions have come to realize the crucial role played by QA. In recognizing the importance of QA practice, many higher education institutions, including ODE institutions and ODE programs, have established QA systems in compliance with national QA and accreditation frameworks, and some have even gone as far as adopting regional and/or international QA systems.

QA management systems. There are three QA management systems identified in ODE institutions and programs, namely, centralized, collective, and dispersed (Jung & Latchem, 2007, p. 241).

Centralized QA systems are run by QA centers or senior managers who oversee the whole QA process, often to be seen in relatively large-scale ODE institutions such as Universitas Terbuka in Indonesia, Open Universities of Malaysia and Sri Lanka, Allama Iqbal Open University in Pakistan, Botswana College of Distance and Open Learning, and The National Open University of Nigeria, to name but a few. Latchem (2016) reports that over 70 of 100 commonwealth universities, including both dedicated ODE institutions and ODE programs within conventional universities, have a centralized QA unit or staff dedicated to QA for their ODE programs.

Collective QA systems are, invariably, operated by committees, councils, and/or boards which play distinctive roles in the different aspects or stages of QA, an example might be that the quality of ODE programs and courses are adjudicated and approved by a program committee or program review team, whereas learning outcomes are evaluated by an Examinations Office. Several ODE institutions such as Indira Gandhi National Open University in India, Anadolu University in Turkey, and the Open University of Hong Kong adopt this collective QA management system.

In contrast, dispersed QA systems share the QA responsibility across various management units. Korea National Open University makes every administrative office and academic unit accountable for quality. The Open University UK, Athabasca University in Canada, and the Open University of China also have adopted the dispersed QA system.

While there is, as yet, no evidence demonstrating and comparing the relative effectiveness and efficiency of the different QA management systems, it is noted in Jung (2005) and Jung, Wong, and Belawati (2013) that a centralize system may be the most effective when an institution first introduces the QA system and a dispersed or collective QA system might work better once the QA system is in place and a quality culture has had time to develop at institutional level.

Focus areas of QA. Most institutional QA systems are focused on the input and process variables such as planning, management, courses and course materials, curriculum, teaching and learning, learner support, staff support, assessment and evaluation, and technology infrastructure – the assumption being that if an institution is adequately resourced and properly managed, it will be in a position to ensure the quality of the collective output. Unfortunately, this is not the case, and as a result, output and outcome variables such as the learning performance of students, career advancement of graduates, and/or contribution to community services have begun to be included in some QA systems (Darojat, 2018).

In defining QA focus areas, some institutions follow a regional or international QA agency’s framework; an example of this is Indonesia’s Universitas Terbuka, which has adopted AAOU’s QA framework and specifies ten QA areas (https://www.ut.ac.id/en/content/quality-assurance): (1) policy and planning; (2) human resources; (3) internal management; (4) students and student profile; (5) design and development education program; (6) design and development course; (7) learning assistance services; (8) infrastructure, media, and learning resources; (9) assessment and evaluation of student; and (10) research and community services.

Other ODE institutions have followed the path of developing their own guidelines. One example is the Open University of Catalonia (OUC) in Spain which, since 2007, has implemented an Internal Quality Assurance System (IQAS) tasked with managing the internal QA process. Its IQAS specifies assessment indicators using a matrix of six standards (https://www.uoc.edu/portal/_resources/EN/documents/qualitat/SGIQ/Annex_VI._Processos_i_dimensions_i_estxndards_d_avaluacix_EN.pdf): (1) review and improvement of the IQAS, (2) design revision and improvement of training programs, (3) support systems for learning and orientation to students, (4) teaching staff, (5) material resources and services, and (6) public information, across three processes, strategic, operational, and support.

Yet other institutions follow QA frameworks for conventional higher education. Canada’s Athabasca University, which is accredited by the US Middle States Commission on Higher Education (MSCHE) adopts seven standards (https://www.athabascau.ca/provost-vice-president-academic/msche-self-study/index.html) specified by the MSCHE:

• Standard I: Mission and goals

• Standard II: Ethics and integrity

• Standard III: Design and delivery of student learning experience

• Standard IV: Support of student learning experience

• Standard V: Educational effectiveness assessment

• Standard VI: Planning, resources, and institutional improvement

• Standard VII: Governance, leadership, administration

Each standard specifies a set of quality criteria. Pakistan’s Allama Iqbal Open University uses two QA frameworks: Pakistan Higher Education Commission’s 11 institutional performance evaluation standards (https://www.hec.gov.pk/english/services/universities/QAA/Pages/Institutional-Performance-Evaluation.aspx) and the Commonwealth of Learning Review and Improvement (COL-RIM) model of 10 quality indicators (http://oasis.col.org/bitstream/handle/11599/602/COL-RIM_Handbook_2014.pdf?sequence=1&isAllowed=y) (Zuhairi et al., 2020).

Due to the nature of ODE, particular attention has been paid to input variables such as the design, development, and delivery of courses and programs and learner support (Jung & Latchem, 2007; Latchem, 2016; Zuhairi, 2020). Examples include the Universitas Terbuka in Indonesia and the Indira Gandhi National Open University in India which specify detailed structures and components of design and development within ODE course packages, learning activities, and assessments and require several procedural steps of internal and external reviews of those components (Darojat, 2018; Samdup & Nembiakkim, 2013).

Standards, best practices, and performance indicators. The terms standards, criteria, components, best practices, and performance indicators are used confusingly in different QA frameworks. For the purposes of this chapter, a QA framework is defined as “a system specifying a set of QA standards consisting of best practices or sub-standards across the focus areas of QA.” Several ODE institutions do indeed use best practices to guide and assess institutional quality across a set of QA areas or standards. For example, under “policy and planning,” one of the ten QA areas (https://www.aaou.org/quality-assurance-framework/) in Universitas Terbuka in Indonesia, best practices state the following: (1) The institution has a well-defined vision and mission statement which incorporates the internal and external educational environment, its potential, national development agenda, and international trend in education. (2) The institution has vision and mission that are shared by all management and staff members. On the other hand, under “learner support” (one of the QA standards of Quality Matters that universities in the USA widely use to assess the quality of their online and blended courses/programs), substandards state the following: (1) The course instructions articulate or link to a clear description of the technical support offered and how to obtain it. (2) Course instructions articulate or link to the institution’s accessibility policies and services. (3) Course instructions articulate or link to the institution’s academic support services and resources that can help learners succeed in the course. (4) Course instructions articulate or link to the institution’s student services and resources that can help learners succeed.

Performance indicators, meanwhile, are used to assess output and outcome performance, often based on numerical data. Common performance indicators for ODE are course/program registrations, course/program completion and graduation rates, collaborative course development and delivery with other ODE institutions, graduate students’ satisfaction, employer satisfaction, and the economic impact of an ODE institution on its local community (Shale & Gomes, 1998) and grades earned on individual assignments, course final grades, discussion board participation and thread initiation, and standardized test scores (Alstete, 2004).

From the abovementioned examples, best practices or substandards can be seen tending to focus on input and process variables and are often stated in qualitative terms, whereas performance indicators focus on the measurement of output and outcome variables and often call for quantitative data collection. While the output/outcome-based approach to QA is highly recommended, difficulties in identifying performance indicators for ODE and quantitatively measuring the quality of ODE need to be considered (Alghamdi & Alanizan, 2018; Shale & Gomes, 1998). Critical, also, is consensus among different stakeholders regarding benchmarks and performance indicators for there to be reliable and valid internal and external evaluations of both outputs and outcomes (Robinson, 2004).

QA procedure. QA is, in effect, a cyclical process in which an institution either as a single entity or as a summation of individual units undertakes self-evaluation, undergoes internal review, and seeks external review and (re)accreditation. This process is often referred to as quality audit and carried out in compliance with national QA requirements.

Quality auditing within an ODE institution starts with self-evaluation, self-study, or self-monitoring. The institution in question collects and analyzes up-to-date information on its education and services and communicates the results to its members and the outside world. Some ODE institutions, such as Universitas Terbuka Indonesia, carry out self-evaluation on a continuous basis, while other institutions, such as Athabasca University in Canada, conduct self-study every few years in preparation for accreditation by a chosen external agency. Allama Iqbal Open University in Pakistan employs annual confidential reports based on staff performance evaluations by section heads; conversely, the Open University of China and Open University Malaysia adopt student evaluations to measure the quality of their courses and services (Jung & Latchem, 2007). In any case, self-evaluation addresses several questions related to QA areas such as the following as listed in the Malaysian Qualifications Agency’s Code of Practice for Programme Accreditation: Open and Distance Learning (https://www2.mqa.gov.my/qad/garispanduan/COPIA/2019/Final%20COPPA-ODL%202nd%20edition%204.12.19.pdf) (p. 63):

• 1) What actions are undertaken in relation to each of the QA standards? Why were these actions taken? Are these actions appropriate?

• 2) How is their effectiveness measured? What performance indicators are available? Are the indicators appropriate?

• 3) What subsequent action should be taken as a result of the review?

• 4) Can the degree of achievements be measured? What are the actual outcomes?

• 5) Can the existing actions be improved, even those that are already effective?

QA standards and guidelines specifically for online education can be found in A Guide to Quality in Online Learning (https://www.tonybates.ca/wp-content/uploads/Guide_Quality_Online.pdf) (Butcher & Wilson-Strydom, 2013).

In order to answer the kind of questions posed above, ODE institutions collect and analyze data obtained from student surveys, including teaching effectiveness surveys, satisfaction surveys, and freshmen/graduates’ surveys, and interviews with staff and other members, enrollment, re-enrollment figures, exam pass and dropout data, and percentage of graduate students, along with other data related to the evaluation of institutional effectiveness and efficiency. The results of the resulting self-evaluation are often published as reports and shared with the governing body of the institution and outside organizations such as the relevant Ministry of Education and national QA agency.

The purpose of an external review is to verify the self-evaluation reports and other related documents by external review teams made up of independent experts who are carefully selected having fulfilled certain criteria and training by the national QA agencies. A common model of external review can be found in the European Association for Quality Assurance in Higher Education’s Considerations for Quality Assurance of E-learning Provision manual (https://www.enqa.eu/wp-content/uploads/Considerations-for-QA-of-e-learning-provision.pdf). It specifies a number of methods for external review, including (1) a self-assessment or equivalent, (2) a site visit, (3) a report resulting from the external assessment, and (4) a consistent follow-up (Huertas et al., 2018, p. 18). For the self-assessment component, several indicators are suggested including the institutional strategy, pedagogical approaches, and virtual learning environment, the innovation and quality of instructional design, the qualifications and experience of academic staff, and the quality of the online courses/programs. For the site visit, indicators such as the institution’s technical infrastructure, virtual learning environment, classrooms, e-library, and interviews with various stakeholders are suggested (Huertas et al., 2018, p. 19). Based on the findings of the self-assessment and the site visit, a report (or reports) is prepared by the external reviewers, and follow-up improvements with a (re)accreditation or audit decision are requested.

Some argue that quality audit conducted through internal and external reviews is pointless and bureaucratic because it tends to focus on input factors mostly in the areas of teaching and research (Cheng, 2010), use data that are not always reliable and valid (Seyfried & Pohlenz, 2018), and often exclude students’ involvement (Ryan, 2015). Conversely, others report QA processes as having positive aspects; Schwegler, Altman, and Bunkowski (2014), for example, reveal that faculty members who participated in the Quality Matters peer review process for their institution’s online courses thought that peer reviews were helpful for them to improve the quality of their courses, acquire new techniques with online technology, and better understand the issue of quality in online education. These contrasting studies indicate the need for more attention to be paid to QA strategies that maximize the positive and minimize the negative.

QA manual and staff training. Development and utilization of QA manuals are not common across ODE institutions. Latchem (2016) reports that only 36% of the ODE institutions surveyed use QA manuals while carrying out the QA activities. A typical QA manual contains QA standards, best practices or substandards, and performance indicators for important QA areas; it lists QA procedures to follow and resources and actions needed for quality enhancement and improvement. In carrying out its internal QA activities, OUC in Spain follows the Internal Quality Assurance System Manual (https://www.uoc.edu/portal/_resources/EN/documents/qualitat/SGIQ/Manual_SGIQ__v.1_per_Llengua_EN_20190219_PORTAL.pdf) developed in 2017. The Manual explains OUC’s internal QA system and includes various appendices which specify OUC’s QA processes, dimensions, and standards along with responsibilities. Universitas Terbuka in Indonesia uses over 200 QA manuals as working guides for all of its QA components (Belawati & Zuhairi, 2007). Each manual outlines a flowchart of all processes, steps to complete a certain task, the person in charge, timeline, and output quality indicators for each activity (Hardini, Sunarsih, Meilani, & Belawati, 2013, p. 86). To support ODE institutions in carrying out internal reviews, the Commonwealth of Learning’s Quality Assurance Toolkit for Distance Higher Education Institutions and Programmes (http://oasis.col.org/bitstream/handle/11599/105/pub_HE_QA_Toolkit_web.pdf?sequence=1&isAllowed=y) offers QA standards, best practices, and performance indicators (Clarke-Okah & Coomaraswamy, 2009).

Several ODE institutions have provided training programs, or sessions, to develop staff competencies with a particular focus on course design and development, and more recently, the focus has shifted to online interactions and the use of various online technologies to ensure managers and staff are fully trained in QA principles and methods (Jung & Latchem, 2007); an instance of this is the Staff Training and Research Institute of Distance Education at IGNOU, which offers staff training with a manual for course/program design and multimedia development, while the Digital Media Center in collaboration with the Institute of Distance Education at Korea National Open University provides training for online course development and implementation to its academic staff. The Centre for Distance Education at Athabasca University in Canada and the eLearn Center at OUC in Spain offer both professional training sessions and MA and PhD degree programs for further postgraduate education. Many conventional universities also offer faculty training and professional development sessions aimed at improving their faculty competencies in online and blended education via the Center for Teaching and Learning or a similar unit.

Conclusion: Future of QA in ODE Institutions

This chapter has shown that ODE institutions and national, regional, and international bodies are seeking to ensure quality in ODE and develop appropriate QA and accreditation models for ODE, but also that there is still a need for more balanced, effective, and yet flexible QA frameworks and guidelines for the ever-changing landscape of ODE. For just such QA systems, Jung (2010, p. 25) suggests an ecological QA framework that “emphasizes inter-relation transactions between elements, i.e., providers, learners, cultures, and policies and systemic integration of those elements, and stresses that all these elements, within a QA system, play an equal role in maintaining the balance of the whole.” The ecological QA approach argues that ODE institutions should build an all-stakeholder-oriented QA system as the existing provider-centered QA approach tends to focus mostly on providers’ perceptions and ignore the inter-relational and dynamic nature of the QA system. It also highlights the importance of creating a globally oriented and yet locally adaptive QA system to reflect sociocultural diversity in QA concepts and practices.

While ODE institutions should undoubtedly consider both internal and external accreditation and QA requirements and standards in various areas, they need to place quality in pedagogical dimensions such as course design and development, learning support, and assessment and evaluation at the center of the accreditation and QA system as these dimensions define the quality for student learning (Conrad, 2002; Daniel et al., 2008; Marciniak, 2018). This can be achieved by specifying procedures for courses/material design and development; involving both internal and external experts; considering changing needs and demands of learners; offering suitable training for faculty, tutors, and other support staff on a regular and continuous basis; and relating teaching and learning and learner supports to learning outcomes (Jung, 2013).

As Bradley (2005) argues, many QA models tend to encourage accountability and conformity rather than innovation, diversity, and inclusion. Future QA needs to consider ways to attend to, and negotiate with, a wide range of needs, values, and perspectives of stakeholders (Ryan, 2015) while addressing diverse learning paths and delivery modes in the QA process as the convergence of on-campus and online education becomes ever more widely adopted in higher education and the awareness of consumer rights is heightened. We still see many ODE institutions that have flaws in applying QA standards and guidelines regularly and consistently in both course and material design and development, assessment and evaluation methods, and learner and staff support, despite having well-established technology infrastructure and producing good reports for external QA reviews. To overcome such flaws, the ODE institutions need to move from the existing control framework to a culture creation framework and integrate QA activities into their institutional cultures and everyday practices.

As discussed above, ODE institutions have tended to pay more attention to input and process variables and have ignored output/outcome variables such as learning outcomes. Recent years have seen a growing demand for review by and approval of learning outcomes from various QA agencies and society in general as these can be used to guide students’ learning paths, design focused learning activities, and thus improve course/program design and also provide effectiveness of a course, program, or institution. Gallavara et al. (2008, p. 12) argue that learning outcomes are “a tool to describe and define a learning and assessment process and its product, which can lead to improved pedagogical practice in education and improved student learning practice.” We are observing a paradigm shift in QA in some innovative ODE institutions such as the Open University of Sri Lanka and Universitas Terbuka in Indonesia (Latchem, 2016; Zuhairi et al., 2020), with movement from applying a criterion/standard-based approach to the design, delivery, and assessment of ODE courses/programs/materials to an outcome-based QA approach focusing on learning performance. Yet despite the above, further elaboration of accreditation and QA indicators measuring varied types of learning outcomes and the development of outcome-based quality culture would help ODE institutions integrate the outcome-based approach into their existing QA system.