Abstract
This chapter overviews the 14 chapters in Section VI of the Handbook, which are focused on the design, delivery, and assessment of open, online, and digital learning. While the chapters address a wide variety of factors relevant to ODDE, I discuss several common themes, including evidence for the effectiveness of ODDE, standards to guide effective ODDE design, assessment and credentialing of ODDE learning, and the need for future research to move beyond studying individual factors (micro level) in higher education. Finally, I conclude with a call to how we can design the future of ODDE, including first, seeking to understand systemic nature of learning; second, embracing the sociocultural nature of learning, and third, utilizing the full repertoire of diverse scholarship methods to develop knowledge in our field.
You have full access to this open access chapter, Download reference work entry PDF
Similar content being viewed by others
Keywords
Introduction
A major challenge in the field of open, distance, and digital-based education (ODDE) has been the overemphasis of the role of technology in learning versus pedagogy, strategy, and design. To be certain, new technologies provide new affordances that can anchor new approaches to teaching and learning, and oftentimes, technological innovation precedes pedagogical creativity. However, when we focus too much on the technology implementation, we may miss the bigger concerns with how to design the context to benefit from the technology. In doing so, we may make the error of putting “new wine in old bottles” by resurrecting the “media vs. methods” errors of the 1980s/1990s in the new flasks of online and open education (Lockee, Moore, & Burton, 2001).
In fact, Honebein and Reigeluth (in press) have found that we often do exactly that – and may be going backwards as a discipline. In their analysis, they found that there is research to improve learning by focusing on strategy and research to prove one media-based approach is better than another through media comparison research. From 2010 to 2019, in their analysis of educational technology journals, they found “a significant rise of experimental, research-to-prove papers. These papers appeared to (1) confound media and instructional methods, (2) not include sufficient information about the instructional objective, (3) omit one or more of the effectiveness, efficiency, and appeal outcomes, and (4) not report whether or not the researchers conducted formative evaluation” (para. 35).
Because of the tempting siren song toward media comparison research, it is especially critical that scholars of open, distance, and digital learning focus on research to improve by emphasizing how we design, develop, and assess effective learning in these digital spaces. For this reason, this section of the Handbook is the largest of the book, and focuses on the design, development, and assessment of learning in ODDE spaces.
Organization of Section VI
Topics for this section of the Handbook were developed in consultation with the overall Handbook editors and were an attempt to represent major theoretical thrusts in the discipline. Authors were encouraged to focus on the design, development, and assessment issues related to these topics, rather than on technological developments.
An attempt was made to find some of the most highly cited and respected authors in these areas while representing global diversity. In addition, an attempt was made to represent diversity by seeking some authors that were established experts as well as some rising stars in their areas. I believe that the following chapters represent some of the best thinking and current practices that can guide the effective design of ODDE learning.
The section includes several chapters that overview the design of different kinds of distance learning. First, Graham and Halverson provide guidance on designing blended and flexible learning environments. Meanwhile, Martin and Bollinger lend their considerable expertise to understanding the design of online learning in higher education. In contrast, Borup and Archambault, former editors of the Journal of Online Learning Research, describe the effective design of online learning for children and youth. Similar overview chapters are also provided by Schrader on game-based learning and Ge and Huang on problem-based/inquiry-based learning .
After these more general chapters, there are additional chapters the probe strategies for the effective design of specific technology-afforded learning environments such as flipped classrooms (Lee), online learning communities (Cleveland-Innes & Hawryluk), MOOCs (Stracke et al.), and computer-supported collaborative learning environments (Hmelo-Silver & Jeong). Dron and Anderson, and Bozkurt et al. each provided valuable perspectives on informal learning and social media-based learning, respectively, while Bond and Bergdahl shares strategies for addressing one of the most often discussed challenges in ODDE (and any) learning environment – improving student engagement. Finally, assessment within ODDE environments is then discussed by Hickey et al. generally, and by West and Cheng more specifically in the area of open micro/alternative credentials.
Even though this section of the Handbook is large, it is unfortunately not large enough. An entire handbook could be produced, and often has been, on each of these topics, and by limiting ourselves to only 14 chapters, there were many worthy topics that were excluded. Still, the authors in this section have provided, in my view, enough powerful and useful strategies and frameworks to improve the effective practice of ODDE learning by any scholar/practitioner who reads their chapters.
Overarching Themes
While these chapters are individually diverse, there were a few interesting cross-cutting themes.
Originary Theory to Guide ODDE Design
Graham and Halverson cited McDonald and Yanchar’s (2020) description of two types of theory available to ODDE scholars/practitioners. The first is imported theory, transferred from other disciplines and applied to ODDE environments. Examples of imported theory could be the application of Flow theory, Self-determination Theory, Control-Value Theory, and the Player Experience of Need Satisfaction model, all of which Schrader demonstrated can effectively describe and inform game-based learning. Another helpful example is in Hickey et al.’s chapter, where they describe how four main paradigms on learning theory (differential, cognitive-associationist, cognitive-constructivist, and situative-sociocultural) affect our understanding of effective and efficient online assessment. Finally, Bond and Bergdahl described how Astin’s (1984) theory of community has helped to characterizes a student’s involvement, or engagement, in learning.
The second type of theory is originary theory, developed within a research discipline and specific to that discipline. Because ODDE learning is, and has been, changing rapidly, originary theory in our discipline is still more rare and underdeveloped. However, the authors of these chapters in this section identified several existing originary theory frameworks that can guide effective ODDE design. These include:
-
The Community of Inquiry framework, cited by Graham and Halverson as the most referenced theory in blended learning and then discussed extensively by Cleveland-Innes and Hawryluk, as well as Bond and Bergdahl and Hickey et al. This theory illuminates the types of interactions and presence that members of an online environment can achieve.
-
The Academic Communities of Engagement framework, cited by Borup, Graham, West, Archambault, and Spring (2020) and Graham and Halverson, which adapts ideas from the COI theory, as well as ideas from research into social learning and student engagement, to describe how personal and course communities interact to support student engagement, particularly for adolescents.
-
Cognitive Theory of Multimedia Learning, cited by Schrader to explain how game-based learning affects cognitive processing, as well as by Lee.
-
Czerkawski and Lyman’s (2016) e-learning engagement design framework, Gao and Ji’s (2019) Five-Star Teaching Cycle Framework of Online Courses, and Conole’s (2014) 7Cs of Learning Design Framework, all cited by Martin and Bolliger.
In addition, several authors in this section proposed new originary theories, or at least descriptive frameworks, that provide new perspective to guide designers of ODDE learning. Among these new ideas include the following:
-
Bond and Bergdahl’s bioecological model of engagement, which places the student at the center of engagement, but affected by a microsystem, mesosystem, exosystem, and macrosystem of affecting factors. In addition, these authors described student engagement at the micro level as not only including the traditional descriptions of affective, behavioral, and cognitive dimensions, but also a social dimension – representing the important sociocultural perspective on learning that also affects students’ engagement and motivation to learn.
-
Dron and Anderson’s conceptualization of dimensions of learning, including informal/formal, intentional/incidental, and self-directed/dependent – dimensions that can be helpful in defining and better conceptualizing informal learning.
-
Ge and Huang’s pedagogical framework that summarizes the four key aspects to designing online problem-based learning as preparation/planning for the PBL, design and development, implementation and facilitation, and assessment (along with strategies and suggestions for each phase).
-
Hickey et al.’s helpful organizing of assessment practices as assessment of learning, for learning, as learning, and then even how assessment is sometimes used as compliance or as sabotage – and then framing these assessment practices around various learning theories and assessment levels and formats.
-
Hmelo-Silver and Jeong’s organizing of the computer-supported collaborative learning literature into four different clusters representing different design approaches to CSCL learning, based on technologies, pedagogies, and collaboration modes used.
-
Martin and Bolliger’s support framework, which identified four types of support (based on a previous paper) that instructors need to be effective online: administrator, personnel, pedagogical, and technological support.
-
West and Cheng’s description of an open education infrastructure that focuses on open content, open practices, and open recognition (synthesizing previous ideas from Wiley, 2018) .
The Need for Research to Expand Beyond Study of Individuals in Higher Education
This Handbook has adopted a 3M framework for considering research, first discussed by Zawacki-Richter (2009). West and Cheng further develop this framework in the area of open credentials by describing how the micro level of research focuses on individuals, while the meso level represents research on institutional impacts and macro level research studies impacts at the societal level. Authors in this section of the Handbook identified several new areas for necessary research exploration, which can be organized within this 3M framework. For example,
-
Bond and Bergdahl argued that student engagement includes a largely unresearched social dimension, particularly in ODDE settings, as well as the relationship between the bioecological levels in their model of engagement. In addition, they called for more research into disengagement as a separate construct from engagement itself.
-
Graham and Halverson argued that blended learning research to date has focused on the individual, micro, level, and research at other levels is needed. They cite several emerging frameworks for understanding institutional adoption of blended learning that can guide future research, including suggestions about necessary blended learning competencies and professional development.
-
West and Cheng similarly found that the research on the impacts of open credentials is focused at the micro/individual level, and is despairingly scant at other levels.
-
Lee documented explosive growth in research into flipped learning, but mostly these studies have focused on university learners.
-
Hickey et al. argued that while online learning has embraced situative/sociocultural perspectives on learning, there is little research consideration of how this theory affects assessment practices. They also caution against throwing out traditional assessment practices in favor of “authentic” assessment, when research is still needed into how all these manners of assessment can be useful in ODDE teaching.
-
Martin and Bolliger stated that of Moore’s (1989) three types of interaction, learner-content interaction is surprisingly under-studied. Moving towards the meso-level of the 3 M research framework, they argued that the interaction between instructors and instructional designers within institutions is under-studied.
ODDE Learning Can Be Effective
All learning environments can be either effective or ineffective depending on a slew of other variables such as personal investment, social learning factors, the design of the experience, and the learning strategies employed. However, the authors in this section were still able to document many positive effects from including that ODDE learning, including the following:
-
Provides more effective learning (e.g., Hmelo-Silver & Jeong, Lee, Schrader).
-
Improves student communication/collaboration (Bozkurt, Hmelo-Silver & Jeong). Similarly, Schrader explained the relationship in reverse, that student collaboration promoted the strongest learning in game-based environments.
-
Promotes student engagement (e.g., Bond & Bergdahl, Bozkurt, Graham & Halverson, Hmelo-Silver & Jeong, Lee, Schrader).
-
Incorporates more active student learning (e.g., Hmelo-Silver & Jeong, Lee).
-
Improves flexibility and individualized learning as well as more flexible teaching (e.g., Bozkurt, Lee, Stracke et al., West & Cheng).
-
Improves student autonomy (e.g., Schrader, West & Cheng).
-
Supports feedback to learners on performance (e.g., Hickey et al., Hmelo-Silver & Jeong, Schrader, West & Cheng).
However, these effects, unsurprisingly, were very nuanced, and results varied depending on many variables including effectively matching the ODDE learning model with individual learners who would most benefit, utilizing effective design/teaching strategies, and employing strong learning activities.
Standards to Guide ODDE Design
Several authors identified helpful standards frameworks that can guide our effective design and implementation of ODDE environments. For example,
-
Florence and Bolliger identified eight different online course development rubrics or standards frameworks, both within the United States and internationally, for designing higher education online learning. They then synthesized these standards and rubrics to create the Online Course Design Elements instrument that identifies five categories of design elements that should be attended to in creating effective distance education courses.
-
Similarly, Stracke et al. presented the Quality Reference Framework as synthesis of international standards in the form of seven dimensions for creating effective online learning, and in the case of their chapter, MOOCs.
How Assessment Supports Learning
Three chapters in particular argue that rather than simply assuming that assessment follows the teaching/learning process, that effective and innovative approaches to assessment and credentialing can restructure and prompt changes in how we teach and “do” education. This is particularly true for ODDE environments where, as Hickey et al. argued, learning assessment takes on wholly new meaning because the signals for what we value are different. For example, as delivering content and participating in learning interactions became more flexible due to online technologies, instructional designers are increasingly moving away from seat time and credit hours as evidence of “learning” and focusing more on skills, performance, and abilities. But this requires new thinking when it comes to how we assess learning. In addition, the amount of time required for individual feedback in online interactions can be overly strenuous – again requiring creativity in how to provide feedback and assessment in efficient ways.
Similarly, Dron and Anderson argued that assessment and digital credentialing can be especially powerful in supporting informal learning, by providing a mechanism for recognizing the value of skills/knowledge gained in informal learning. This system benefits from the use of challenge assessments, decoupled from the learning itself, to allow for more learner flexibility. In addition, technological systems are developing – but need to develop more – to improve how we store, recognize, and share these credentials.
Finally, West and Cheng offered definitions to the complicated world of micro/open/alternative credentials, and review research findings about the impacts of these credentials at the micro/meso/macro levels of education. While acknowledging that the research on these types of credentials is small and emerging, the authors argued that the core ideas of transparency, openness, learner agency, and clear data representation are sticky ideas that will continue to impact education in one form or another.
Designing the Future of Open, Distance, and Digital Education
During the COVID-19 pandemic, the world learned the value of instructional designers. As schools closed and laptops opened; parents, teachers, and students learned quickly the difference between well designed instruction and emergency teaching (Hodges & Fowler, 2020). Instructional designers around the world were highly sought after to help soften the blow. This affirmed the value the discipline needs to play on the world stage, and the growth and maturity of the field over the last century.
The chapters in this section describe well this maturity. We have a solid foundation of theory and research to describe precisely how effective learning can be designed and assessed in many situations, levels, and subject areas. But once the foundation of a home is established, it is not the time to sit down in the recliner, but rather to continue building the walls upward.
So what are the next steps that will enable the field of open, distance, and digital learning to build upon its strong foundation? The chapters in this section suggest many ideas, and I conclude with the following overarching observations.
First, we understand well how to design and develop an effective product, curriculum, or training, but we need greater focus on understanding the systemic nature of learning. Much of the research in the field focuses, as mentioned above, on the micro level of the 3M framework. In addition, research tends to study single variables, and designs often hyperfocus on a single theory for explaining learning, and in both cases the complexity of the learning environment can be ignored. It is difficult to conduct studies sophisticated enough to understand the complex interconnection between all of the levels of the 3M framework. But we need to research and design the systems, in order to best affect the individuals.
Second, but similarly, we need research and design to fully embrace the sociocultural nature of learning. Social relationships affect how people learn (Cleveland-Innes & Hawryluk), how they collaborate (Hmelo-Silver & Jeong), how they get engaged in education (Borup et al., 2020) and how they should be assessed (Hickey et al.). When we talk about a systemic view of learning, a major and critical part of that system are the relationships learners have with each other, their teachers, and even their personal support systems (see Borup et al., 2020).
Third, we need research methods as rich and varied as the humans we study. While educational methodologists have developed increasingly more sophisticated methods for studying the complex process that is human interaction and learning, our research journals tend to remain unbalanced. Far too many published research studies re-examine research questions we already know the answer to, using methods we have already seen before. Other disciplines have pushed the edge of their expertise developing a wide variety of approaches to research, and we need similar variety in ODDE scholarship.
As I described in another article (West, 2020) there are different ways of creating knowledge and legitimate scholarship, and we need to appreciate and encourage, in our journals, publication of all these ways of knowing, including:
-
Review scholarship that rigorously summarize what other studies have found.
-
Conceptual scholarship that identifies and qualifies key concepts that characterize a phenomenon and model the relationships between items in a system.
-
Theoretical scholarship that critiques existing ideas, establishes the boundaries of existing theory, and develops new paradigms that create possibilities for new thinking.
-
Exploratory research that develops theory by exploring variables and their relationships through empirical observation and often bottom-up inductive thinking.
-
Explanatory research that tests theory by objectively validating its generalizability.
-
Critical research that explores what ideas and experiences mean to various groups, and how we can productively put theory into action.
-
Design-based research and design cases that develop local and generalized theory by designing interventions for real learners in real contexts.
In the last 20 years, the world of open, distance, and digital learning has evolved dramatically, as social media (Bozkurt et al.), microlearning (West & Cheng), complex educational games (Schrader), MOOCs (Stracke et al.), and other technological innovations have changed the way we can teach and learn. It cannot be known what new technologies will be developed in the future to meet the needs of future learners, but only through a systematic understanding of learning, and especially the rich sociocultural layers that affect education, gained via a variety of research methodologies will we be prepared for this future. The chapters in this section provide the foundation, but now we need to push forward to finish the building.
References
Astin, A. W. (1984). Student involvement: A developmental theory for higher education. Journal of College Student Development, 25(4), 297–308.
Borup, J., Graham, C. R., West, R. E., Archambault, L., & Spring, K. J. (2020). Academic communities of engagement: An expansive lens for examining support structures in blended and online learning. Educational Technology Research and Development, 68(2), 807–832.
Conole, G. (2014). The 7Cs of learning design – A new approach to rethinking design practice. In S. Bayne, C. Jones, M. de Laat, T. Ryberg, & C. Sinclair (Eds.), Proceedings of the 9th international conference on networked learning 2014 (pp. 502–509). Edinburgh: University of Edinburgh. Retrieved from https://www.lancaster.ac.uk/fss/organisations/netlc/past/nlc2014/abstracts/pdf/conole.pdf
Czerkawski, B. C., & Lyman, E. W. I. I. I. (2016). An instructional design framework for fostering student engagement in online learning environments. TechTrends, 60, 532–539. https://doi.org/10.1007/s11528-016-0110-z.
Gao, D., & Ji, Y. (2019). Research on the framework of online course design based on “First Principles of Instruction”. In 2nd Conference on education and cognition, behavior, neuroscience (ESSP proceedings series, pp. 30–38). https://doi.org/10.25236/icecbn.2019.006.
Hodges, C. B., & Fowler, D. J. (2020). The COVID-19 crisis and faculty members in higher education: From emergency remote teaching to better teaching through reflection. International Journal of Multidisciplinary Perspectives in Higher Education, 5(1), 118–122.
Honebein, P. C., & Reigeluth, C. M. (in press). How do we solve a problem like media and methods? In R. E. West, & H. Leary (Eds.), Foundations of learning and instructional design technology. https://edtechbooks.org/
Lockee, B., Moore, M., & Burton, J. (2001). Old concerns with new distance education research. Educause Quarterly, 24(2), 60–68.
McDonald, J. K., & Yanchar, S. C. (2020). Towards a view of originary theory in instructional design. Educational Technology Research and Development, 68(2), 633–651.
Moore, M. G. (1989). Editorial: Three types of interaction. The American Journal of Distance Education, 3(2), 1–7. https://doi.org/10.1080/08923648909526659.
West, R. (2020). The role of conceptual and theoretical scholarship in understanding research in digital learning. Revista de Educación a Distancia (RED), 20(64).
Zawacki-Richter, O. (2009). Research areas in distance education: A Delphi study. International Review of Research in Open and Distance Learning, 10(3), 1–17.
Author information
Authors and Affiliations
Corresponding author
Section Editor information
Rights and permissions
Open Access This chapter is licensed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license and indicate if changes were made.
The images or other third party material in this chapter are included in the chapter's Creative Commons license, unless indicated otherwise in a credit line to the material. If material is not included in the chapter's Creative Commons license and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder.
Copyright information
© 2022 The Author(s)
About this entry
Cite this entry
West, R.E. (2022). Introduction to Design, Delivery, and Assessment in ODDE : Theories, Research, and Practices to Guide ODDE Design and Assessment. In: Handbook of Open, Distance and Digital Education. Springer, Singapore. https://doi.org/10.1007/978-981-19-0351-9_89-1
Download citation
DOI: https://doi.org/10.1007/978-981-19-0351-9_89-1
Received:
Accepted:
Published:
Publisher Name: Springer, Singapore
Print ISBN: 978-981-19-0351-9
Online ISBN: 978-981-19-0351-9
eBook Packages: Springer Reference EducationReference Module Humanities and Social SciencesReference Module Education
Publish with us
Chapter history
-
Latest
Introduction to Design, Delivery, and Assessment in ODDE- Published:
- 22 October 2022
DOI: https://doi.org/10.1007/978-981-19-0351-9_89-2
-
Original
Introduction to Design, Delivery, and Assessment in ODDE : Theories, Research, and Practices to Guide ODDE Design and Assessment- Published:
- 21 September 2022
DOI: https://doi.org/10.1007/978-981-19-0351-9_89-1