Introduction

Traditionally, learning technology design has emphasized strategies that support learners based on prior knowledge using scaffolding, self-regulated learning theories, and others (Kim et al., 2018; Xie et al., 2019). However, with the emergence of complex learning challenges—such as diverse learners and modalities of technology (Adamson & Darling-Hammond, 2012; Ahn, 2019)—theorists argue for a more comprehensive approach that considers broader learner experiences with technology. This approach to design, often described as learning experience design (LXD), accounts for the evolution of technology and its impact on the field of learning design and technology (LDT). In many ways, LXD represents a nascent view that, while commonly referenced in professional practice (Wang et al., under review), remains relatively under-studied in research. As discourse emerges on how to support learning beyond content development (Hokanson et al., 2020), it may be unclear if the experience alludes to aspects of user-experience design (UXD) or other elements, such as socio-technical considerations.

Consensus around the elements that comprise LXD is important for multiple reasons. In prior literature, Reiser recognized (2007) the need for researchers to engage in thoughtful and intelligent discussions regarding the conceptual, theoretical, and lexical basis of their work. Ultimately, Reiser’s (2007) work underscores the importance of clear and precise consensus in advancing the field of instructional design and technology so that we will “at least be able to discuss intelligently” (p. 1). In terms of elements of LXD related to practice, Quintana and Quintana (2023) suggest that LXD must identify the “process, activities, and tools for designing learning experiences, how to navigate the design process” (p. 1). In the current paper, we have endeavored to address these issues that impact research and practice. Indeed, establishing consensus is a key aspect of identifying and developing LXD elements, including the competencies needed to train practitioners. In terms of scholarship, this plays a critical role in establishing a common conceptual and theoretical basis for building and extending subsequent research. Identifying the elements that comprise a phenomenon is therefore essential in advancing understanding and knowledge; however, this presents a range of challenges. For example, identifying elements of well-defined fields typically borrows from existing and accepted research. However, in fields that are emerging or that lack substantial research focus, establishing consensus around elements that comprise phenomena becomes more difficult. Compounding this, there is no commonly accepted methodology for development of consensus because various researchers approach this task in different ways that are predicated on their disciplinary backgrounds, research methods, theory, etc.

Reiser’s (2007) work emphasizes the crucial role of establishing a shared understanding of a phenomenon being studied, especially in a field that is constantly evolving. In the area of LXD, scholars (Chang & Kuwata, 2020) have identified three key areas requiring attention: (1) the development of a clear and coherent definition, (2) the establishment of robust methodologies, and (3) the construction of relevant theoretical frameworks. As these become established, the field can explore LXD, develop best practices in design, measure using appropriate methodologies, and identify present gaps that should be addressed among the field. It is thus important for researchers to engage in transparent and collaborative discussions around the conceptual and theoretical foundations of their work. Building upon prior work, the present article aims to refine and clarify elements of LXD. Specifically, we report our extension of Schmidt and Huang’s (2022) systematic qualitative content analysis aimed at establishing a clear and comprehensive overview of LXD elements. This study offers empirical evidence on the fundamental elements of LXD as reported by practitioners, contrasting with previous research that has been predominantly theoretical. This allows us to gain a deeper understanding of the constructs presented in the aforementioned article and to explore potential areas of consensus and divergence among LXD experts in the field.

Problem, purpose, and research question

To date, there is little consensus in the field regarding how LXD should be defined, which limits the ability of researchers and practitioners to communicate and understand the phenomenon. The development of clear and precise understanding of LXD and the elements of which it is comprised requires a combination of rigorous analysis, critical reflection, and ongoing dialogue and debate. This lack of consensus can hinder the cohesion of theories and methodologies, as well connections to other research studies and important gaps. Consensus around the elements that comprise LXD is therefore needed to better understand how it contributes to learning outcomes and influences both experiential and perceptual elements of learning. In the field of LDT, researchers have acknowledged that the emerging phenomenon of LXD is poorly articulated and requires attention (Gray, 2020; Schmidt & Huang, 2022; Tawfik et al., 2022). The purpose of the current research, therefore, is to advance our conceptual and theoretical understanding of LXD by establishing a comprehensive description of the elements, of which it is comprised through a systematic and collaborative approach. While other studies might employ a qualitative approach to explore a phenomenon, this approach is potentially limiting because it is difficult to quantify patterns on emergent topics. The Delphi methodology addresses this issue, as it generates consensus among SMEs within a field using repeated surveys that are iterated based on the responses from participants (Hasson et al., 2000; York & Ertmer, 2011). By conducting a Delphi study with SMEs, our objective was to explicate areas of agreement and difference regarding the characteristics that define LXD. This approach was intended to enhance and elucidate our comprehension of the constituent elements that comprise this multifaceted phenomenon. Our goal is, therefore, to provide a foundation for future research and practice in LXD, enabling scholars and practitioners to communicate more effectively and to develop more targeted approaches to supporting learners in a rapidly evolving technological landscape.

The research question that guided this study were as follows:

  1. 1.

    To what extent do experts agree or disagree with perspectives that describe elements of LXD?

    1. a.

      To what extent do experts agree or disagree with design perspectives that describe elements of LXD?

    2. b.

      To what extent do experts agree or disagree with disciplinary perspectives that describe elements of LXD?

    3. c.

      To what extent do experts agree or disagree with theoretical perspectives that describe elements of LXD?

    4. d.

      To what extent do experts agree or disagree with methodological perspectives that describe elements of LXD?

Literature review

A limited number of scholarly publications have sought to provide clear and precise perspectives of the LXD phenomenon. Although these emergent viewpoints sometimes overlap, they also highlight unique nuances as theorists and researchers try to provide clarity about LXD. Ahn’s (2019) work contributes with its broad description of LXD as he argues for an emphasis on using various tools in designing for knowledge, interest, and identity. Efforts to explicate the LXD phenomenon have been diverse, ranging from theories emphasizing human-centered design (Chang & Kuwata, 2020) to holistic approaches incorporating socio-technical lenses (Gray, 2020; Jahnke et al., 2020). Beyond conceptual discourse, researchers have also used other data sources to explore aspects of LXD. For example, Schmidt and Huang (2022) employed qualitative content analysis to present initial findings. Additionally, Tawfik et al. (2022) research identified two central aspects of LXD (“interaction with the learning environment” and “interaction with the learning space”), while Schmidt et al.’s (2020a, 2020b) introductory chapter of an edited volume on LXD introduced three axia of the phenomenon focusing on (1) complexity, (2), transdisciplinarity, and (3) multiple literacies. Clearly, a range of views informs the LXD phenomenon, and understanding the similarities and differences between these conceptualizations is crucial for scholars and practitioners alike. One way to approach this is by examining each perspective individually and then comparing and contrasting them with each other, which we do in the following paragraphs.

LXD presented in or derived from a 2020 edited volume

Most of the scholarship we present in this section is drawn from the edited volume entitled Learner and User Experience Research: An Introduction for the Field of Learning Design and Technology (Schmidt et al., 2020a, 2020b), which in some ways represents a seminal work from a collection of scholars. Of these, the first we present is that of Vann and Tawfik (2020), who emphasized how interface design aligns with principles of human–computer interaction and learning processes to support learners’ knowledge construction. This view highlights the importance of designing interfaces that are intuitive and user-friendly, which can facilitate learning by reducing cognitive load and allowing learners to focus on content. Similarly, Jahnke et al. (2020) describe LXD as having a focus on improving the usability and learning experience with technology from the perspective of the learner. Both emphasize the importance of designing technology that is learner-centered and user-friendly, which can enhance learning outcomes as it fosters motivation and reduces barriers to engagement.

In contrast, Chang and Kuwata (2020) suggest how LXD focuses more broadly on the practice of designing learning as a human-centered experience that leads to a desired goal. In doing so, they highlight elements that underscore the importance of designing holistic learning experiences that are goal-oriented and take into account learners’ individual needs and preferences. Similarly, Abbott (2020) proposes how LXD might foreground learners and their desired outcomes in a goal-oriented way, acknowledging individual experience. Characteristics of LXD thus highlight the importance of designing learning experiences that are tailored to learners’ needs and preferences, which can promote personal relevance for the individual. Stefaniak and Sentz (2020) take a similar approach, yet extend LXD to include a systems-level view that designs products that are relevant to the everyday experiences of users or learners, and which encompasses the ability for a designer to address all the ways a learner will interact with the product being developed. LXD therefore emphasizes the importance of designing learning experiences that are contextual and relevant to learners’ everyday lives, which can promote engagement and motivation by making learning more personally meaningful.

The multiple views represented in the above views of LXD were acknowledged as three broad axia in the introduction to the edited volume (Schmidt et al., 2020a, 2020b). First, the Axiom of Transdisciplinarity emphasizes that LXD is a confluence of multiple areas, including UXD and Learning, Instruction, Design & Technology (LIDT). This suggests that learning experience designers must possess a broad and diverse range of skills derived from multiple sources, along with knowledge to effectively design learning experiences. Second, the Axiom of Complexity reinforces this notion by highlighting that LXD must consider the complex and interconnected relationship between the learner, the intervention, and the learning context. LXD therefore is not limited to the creation of effective learning interventions, but must also consider the broader context in which learning occurs. Third and finally, the Axiom of Multiple Literacies further reinforces the transdisciplinary nature of LXD by emphasizing that it requires a diverse range of literacies, skills, and abilities. These multiple literacies are necessary to effectively design learning experiences that account for the complex and dynamic relationship between the learner, intervention, and context. This highlights the importance of interdisciplinary collaboration and the need for learning experience designers to be proficient in a range of disciplines, including instructional design, cognitive psychology, human–computer interaction, and educational technology.

Whereas previous scholars provided conceptual considerations, it is largely theoretical and thus a gap exists between how practitioners and scholars conceive of LXD. Schmidt and Huang (2022) sought to synthesize the viewpoints presented above using qualitative content analysis methods. Through a rigorous, multi-phase process, the authors developed a definition of LXD that characterizes it as a human-centric, theoretically-grounded, and socio-culturally sensitive approach to learning design, intended to propel learners towards identified learning goals, and informed by UXD methods (p. 1). Their work highlights the importance of designing learning experiences that are based on sound theoretical principles and are sensitive to learners’ sociocultural contexts, which can promote deeper learning and more positive outcomes for a more diverse range of learners.

LXD descriptions presented independently of the 2020 edited volume

Outside of the discourse that derives from seminal work presented within the aforementioned edited volume on the topic of LXD, a cadre of researchers have provided descriptions of the phenomenon. Perhaps the earliest, formal, academic description of LXD is that provided by Ahn (2019), which highlights the emergence of the term as a response to the changing nature of learning design practice. Ahn (2019) describes LXD as a more recent development in comparison to the more traditional approach of instructional design. While ID focuses on creating curriculum and learning experiences in classroom and formal online course environments, Ahn (2019) argues that LXD expands beyond these settings to include a wider variety of learning situations, such as those found in museums, libraries, public spaces, and various online and virtual environments. Similarly to elements presented above, Ahn’s (2019) description acknowledges the importance of designing effective learning experiences, albeit with different levels of detail, and recognizes the need for designers to consider the learning context and the user experience when creating learning interventions.

Other scholarship highlights specific interactions as learners engage with the designed learning environments. For example, Floor suggests LXD (2023) approaches provide experiences that allow an individual to accomplish learning in a human-centered and goal-oriented fashion. Schatz’ (2019) work places more emphasis on the practical aspects of LXD, such as design thinking, usability, and interaction design methods, and the application of user-centered design. Schatz highlights the shift in focus in LXD from learning product design to focus “more on broad learning outcomes with an extensive toolkit to apply towards this end” (p. 83). This description focuses more on technical aspects of designing learning experiences, whereas other descriptions take a more comprehensive and human-centered approach. Similarly, Gray and Boling (2023) suggest that LXD is not a new field, but instead a philosophy of design that is more inclusive of UX, HCI, and social justice, among others. Finally, Tawfik et al (2022) present a conceptual framework that focuses on specific learning-technology interactions, namely (a) the interaction between the learning environment and (b) interaction with the learning space. The former largely consists of UX elements of the learning environment that relate to the utility of the technology, such as customization, content placement, functionality of components, interface terms aligned with existing mental models, and navigation. The learning space, on the other hand, describes interaction elements that support learning, including engagement with the modality of content, dynamic interaction, perceived value of technology features to support learning, and scaffolding. This view shares a common focus with the Axiom of Complexity regarding the interconnected and interdependent relationship between the learner, the designed intervention, and the learning context. Specifically, Tawfik et al. (2022) emphasize the importance of considering how learners perceive interface elements, such as the modality of content and the value of technology features, consistent with the Axiom of Complexity’s emphasis on the interplay between the learner, the designed intervention, and the learning context.

In summary, while descriptions of LXD differ in their focus and scope, they share several key similarities. Beyond a content-driven approach to design of learning environments, the view of LXD tends to emphasize the importance of designing learning experiences that are user-friendly, relevant to learners’ everyday lives, and goal-oriented. Additionally, many highlight the importance of taking a human-centered and learner-centric approach to learning design, which involves tailoring experiences to learners’ individual needs and preferences. Finally, LXD scholarship stresses the importance of incorporating broader theoretical principles and taking into account learners’ sociocultural contexts when designing learning experiences, which can promote deeper learning and more positive outcomes. Although theorists have proffered various ideas rooted in theory, to-date, no attempts have been made to investigate experts’ perceptions of these descriptions of the LXD phenomenon and, specifically, to explore their perceptions regarding what elements are characteristic for LXD practice.

Methodology

Participants

In line with the number of participants in similar eDelphi studies, researchers recruited 16 learning experience designers using convenience and purposeful sampling methods. The issue of sampling in a Delphi study is somewhat debated, with many researchers suggesting a combination of convenience and purposive sampling is appropriate in educational research given the study goals (He et al., 2021; McPherson et al., 2018; Pedrosa Carrasco et al., 2022). Initially, we utilized convenience sampling as a means to reach out to potential participants. We sent out emails to members of the Association for Educational Communications and Technology (AECT) and leveraged social media platforms to make initial contact. The decision to use convenience sampling for this phase of participant recruitment was based on the accessibility of individuals within the AECT community who might possess the expertise we were seeking for our study. From within this sample frame, purposeful sampling was employed to select participants who met specific criteria deemed relevant to our research objectives. Our criteria included the following factors: (1) an advanced degree, (2) significant contributions to scholarship and/or practice, and (3) recognized expertise regarding LXD. Although an ideal sample would have drawn equally from K-12, higher education, and corporate, the pool was likely biased toward higher education given the inclusion of participants from AECT, as evidenced by our resultant sample: twelve participants from higher education; one from K-12; one from corporate; and two who indicated “other” as their industry. Of the 16 participants, 11 were female and five were male. Fourteen participants resided in the United States, one resided in the Netherlands, and one resided in the United Kingdom. All participants were adults, each of whom signed a consent form prior to answering any survey questions.

Procedure

Developing initial Delphi instrument

The Delphi technique is designed to garner a consensus among ill-structured and undefined issues within a domain. The Delphi methodology is primarily a survey-driven methodology, but it is unique in that the initial instrument is amended in subsequent rounds as metrics identify consensus among participants. Hence, the “Delphi technique straddles the divide between qualitative and quantitative methodologies” (Thangaratinam & Redman, 2005, p. 11). The research approach thus provides a “flexible and adaptable tool to gather and analyze the needed data”(Hsu & Sandford, 2007, p. 5). Although the Delphi study is largely applied in the medical domain (Beattie & Mackway-Jones, 2004; Dekker et al., 2021), it has been used in the field of education around emergent concepts where it may be difficult to determine shared agreement, such as an initial definitions of instructional design (York & Ertmer, 2011) and early strategies for online learning (O’Neill et al., 2011; Zawacki-Richter, 2009).

In terms of instrument development for the Delphi methodology, Hsu and Sandford (2007) note that “it is both an acceptable and a common modification of the Delphi process format to use a structured questionnaire in Round 1 that is based upon an extensive review of the literature.” While the Delphi method can identify consensus where no established theory exists, a threat to validity includes the initial list of survey items. Hence, researchers recommend that researchers generate the initial items in an open-ended format, one of which includes a literature review (Hsu & Sandford, 2007; Thangaratinam & Redman, 2005). Towards validity of the instrument used in the current study, the context of this article derived initial survey questions from the systematic qualitative content analysis from Schmidt and Huang (2022) on LXD that consisted of the following perspectives: design, disciplinary, methodological, and theoretical. Upon finalization of the survey, participants were directly emailed the survey items, which presented 50 Likert scale questions, followed by an optional free-form text “Reason/Explanation” item.

Analysis

Metrics describing consensus are subject to debate given the balance of quantitative and qualitative approaches inherent within the Delphi methodology (Diamond et al., 2014; Thangaratinam & Redman, 2005). Whereas some studies suggest only the interquartile range may be sufficient (von der Gracht, 2012), others contend that both measures of centrality and dispersion are necessary for rigorous analysis (York & Ertmer, 2011). Based on an overview of the Delphi methods presented by Birko et al. (2015), an item was deemed as consensus if it met the following conditions: (a) interquartile range was less than or equal to 1, (b) standard deviation was less than or equal to 1, and (c) there was 80% agreement on a rating of 1–3 (disagree) or 4–6 (agree). Survey items that met the consensus thresholds during Round 1 were excluded in Round 2, and three questions were added or modified (Table 1).

Table 1 Table where LXD items reached consensus
Full size table

Results

RQ 1.a: Consensus of LXD design perspectives

Design perspectives are the approaches applied in LXD design practices, such as color design, contextual design, design thinking, and emotional design. Experts reached consensus with the general question as to how LXD “is used to describe design practice” (Q1) (IQR = 1.00, STDV = 0.85, Agreement = 80%). A more detailed analysis suggests areas of agreement for design perspective appear to (a) situate the learner at the forefront of design and (b) describe what is included as part of the design process (i.e., methods, processes, etc.). Other agreement items present a nuanced view as it describes LXD as human-centered (Q5) (IQR = 1.00, STDV = 0.63, Agreement = 93%), how learner perceptions influence performance (Q3) (IQR = 0.00, STDV = 0.68, Agreement = 93%), it accounts for learner differences (Q4) (IQR = 0.00, STDV = 0.86, Agreement = 80%), and the interaction with the learning environment (Q2) (IQR = 1.00, STDV = 0.50, Agreement = 93%). Collectively, these items often describe a profile of the learners that is driving the interaction. As to the specific design elements, participants agreed that LXD considers pedagogy, socio-emotional aspects (emotion [Q8], empathy [Q6]), and the broader contexts in which learning takes place (physical [Q7], contextual [Q10], social/group dynamics [Q12]).

The Delphi results provide additional insight as it identifies where consensus was not reached (see Table 2). It is important to differentiate items that approached consensus and those in which there was considerable disagreement. Those that approached consensus (e.g., close to the three thresholds) include items such as: “solicits learner input during the design process, but the design team makes final design decisions” (Q26) (IQR = 1.00, STDV = 1.03, Agreement = 73%) and “… conceives of learners as active participants in the design process who are valued as equal contributors” (Q31) (IQR = 1.00, STDV = 1.27, Agreement = 80%). It is noteworthy these items are largely focused on the degree to which learners are engaged within the design process. Some items in which consensus considerably diverged suggested LXD “… is used to describe design products (i.e., learning app, online learning environment, educational video game)” (Q30) (IQR = 2.00, STDV = 1.65, Agreement = 53%). As such, one might conclude that there was less consensus on items that referred to the design outputs compared with items about learners’ role in the design process itself.

Table 2 Table where LXD items failed to reach consensus
Full size table

RQ 1.b: Consensus of LXD disciplinary perspectives

Whereas design perspectives outline more procedural aspects, disciplinary perspectives are those disciplines that “contribute to and/or influence LXD,” such as HCI and instructional design ((Schmidt & Huang, 2022). Participants agreed that LXD draws from multiple disciplines (Q13) (IQR = 1.00, STDV = 0.70, Agreement = 93%). When asked about a specific definition, two of the definitions from the instrument met the threshold of agreement: “the practice of designing learning as a human-centered experience that leads to a desired goal” (Q16) (IQR = 0.00, STDV = 0.64, Agreement = 87%) and “an approach to learning design that foregrounds learners and their desired outcomes in a goal-oriented way, acknowledging individual experience” (Q18) (IQR = 1.00, STDV = 1.00, Agreement = 80%). That is, they agreed with the goal-oriented nature of learner-centered experience, as opposed to a broader and ill-defined view of LXD. Other areas of agreements seemed to reinforce this data, especially in terms of where the different disciplines might have emerged. For example, experts agreed that HCI influences learning effectiveness (Q14) (IQR = 1.00, STDV = 0.70, Agreement = 93%) and extends user-centered design towards a learner-centric view (IQR = 1.00, STDV = 0.95, Agreement = 80%). Beyond just conceptual perspectives that draw from various disciplines, a noteworthy area of agreement is that LXD requires specialized knowledge and skills that derive from multiple disciplines (Q15) (IQR = 1.00, STDV = 0.76, Agreement = 87%).

Of the original 18 items within the disciplinary perspective, twelve items did not reach the consensus among LXD practitioners. In terms of the specific disciplines, the following did not reach consensus: the intersectionality of learner-centered design and user experience design (Q36) IQR = 2.00, STDV = 1.50, Agreement = 73%) or alignment of HCI with learning processes (Q37)( IQR = 1.00, STDV = 1.19, Agreement = 60%). Similarly, participants also did not agree on questions that targeted defined LXD in terms of UX (Q39) (IQR = 1.50, STDV = 1.15, Agreement = 80%) or usability (Q47) (IQR = 2.00, STDV = 1.63, Agreement = 53%). Along these lines, participants disagreed that LXD defined a discrete field of study (Q38) (IQR = 1.00, STDV = 1.44, Agreement = 67%) or a unique phenomenon (Q46) IQR = 1.00, STDV = 1.25, Agreement = 67%). In terms of the focus of LXD, participants disagreed that LXD merely focused on tasks (Q42) (IQR = 1.50, STDV = 1.76, Agreement = 67%) or technologies specifically designed for learning (Q43) ( IQR = 4.00, STDV = 2.22, Agreement = 47%). Finally, they noted that LXD is not synonymous with job titles associated within the domain, such as “instructional designer” (Q44) (IQR = 1.50, STDV = 1.60, Agreement = 53%) and “learning designer” (Q45) (IQR = 2.00, STDV = 1.55, Agreement = 53%).

RQ 1.c: Consensus of LXD methodological perspectives

Methodological perspectives entail methods and processes applied in LXD practices, cognitive walkthrough, personas, prototyping. In this view, questions were focused on how LXD practitioners applied their processes, especially as it relates to evaluation. Respondents agreed with two items that appeared to reference the user-experience aspect of LXD: “…is informed by user experience design (UXD) methods” (Q19) (IQR = 0.75, STDV = 0.66, Agreement = 93%) and “… adapts user experience design (UXD) methods for learning design contexts” (Q22) (IQR = 0.00, STDV = 0.80, Agreement = 80%). Collectively, it appears as though participants agreed that methods are largely derived from UXD processes. As to when the methods happened, participants agreed that this happened in-situ.

As in the case of the design and disciplinary perspective, a further analysis of non-consensus items provides important insights as to how LXD is a unique entity. For example, questions related to whether LXD prescribes specific methods did not reach the threshold of consensus (Q49) (IQR = 2.50, STDV = 1.73, Agreement = 40%). One might thus conclude that LXD ‘adapts’ methods that ‘inform’ practice, but lacks specificity. This may suggest that those well versed in LXD may not feel as though there is a core set of methods that must be employed in a prescriptive way.

RQ 1.d: Consensus of LXD theoretical perspectives

Theoretical perspectives are specific theories explicitly applied by LXD practitioners to guide LXD practice, such as cognitive load theory, flow theory, activity theory, and social constructivism. There were considerably fewer items regarding this overarching construct relative to the other perspectives investigated. Similar to HCI-oriented questions found within the disciplinary perspective items, participants agreed that LXD “is guided by theories from the fields of human–computer interaction (HCI) and user experience (UX)” (Q23) (IQR = 0.75, STDV = 0.88, Agreement = 0.87). However, this was not exclusive, as participants also reached consensus with the following related to learning theories: “is guided by theories from the field of learning/instructional design and technology” (Q24) (IQR = 1.00, STDV = 0.80, Agreement = 0.87). This suggests that LXD professionals may adopt more of an eclectic stance towards theory, as opposed to a theoretically-pure or dogmatic approach.

Discussion

In prior years, the focus of ID often focused on the design and development of learning technologies. As the field continues to engage in discussions that extend learning beyond content (Hokanson et al., 2020), many scholars advocate for a broader view for the design and development of learning environments. Although the LXD term has gained traction among scholars and practitioners, Chang and Kuwata (2020) argue, “there is a need to provide a concrete definition of LXD to guide the conceptualization and practice of learning design” (p. 146). Indeed, initial conceptualizations have emerged that consider LXD in terms of usability (Gray & Boling, 2023; Tawfik et al., 2020) and socio-technical considerations (Jahnke et al., 2020), while others focus on the unique characteristics when using specific modalities (Oprean & Balakrishnan, 2020). As learning technologies evolve, it is important to extend conceptual discourse around LXD and empirically explore elements of LXD that might guide the field. To date, scholars have attempted to provide some empirical basis for LXD, such as in the form of grounded theory (Tawfik et al., 2022) or content analysis (Schmidt & Huang, 2022). This study extends these conceptual views as it researches perspectives about what elements research participants identified as characteristic of LXD. Given the data from practitioners, the following definition emerges for LXD from the results of this Delphi study: LXD not only considers design approaches, but the broader human experience of interacting with a learning environment. In addition to learners’ knowledge construction, experiential aspects include socio-technical considerations, emotive aspects (e.g., empathy, understanding of learner), and a detailed view of learner characteristics within context. As such, LXD perspectives and methodologies draw from and are informed by fields beyond learning design & technology, educational psychology, learning sciences, and others such as human–computer interaction (HCI) and user-experience design. Identifying LXD elements is important given that researchers have often looked outside of LDT to find conceptual guidance and methods (e.g., HCI), which can be problematic in that relying solely on views external to the field may limit the sophistication of evaluation and may not capture the full ecology of LXD and its related outcomes.

This eDelphi study aims to provide conceptual clarity by way of assessing consensus among SMEs for elements that are characteristic for LXD practice. Generally speaking, panelists tended not to agree on explicit prescriptions regarding methods and theory. Based on the results, we pose the question of whether this could be a call for the field to develop its own specific methods and theories related to LXD, as opposed to extensive reliance on external views that may not account for unique learning processes. Rather than draw from other tangential fields, the areas of consensus found in this eDelphi study provide directions for future research to consider how we might better focus design practice and theoretical work rooted in LXD. Below, we provide a more in-depth discussion in terms of the four LXD perspectives: (1) design, (2) disciplinary, (3) theoretical, and (4) methodological.

LXD design perspectives (RQ1.a)

Experts’ open ended responses regarding design perspectives were characterized by a range of diverse viewpoints and perspectives. In the following sections, we describe the areas in which consensus was reached, followed by areas in which consensus was not reached.

LXD design perspectives that reached consensus

Central to experts’ perspectives that reached consensus was a shared agreement that LXD is not solely rooted in theoretical concepts, but also in practical design processes within the field. In terms of advancement of understanding of elements of LXD among practitioners, participants focused on two areas s: (a) the effectiveness of designed learning interventions, and (b) the human experience of interacting with learning environments. One might argue that the field of LDT has traditionally emphasized the former, but is less focused on the latter. The finding presented above emphasizes the importance of considering practice and research related to learning-technology interactions. Beyond the consensus items, there were some notable disagreements regarding specific elements of LXD, such as design perspectives related to user-centered design and HCI.

General consensus was found regarding LXD’s design-oriented approach that goes beyond mere product or artifact-driven practices. This aligns with the growing evidence challenging the techno-centric view of learning technologies that persists within LDT (Schmidt & Glaser, 2021). By shifting towards a human-centered view and situating the learner experience within the broader context of socio-technical systems, the conceptual focus of LXD moves beyond the product alone and places greater emphasis on the individual within the learning process. An important shift involves considerations of how learners interact with the learning environment, both from a usability standpoint and as a means of facilitating effective learning experiences. Although some aspects of these LXD interactions have been explored (Johnson et al., 2022; Novak et al., 2018), there remains a dearth of knowledge regarding the specifics of these interactions that drive learning outcomes. For example, future research should further investigate the learning-technology interactions that are essential to collaborative technologies, particularly as many modern tools integrate social aspects of learning into their designs. Moreover, a comprehensive understanding of learner needs—including neurodiverse learners, underrepresented populations, and diverse organizational learning settings—is essential for a human-centered design approach.

LXD design perspectives that failed to reach consensus

In terms of identifying the elements of LXD among practitioners, a number of LXD design items failed to reach consensus when reviewed by experts. First, the degree of involvement of learners in the design process was an area of disagreement within the available data. While there was agreement on the importance of centering the learner and engaging them, the extent and timing of their involvement remains an open question as LXD advances. Regarding seeking learner validation throughout all phases of the design process, LXD experts expressed concerns regarding granting learners the final decision-making authority. Instead, experts seemed more comfortable with the design team retaining the responsibility for making final design decisions, but with the caveat that learners should be empowered to provide regular and valuable input. This approach may diverge from a "pure" co-design approach (Cavignaux-Bros & Cristol, 2020), as it falls short of actually giving learners decision-making authority. However, it does emphasize how practitioners can effectively engage learners in what may be better characterized in a participatory design manner. This indicates a recognition of extensively incorporating learner views in LXD; however, the specific degree of learner involvement and decision-making remains unclear, suggesting a need for further research.

Another noteworthy finding is that consensus was not reached regarding questions about improving usability and UX. Rather than merely replace “user” with “learner,” experts’ open-ended responses suggested that they conceived of LXD as not only focusing on improvements of UX and usability, but also more broadly with learning effectiveness and the human experience of interacting with learning environments. Traditionally, usability and UX have been discussed as aspects within existing models [e.g., perceived ease of use in the Technology Acceptance Model (Lemay et al., 2019)], but the findings underscore that LXD is more comprehensive in nature.

LXD disciplinary perspectives (RQ1.b)

LXD disciplinary perspectives that reached consensus

Consensus was reached among experts that LXD is more multidisciplinary when compared with previous approaches of instructional design, as evidenced by consensus about questions such as LXD “integrates perspectives of multiple disciplines in ways that can lead to new knowledge” (Q13) and “requires specialized knowledge, skills, and abilities from multiple disciplines.” (Q15). There was also expert consensus that LXD is more multifaceted compared to traditional views of ID (Q15), including empirical validity that it draws from disciplines such as HCI (Q14) and user-centered design (Q17). As in the case of the design perspectives, the eDelphi study participants’ responses suggest that they understand that the concept of LXD is more complex than merely replacing the term “user” with “learner.” Rather, expert responses suggest the presence of unique elements in LXD interactions that extends beyond the traditional emphasis of LDT. We argue that this integration is a natural evolution, considering that HCI often draws on cognitive psychology and our field frequently references theories such as distributed cognition (Angeli, 2008; Vasiliou et al., 2014) and activity theory (Barab et al., 2005; Jonassen & Rohrer-Murphy, 1999; Yamagata-Lynch, 2007). However, LXD disciplinary perspectives goes beyond merely overlaying frameworks or theories onto the learning setting; instead, it requires specialized knowledge, skills, and abilities rooted in these disciplines (Q15). Previous research on instructional designers has indicated that practitioners often implicitly reference theories or respond to contextual needs (Honebein & Honebein, 2014; Sentz et al., 2019). Although LXD experts identified the need for broader disciplinary perspectives, there remains a gap in understanding what these specialized LXD knowledge, skills, and abilities should entail. In addition, many instructional design programs often do not train on HCI theories and methods. This represents an ongoing discussion within the field and necessitates further study, potentially through qualitative or grounded theory approaches, to explore and define the unique expertise required in LXD.

LXD disciplinary perspectives that failed to reach consensus

It is worth noting that there was no consensus regarding items that characterized LXD as a discrete field of study, unique, and with its own characteristics, methods, and processes. This dovetails to a certain extent with an area of consensus, namely, that LXD is an extension of UCD and HCI. This is not necessarily a contradiction, but does call for nuanced interpretation. While experts recognize that LXD requires specialized knowledge and skills (as discussed earlier), they disagreed that it should be considered a completely distinct and separate field from more traditional ID (Q38). That is, LXD should not be seen as separate or opposed to ID but rather as an emergent perspective within an existing field. LXD emphasizes the importance of positioning the learner within a human-centered framework, aligning with the evolving nature of instructional design practices. This extension aligns with previous discussions, such as Tawfik et al. (2022) and their argumentation for “confluence” of interaction, along with Schmidt and Huang’s (2022) argument that LXD exists alongside ID as a complementary approach to learning design. Rather than considering LXD as entirely unique, this more nuanced viewpoint acknowledges its specialized nature, while recognizing its integration within the field of ID and its interconnectedness with other disciplines such as HCI, UCD, and UXD.

LXD methodological perspectives (RQ1.c)

LXD methodological perspectives that reached consensus

In terms of clarity around methodologies to evaluate LXD, expert responses identified consensus around the notion that LXD utilizes methods from many fields, particularly UXD, as opposed to methods being solely grounded in the traditional canon of LDT. While Schmidt et al. (2020a, 2020b) edited volume provides a single chapter that explores LXD methods, we argue that further sophistication and research are needed to develop methods specifically tailored to the more nuanced view of LXD. It is important to note that, comparatively speaking, fewer items in the eDelphi instrument were related to methodological perspectives, which in some ways is a representation of the source material (i.e., chapters in the edited volume) that informed the development of the instrument. This suggests that the edited volume on learner and user experience research—which was designed to be a seminal scholarly work—may have been representative of the nascent conceptual view of LXD. That is, it focuses more on design and theory, and more development is needed on valid and reliable instruments that are representative of LXD.

A specific methodological consensus point of LXD that was emphasized in the experts’ responses was the iterative nature of the LXD process [i.e., “LXD requires iteration for design and evaluation” (Q20)] and that LXD involves gathering data in situ [i.e., “LXD uses learner data collected in-situ to inform product development” (Q21). These points of consensus are important in that they foreground the importance of evaluation to iterative design which in traditional ID ADDIE often occurs as the last step in the design process (Branch, 2009), although this view appears to be shifting in recent learning design models (Reigeluth & An, 2020). In contrast to gathering user data in a summative manner, LXD emphasizes the importance of in-situ data collection and data-informed iteration. This implies that it may be beneficial for LXD to establish stronger connections with learning analytics and data mining techniques that are able to capture real-time learner data.

LXD methodological perspectives that failed to reach consensus

Experts failed to reach consensus on the item asking whether LXD prescribes specific methods (Q49). Open-ended responses noted that LXD draws from a variety of different design approaches and methods, underscoring its interdisciplinary nature. One expert expressed a more nuanced disagreement, acknowledging that while they personally use very specific methods, they recognize that different LXD researchers might employ different methods. However, they emphasized the importance of studying usability, user experience, learning effectiveness, efficiency, and student appeal, suggesting that these aspects can be assessed using various methods. Overall, there was no consensus on specific or prescriptive methods for LXD, which again highlights the diversity and flexibility of the approach.

Theoretical perspectives (RQ1.d)

LXD theoretical perspectives that reached consensus

Only two items in the eDelphi instrument were related to theoretical perspectives, both of which reached consensus (Q23, Q24). These focused on LXD drawing from theories both from within the field of LDT and outside the field. In line with other data presented above, there was consensus around LXD and its relationship to HCI and UXD. Open-ended responses suggested that LXD does not draw from any of these individually, but from all of them simultaneously and in a transdisciplinary manner. Therefore, these theoretical perspectives intersect and complement one other in guiding LXD practices. While there are existing theories that bridge these domains, such as activity theory (Barab et al., 2005; Jonassen & Rohrer-Murphy, 1999; Yamagata-Lynch, 2007), as the use of technology in learning environments continues to expand, it may be necessary to further explore and develop theories that specifically address the unique characteristics, and guide the methods and processes of LXD. This highlights the need to not only draw from HCI and UXD theories, but also to simultaneously draw from established learning theories. This coincides with emerging arguments that argue that LXD does not contradict traditional views of ID, but rather that LXD progresses alongside.

Limitations and future research

While the data presents some clarity regarding LXD, there are multiple opportunities to build on the findings. The Delphi study is designed to generate consensus with an established set of experts to understand what elements are critical to LXD, which allows drawing from a specific sampleset. The Delphi methodology and sampling strategy might present a potential for bias given the nature of participants; therefore, there is an opportunity to build on the study with alternative research approaches or a broader range of SMEs. For example, an observational study might explicate how individuals engage in LXD practice. In addition, one might explore the degree to which a broader sample agrees or disagrees with the final items presented within the current study. Although the number of SMEs align with other Delphi studies (Birko et al., 2015), the practitioners generally were skewed towards applied LXD research and practice within a higher education setting. It is therefore possible that some variation might emerge as the sample scales to other practitioners, especially if a study were to investigate LXD across different disciplines (e.g.—STEM) and domains (e.g.—higher education; workforce development).

A future study could also be done using a different threshold for acceptance. In the current research, an item had to meet three criteria: (1) IQR greater than 1, (1) standard deviation greater than 1, and (3) 80% agreement on specific items. To date, there is no predefined set of criteria to determine consensus, so prior studies might employ two criteria (Birko et al., 2015) or a lower threshold for acceptance (York & Ertmer, 2011). While the strict criteria that support construct validity, other studies might replicate the study using different thresholds of central tendency and agreement.

Another opportunity for future research relates to the initial corpus that served as the foundation for the Delphi items. As noted earlier, the study relied on the systematic analysis from Schmidt and Huang (2022), which was based on a specific edited volume on LXD. While the chapters are an attempt to serve as a seminal discussion on the topic of LXD, one might argue that the initial set of items could be derived from a focus group or systematic literature review. In terms of the latter, a traditional systematic review that included specific databases and search terms within the last 5 years might result in different items for the Delphi instrument. This could provide additional clarity, especially around some of the disciplinary and theoretical perspectives inherent within these databases.

Conclusion

Reiser (2007) notes that a failure to establish conceptual clarity can become a source of confusion and controversy within a field, leading to disagreements and misunderstandings among researchers. Moreover, it can also be difficult to outline research trends and gaps as individuals struggle to identify a common foundation within a domain. The current eDelphi study attempts to define aspects of LXD, demonstrating its multidimensional nature and its growth beyond content-focused views of design. As technology and learning environments continually adapt, the need for concrete elements of LXD that help to characterize it as a design philosophy becomes more pressing. Through the current eDelphi study, we found that consensus was reached in several areas, such as the multidisciplinary nature of LXD and its focus on the broader human experience of learning, but also that disagreements persisted around specific elements and the extent of learner involvement in the design process. Findings underscore the integrative nature of LXD, drawing from various related disciplines like HCI, UCD, and UXD. In addition, the lack of consensus regarding theory items suggests conceptual gaps about theories and models that should be used to inform and guide LXD practices. This eDelphi study not only provides a comprehensive overview of current and evolving ways to define elements of LXD, but also paves the way for future research to further elucidate and refine the phenomenon. In the face of technological evolution and shifting learning paradigms, defining LXD is not just an academic endeavor, but a step towards shaping the future of holistic, diverse, and impactful learning experiences.