Introduction

What kind of education is appropriate to effectively enable learners to contribute to sustainability? Considering that education is frequently framed as a “key enabler” for the achievement of all Sustainable Development Goals (SDGs, Rieckmann et al. 2017; UNESCO 2020; United Nations General Assembly 2017), answering this question is essential not only for Quality Education (SDG 4), but also for sustainability transitions in general (e.g., Abson et al. 2017; Otto et al. 2020). In the final 2021 declaration of the UNESCO World Conference on Education for Sustainable Development (ESD), the participating UNESCO member states acknowledge that “urgent action is needed to address the dramatic interrelated challenges the world is facing” (UNESCO 2021, p. 1) and therefore call for a comprehensive reorientation of education systems toward sustainability.

In educational practice, however, sustainability often tends to be approached as an “add-on” or “bolt-on”, not as a core component of quality education (e.g., Holst et al. 2024; Sterling 2003; Wals and Benavot 2017; Wals and Mathie 2022). In contrast, the idea of a whole system approach emphasizes that effective and coherent sustainability learning would require sustainability to be substantially integrated and practiced across all levels of education systems (e.g., in policies, administration, learning landscapes and communities, organizations and specific learning situations). As a coherent part of a whole system approach to sustainability in education, the terms Whole Institution Approach (WIA, all areas of education), Whole School Approach (for schools) and Whole-of-University Approach (for higher education institutions) address educational institutions and their surroundings, calling for schools, universities, kindergartens and other places of learning to “walk the talk” by mainstreaming sustainability within all activities (e.g., Buckler and Creech 2014; Henderson and Tilbury 2004; Holst 2023; McMillin and Dyball 2009; Mogren et al. 2019; Rieckmann et al. 2017; Sterling 2003; Wals and Mathie 2022). In the words of the UNESCO declaration, this implies that “institutions become living laboratories for participation and active citizenship” on sustainability, “allowing learners to learn what they live and live what they learn” (UNESCO 2021, p. 3). These appeals are supported by a wide range of scientific literature which suggests that the extent to which sustainability is experienced on a day-to-day basis within educational institutions (WIA) is strongly linked with the effectiveness of sustainability learning (e.g., Buckler and Creech 2014; Henderson and Tilbury 2004; Sterling 2003). The idea of WIAs (hereafter used as a generic term for all areas of education) thereby aims at a double objective: the continuous co-creation of coherent and empowering sustainability learning for all members of educational institutions, and at the same time, real-world impact for more sustainable societies (e.g., manifesting in less climate impact, protection of biodiversity, inclusivity).

While calls for WIAs as high-quality ESD have become a cornerstone of the international discourse on coherent education in times of (un-)sustainability over the past years (e.g., Bautista-Puig and Sanz-Casado 2021; Hargreaves 2008; Henderson and Tilbury 2004; Holst 2023; Lozano et al. 2015; Rieckmann et al. 2017; Tilbury and Galvin 2022; UNESCO 2020, 2021; Wals 2012), we are not aware of large-scale studies on (a) the degree to which sustainability is experienced in educational organizations (e.g., schools, higher education institutions) across all domains of WIAs and (b) how this relates to the effectiveness of sustainability learning. As Kohl et al. (2022, p. 229), among others, point out in the exemplary context of higher education, “more empirical research (…) is needed to fully understand the effectiveness and efficiency of the WIA”. To date, most international studies on the effects of WIAs have either relied on self-reporting data collected from educational leaders, which is prone to bias toward positive reporting (e.g., Nazir et al. 2011; Warner and Elser 2015), or have compared certified organizations with uncertified ones (e.g., Olsson et al. 2016). Particularly in the context of eco-school programs, the available evidence on differences between certified and non-certified organizations (e.g., on behavioral changes in learners) is mixed (Barratt Hacking et al. 2010; Boeve-de Pauw and van Petegem 2013; Goldman et al. 2018; Olsson et al. 2019). Both in school and higher education, tools for (self-)assessments of sustainability performance exist and are used by a range of institutions for reporting and organizational development (e.g., CRUE 2023; DCSF 2009; Fischer et al. 2015; Larrán Jorge et al. 2016; Urbanski and Filho 2015). Interestingly, few attempts have been made to systematically ask those who experience the practices and rules-in-use at their educational institution on a daily basis—learners and educators—about their experiences with regard to sustainability across the different dimensions of WIAs (an exception for educators: Mogren et al. 2019; a focus on perceived participation: Torsdottir et al. 2023). However, considering the double objective of coherent and effective sustainability learning and real-world impact, we argue that WIAs can only be viewed as fully implemented if sustainability is coherently practiced across all areas of action and if this practice is experienced by those within the organization, i.e., by learners and educators. Therefore, this article has a twofold objective in addressing the following research questions:

  1. A.

    How can the WIA be operationalized for quantitative assessments focusing on the experiences of those within educational institutions (learners, educators)?

  2. B.

    To what extent is the experience of sustainability at educational institutions (WIAs) related to key aspects of effective sustainability learning (e.g., empowerment, behavior, knowledge)?

In response to the first question, we present a systematically tested and validated instrument (WIA-Scale) in different lengths that can be used as a process indicator for high-quality sustainability learning. Regarding the second question, we present data from a first large-scale application of the WIA-Scale in Germany. As part of this, we provide data on the implementation in Germany as well as insights into the relationship between the WIA and various constructs related to effective sustainability learning. Among others, these constructs include learners' feeling of empowerment to contribute to sustainability (SDG 4.7, Rieckmann 2018; UNESCO 2020; United Nations General Assembly 2015), motivation to contribute to sustainability (e.g., Brundiers et al. 2021; Grund and Brock 2019; Haan 2010; Hickman et al. 2021), as well as knowledge and behaviors (e.g., Brundiers et al. 2021; Rieckmann et al. 2017; Sass et al. 2023; UNESCO 2020).

As a conceptual basis for operationalization, the following first section introduces core characteristics and components of WIAs to sustainability in education on the basis of international literature. In the methods section, we lay out our approach to the systematic development, pretesting and first large-scale application of the WIA-Scale. In the results, we report on the application of the WIA-Scale with 2,985 learners and educators from school education, vocational education, and higher education in Germany. In this, we report on quality criteria (e.g., validity, reliability), descriptive statistics and on relationships between the WIA-Scale and other scales related to sustainability learning. Last, we discuss the results and their main implications for further research, practice and policymaking.

Conceptual basis: Whole Institution Approaches to sustainability in education

By deliberately emphasizing the socializing effect of learners' experiences in everyday life (e.g., Barth et al. 2012), the idea of WIAs extends the learning and experience space of an educational institution beyond the learning content and the classroom into all areas of experience and action (e.g., campus, community, governance; for an overview, see Gericke 2022; Holst 2023; Tilbury and Galvin 2022; Wals and Mathie 2022). In doing so, the concept of WIAs links the socio-physical context of learning to sustainability-related learning objectives, thus calling for organizations and their members to rethink and redesign what may be considered their standard way of acting. Sterling (2003, p. 344) describes such an integrative orientation as an attempt by educational institutions to function as “a reflective ‘microcosm of a sustainable society’” in which space is provided for collaborative learning along the challenges and opportunities of co-designing sustainability (Buckler and Creech 2014; Gericke 2022; McMillin and Dyball 2009; Rieckmann 2018). As part of a recent systematic review, WIAs were described as “continuous and participative organizational learning processes aimed at institutional coherence on sustainability, consistently linking the formal and informal (hidden) curricula” (Holst 2023, p. 1015). In this process, educational institutions attempt to “mainstream sustainability as a fundamental principle within all activities” and in doing so, to “collaboratively switch the default mode of all social rules-in-use to sustainability” (Holst 2023, p. 1026f.). Conceptually, WIAs are thus closely linked to the frequently suggested importance of informal learning and the hidden curriculum—i.e., the interaction with the social and physical contexts of learning—for learning outcomes (e.g., Gramatakos and Lavau 2019; Hopkinson et al. 2008; Orr 2004; Winter and Cotton 2012). This holistic perspective on learning environments is consistent with latest theories on cognition, in which material environments are not only viewed as distant or passive external factors for cognitive processes, but also as much more directly and actively involved in learning and memorizing processes. In this vein, the perspectives of, e.g., “embodied cognition” (e.g., Fugate et al. 2019), “grounded cognition” (Barsalou 2008) and “tacit knowledge” (Kaaronen 2018; Polanyi 1966, 2009) strongly emphasize the role of material environments for learning (in its application to ESD, see Brock 2022). Findings in this context also show that learning is more effective when material learning environments are consistent with the learning content (Fugate et al. 2019). It is therefore adequate to understand material environments as direct co-constituents of learning processes instead of mere external influencing factors (Brock 2022).

As the conceptual foundation for the operationalization of WIAs within a quantitative scale, we build upon a recent systematic review of the international literature on WIAs to sustainability, in which 104 scientific articles, reports, frameworks and guidebooks were synthesized and a joint framework was developed to characterize WIAs across the different areas of education (Holst 2023).Footnote 1 The WIA-framework consists of a set of five core principles, six concrete areas of actionFootnote 2 and the underlying and interconnected culture of organizations (Fig. 1). As a first principle, institutional coherence is described in the literature as the central core of WIAs, referring to consistency between what is intended to be learned and what is practically lived (e.g., Mogren et al. 2019; UNESCO 2020). The second principle, continuous learning, points to the notion that a WIA is not a destination, but a continuous process of co-designing sustainability between the different stakeholders of an organization (e.g., Mathar 2013; Scott 2015). This links directly to the third principle of participation, which is described as the capacity of an organization to encourage and empower all stakeholders to take part in the process. At the same time, such capacity for participation involves the responsibility of individuals within the organizational community for taking on the challenging task of practicing sustainability (Ferreira et al. 2006; Henderson and Tilbury 2004). The fifth core principle refers to long-term commitment, pointing out that changes within educational organizations are often gradual, and thus require long-term efforts and a certain degree of flexibility and creativity. Together, these five principles describe the foundation of WIAs, which is put into practice within the different organizational areas of action (see Fig. 1):

  1. (1)

    participative and proactive governance and leadership, which empowers and supports a process of continuous institutional development in line with the principles of sustainable development (SD), involving the relevant stakeholders from all parts of the organizational community;

  2. (2)

    cross-disciplinary integration of ESD as an action-oriented, situated, holistic and emancipatory approach to curriculum and learning, in which social–emotional and cognitive learning are equally considered;

  3. (3)

    sustainability-oriented operations and campus management (e.g., buildings, campus, materials, food and drink, mobility), which provide real-world opportunities for collaborative sustainability learning;

  4. (4)

    close and reciprocally engaged embeddedness in local communities and inter-organizational networks as collaborative real-world learning grounds;

  5. (5)

    active capacity building for educators and non-teaching staff by fostering sustainability and ESD competencies, building support structures and prioritizing sustainability within human resource management; as well as

  6. (6)

    clear, consistent and active communication on (un-)sustainability both within the organization and toward the outside.

Fig. 1
figure 1

Overview of the Whole Institution Approach concept (adapted from Holst 2023). The framework includes a set of five core principles, six interconnected organizational areas of action, the underlying organizational culture (“Organizational Culture of Sustainability”) as well as critical external conditions. In higher education, research constitutes a seventh area of action, which is not displayed as it does not relate to all areas of education. ESD Education for Sustainable Development

Full size image

Importantly, the areas of action are viewed as intertwined and interdependent. Connecting all areas of action, the organizational culture describes the shared beliefs, values and meanings that underlie the system of rules-in-use within any given organization and is therefore viewed as of critical importance for WIAs. Lastly, the international literature points to several external conditions for successful WIAs (prioritization in policies, sufficient and long-term funding, access to expertise and support; e.g., Henderson and Tilbury 2004; UNESCO 2020; Wals and Benavot 2017).

As part of the systematic literature analysis, the conceptual categories derived from the international literature were used to develop a set of 53 items (Holst 2023), which served as the conceptual basis for the operationalization in this article. Considering that it is the core of WIAs to live sustainability within every fiber of an educational institution, we view the real-world experiences of those who learn, teach and spend their everyday lives on site—learners and educators—as a central source of reliable information on the actual implementation of WIAs. While many assessments of WIAs primarily focus on self-reporting by organizational leaders and the collection of “objectifiable” data (e.g., emissions), we argue that if nobody in an educational organization actually notices that it is in fact climate neutral, some degree of organizational sustainability may exist, but it should not be considered a WIA to sustainability education. In other words, a WIA to sustainability education can only be considered realized if individual and ideally collective sustainability learning takes place. This is in line with Luhmann (1989), who argued that environmental problems only have social relevance if social resonance arises through communication. Therefore, as an addition to the collection and evaluation of data on the de facto ecological and social footprint of educational organizations, the WIA-Scale introduced in this article focuses on the degree to which learners and educators actually experience sustainability in practice.

Materials and methods

The following sections introduce the development of the WIA-Scale (conceptual basis, qualitative and quantitative pretesting) for the assessment of experienced WIA implementation from the perspective of learners and educators and the study setup for a first systematic assessment in Germany (see Fig. 2). As part of the large-scale assessment with learners and educators from school education, vocational education and higher education (n = 2,985), the scale´s quality was tested (e.g., internal consistency, validity), descriptive statistics are reported and explorative correlation as well as regression analyses were conducted with other relevant scales (e.g., self-reported sustainability-related behavior, subjective knowledge regarding SD or perceived empowerment to solve sustainability issues).

Fig. 2
figure 2

Overview of the methods used for development, pretesting, validation and first large-scale application of the WIA-Scale. The conceptual basis for operationalization comes from a recent systematic literature review on Whole Institution Approaches (see Holst 2023). WIA Whole Institution Approach, ESD Education for Sustainable Development

Full size image

WIA-Scale: instrument development

The following sections provide an overview of the steps in the development and systematic pretesting of the WIA-Scale (on the process of pretesting, see Campanelli 2008; Collins 2003). After considering the conceptual basis of the scale in terms of its validity, the sections on qualitative and quantitative pretesting describe the measures taken to ensure high-scale quality.

Systematic literature analysis: conceptual foundation

The systematic analysis of the international literature on WIAs introduced above served as the conceptual basis for operationalization. As part of the literature analysis, the definitions for the identified facets and sub-facets of WIAs were translated into 53 items (see Holst 2023), which were refined on the basis of a written expert review of the representativeness of the items for the underlying constructs, relevance of the constructs within future assessments of WIAs and comprehensiveness (Olson 2010). In basing the scale development on a systematic literature analysis, high content validity can be assumed with regard to the thematic categories used for operationalization.

Qualitative pretesting: consolidation and comprehensiveness

The first list of items was consolidated in several iterative rounds. First, an explorative expert workshop was conducted with twelve participants from research on WIAs, educational practice, policy and administration, civil society organizations and youth participation as well as co-researchers on the prioritization of facets for assessment (Artino et al. 2014; Gehlbach and Brinkworth 2011). The items were then pretested in two rounds of cognitive interviews (Collins 2003; Willis 1999) with five learners and five educators from different types of schools and higher education institutions. To ensure that the vast majority of respondents are able to fully comprehend the questions, retrieve all necessary information, judge the specific concepts and have fitting options to voice their responses (e.g., Faulbaum et al. 2009), the focus was on participants who were likely to have had little prior knowledge of sustainability, as well as younger participants and participants from non-selective secondary schools and comprehensive schools. As part of the cognitive interviews, thinking aloud as well as paraphrasing and probing with additional questions on comprehensiveness were used (Collins 2003; Weichbold 2014). Items were left out which were found (i) not to link sufficiently to the real-world experiences of most learners and/or educators (e.g., on the development of action plans), (ii) to be conceptually too complex to translate into easy-to-comprehend statements (e.g., on a culture of solidarity and care) and (iii) to show considerable overlaps with other items (e.g., there were originally two items for internal and external visibility). The scale has been consolidated through this process and, at this point, consisted of 21 items for learners and 24 items for educators, which are both relevant and representative for all areas of WIAs and ratable for learners and educators.

Quantitative pretesting: demarcation of conceptual categories, scale improvement

To further demarcate the conceptual categories, assess correlations between items and adjust item difficulties based on the range of responses and their spread (e.g., Campanelli 2008; Weichbold 2014), the scale was subsequently pretested in a small-scale quantitative assessment with 48 learners and 29 educators from school education. In cooperation with the Ministry of Education in the federal state of Schleswig–Holstein (Germany), the scale was distributed among comprehensive schools as well as non-selective secondary schools. The quantitative pretest was conducted in schools instead of higher education or vocational education institutions to make sure that the scale also functions well with the youngest possible participants. The quantitative pretest showed that educators and learners used all response options (Range = 5) for most items, despite the relatively small sample. In addition, most mean values were between 2.5 and 3.5 with standard deviations of just over 1, indicating appropriate item difficulty and good differentiation between participants. None of the learners stated that they had a poor understanding of the questionnaire and even at this stage the scale showed a high internal consistency (αLearners = 0.903; due to the small sample size for educators, no calculation was made here). Individual items showed a much lower selectivity than others and were therefore excluded.

Finalization of the WIA-Scale

The final WIA-Scale consists of 13 items for learners and 15 items for educators (Table 1), which correspond to the six areas of actions (two for Curriculum and Learning, Management, Community, Governance, Communication and Capacity Building (only educators), respectively), as well as two items on Organizational Culture and one item on Participation. These items integrate both the need for practical usability (number of items) and content validity (comprehensiveness, high relevance and representativeness for WIAs). In terms of the different levels of indicators, all items can be regarded as process indicators, although some are also linked to other levels, such as input, output or outcome.

Table 1 Overview of the WIA-Scale for learners and educators sorted by constructs and sub-constructs
Full size table

In addition to the main WIA-Scale, the item selection presented in Table 1 contains two complementary items: “WIA-Coherence” as a possible one-item indicator for the WIA-Scale and “WIA-Sust-Motivation” to offer an outcome indicator on the effects of the everyday experience on learners motivation to act for sustainability. As part of the first large-scale application, correlations between the WIA-Scale and these complementary items were tested (see below).

Large-scale assessment in Germany

The final scale was used in a large-scale quantitative survey in Germany. The sample used for the final validation as well as the methodological approach to the exploratory analyses and the additional instruments used are presented below.

Study setup

The study was conducted as part of the German national monitoring on ESD. In total, 2481 learners (aged 14–24, M = 19.2 years; SD = 2.7 years) and 504 educators (M = 43.2 years, SD = 11.7 years) were questioned from all federal states via an online-access-panel (73 participants were previously excluded due to socially desirable response behavior; see Bundesamt für Naturschutz 2017; Winkler et al. 2006). The number of participants by federal state corresponds approximately to the population distribution in Germany. With regard to gender, the sample for educators is representative for the group (65.7% female, 34.3% male). Among young people, female participants are overrepresented (75.3% female, 23.6% male, 1.1% diverse/others). Learners were included from school education (47.3%), vocational education and training (15.5%) and higher education (37.2%), and educators from school education (82.3%) and vocational education (17.7%). On average, the educators in the sample have 14.5 years of work experience (SD = 11.1).

Statistical methods for testing quality criteria

The range, measures of central tendency (mean, mode and median) and standard deviation of all items as well as the overall scale were determined to describe the basic distributional characteristics of the scale. In addition, all items as well as the total scale were examined for normal distribution using graphical methods (histograms, Q–Q plots), numerical methods (skewness, kurtosis indices) and formal normality tests (Shapiro–Wilk; Razali and Wah 2011). The discriminatory power of individual items was determined using the corrected item–total correlation, and internal consistency (as an indicator of reliability) of the scale was assessed using Cronbach's alpha. All correlation analyses (e.g., to determine convergent validity with the help of frequency of contact with sustainability in a respondent’s own educational institution, correlating the one-item indicator with the overall scale) were determined using the nonparametric Spearman's rank correlation coefficient due to partially missing normal distribution of the variables. To determine the extent to which participants from schools and higher education institutions with a focus or profile on sustainability score higher on the WIA scale than participants from schools without a sustainability focus (criterion validity), the Mann–Whitney U test was used (again, a non-parametric test was chosen because not all variables are normally distributed). A confirmatory factor analysis was conducted to determine factorial validity using the cutoff values proposed by Hu and Bentler (1995) for CFI and TLI (> 0.95).

Statistical methods for exploratory analyses

Non-parametric statistical procedures were also used for the exploratory analyses due to partially missing normal distribution. Consequently, Spearman's rank correlation coefficient was used for the correlation analyses between the WIA-Scale and other relevant sustainability- and ESD-related constructs (e.g., perceived empowerment to make important contributions to the solving of sustainability issues, sustainable behavior, emotions regarding sustainability, well-being, subjective knowledge, perceived relevance of ESD (educators) and participation in ESD-related training (educators) (see instrument descriptions below).

As a last step, multiple regression analysis was calculated for the prediction of sustainable behavior of learners, replicating the analysis by Grund and Brock (2020) using the WIA-Scale instead of the ESD scale by Boeve-de Pauw et al. (2015), which focuses specifically on ESD (holism, pluralism) within class (examination of statistical requirements attached in the supplemental material).Footnote 3 Other predictors included in the model were: emotions and attitudes regarding sustainability, connectedness with nature and humanity, ecological problem awareness, age, gender and money at one’s disposal (for more details on the procedure and scale descriptions, see Grund and Brock 2020).

Further instruments used in validation and explorative analyses

In addition to the WIA-Scale and the measures used as part of the multiple regression analysis for the prediction of sustainable behavior (see above), further instruments are briefly described below.

Well-being was assessed using the WHO-5 questionnaire, which is widely used internationally (Topp et al. 2015). The five questions (6-point Likert scale) allow a time-efficient yet valid assessment of general well-being.

Perceived empowerment by one’s educational institution to solve issues related to SD was assessed using one item (5-point Likert scale) that inquires about the extent to which participants agree with the following statement: “I feel that my school/university enables me to make an important contribution to solving sustainability problems (climate change, loss of biodiversity, social inequality, etc.)” (Grund and Brock 2022).

The subjective knowledge on SD (learners, educators) and ESD (educators) was assessed by asking participants to subjectively grade their knowledge of SD/ESD (1 = very good, 2 = good, 3 = satisfactory, 4 = sufficient, 5 = poor, 6 = insufficient). This scale was recoded for all statistical calculations so that higher values represent more knowledge.

In addition to the assessment of ecological problem awareness for the regression analysis (see above), the assessment of problem awareness was approached with the help of two items using 11-level sliders: first, participants were asked: “How big do you think are the sustainability problems in the world?” (0 = no sustainability problems, 10 = extremely large problems that severely threaten all of humanity and ecosystems). Second, participants were asked about their expectations: "How much will sustainability challenges, such as climate change, species loss and social inequalities affect your personal life in the coming decades?" (0 = no negative effects, 10 = massive negative effects).

The frequency of contact with sustainability in the own educational institution is an item from a scale that records points of contact with sustainability in different areas of life (friends, family, media, etc.). Participants rated on a five-point Likert scale the frequency (1 = never, 2 = rarely, 3 = sometimes, 4 = often, 5 = always) with which they had encountered SD in their formal educational institution.

Educators were moreover asked about the relevance of ESD to teaching using a 10-level slider (1 = ESD has no relevance to my teaching practice, 10 = ESD is highly relevant to my teaching). The number of training sessions on ESD in which educators participated was captured for the last 5 years. In addition, perceived barriers for ESD implementation were assessed with the help of the item "What factors prevent you from integrating sustainability issues more strongly into your teaching?". 12 possible hurdles were named (e.g., a lack of teaching materials, knowledge, further training) to which respondents could respond dichotomously (0 = does not apply, 1 = applies). A sum score was formed to determine the extent of the perceived hurdles.

Results

Quality criteria and scale description

Scale description

As in the pretest, responses on the WIA-Scale were distributed across the entire scale spectrum (Range = 5). Mean scores for all items ranged from 2.51 to 2.95 (MWIA-Scale = 2.71) in the learners´ group and from 2.71 to 3.74 (MWIA-Scale = 3.00) in the educators´ group (see Table 2; for more detailed analysis, see supplemental material). Except for two items in the teacher sample, the median and the mode of all individual items is 3. With one exception in each case, the standard deviations of each single item is above 1 for learners and above 0.95 for teachers. At the aggregated scale level, the values are lower, as is to be expected (SDWIA-Scale = 0.75; SDWIA-Scale = 0.73). These statistics indicate appropriate item difficulty as well as good differentiation between individuals. Nevertheless, the normal distribution indicators (graphical and statistical) show that a normal distribution cannot be assumed in the present sample at the individual item level and at the scale level. Among learners, considerably more participants scored between 1 (“Fully disagree”) and 2 (“Disagree”) (19.1%) than between 4 (“Agree”) and 5 (“Fully agree”) (3.9%). Accordingly, the distribution is strongly right skewed (z = 4.57***). In addition, the midpoint of the scale for learners seems slightly overrepresented (nevertheless, the statistical analysis indicates that there is no kurtosis [z = 1.37]). The statistical indicators for teachers do not point to skewness [z = 0.52] or kurtosis [z = 0.12].

Table 2 Means, standard deviations, item–scale correlation and Cronbach's alpha of the total WIA-Scale and all individual items based on the assessment of learners and educators in Germany (n = 2985)
Full size table

Reliability

Internal consistency can be rated as excellent in both groups (αLearners = 0.913; αEducators = 0.937). This score cannot be further improved among learners by omitting an item. For the teachers, there would be a slight improvement in internal consistency to α = 0.940 if the item on educators as facilitators was left out. The high internal consistency is also reflected in high corrected item–total correlations: these are all between 0.509 and 0.714 for the learners. Among educators, the item on educators as facilitators again stands out with an item–total correlation of 0.325, which nevertheless does not fall below conventional minimums. The item–total correlations of all other items among educators are between 0.557 and 0.799.

Validity

The convergent validity of the overall scale was determined using correlation with an item that measures the frequency of contact points with SD in formal education. It is r = 0.493*** for learners and r = 0.473*** for educators. Both values are conventionally almost in the large effect size range (Cohen 1988), which indicates existing construct validity. For criterion validity, the extent to which participants from schools and higher education institutions with a perceived focus on sustainability score higher on the WIA-Scale than participants from institutions without a sustainability focus was assessed. The Mann–Whitney U revealed statistically significant differences with a medium effect size for the learners (d = 0.583; p < 0.001) and large effect size for the educators (d = 1.065; p < 0.001). To determine factorial validity, a confirmatory factor analysis (CFA) was conducted for a one-factor model using the maximum-likelihood estimator. Chi-square is significant for both groups (χ2[65] = 815.483, p < 0.001 for learners; χ2[90], = 629.421, p < 0.001 for educators). The fit indices are (slightly) below the conventional cutoff values for good model fit (> 0.95 for CFI and TLI; CFILearners = 0.947, TLILearners = 0.936, CFIEducators = 0.885, TLIEducators = 0.866). However, given the finding that single-factor models produce substantially worse fit indices on average than other models (Bonifay and Cai 2017), the theoretically derived one-factor model can be considered empirically confirmed. A CFA with individual factors for each area of action was not calculated due to a lack of degrees of freedom when calculating with less than three items for each factor.

Descriptive group differences

When comparing the responses of learners and educators, it appears that, with the exception of one item (Communication 2: internal and external visibility), teachers describe slightly stronger implementation of a WIA in their educational institutions on average (Table 2). The difference in the overall mean between the groups (MLearners = 2.707, MEducators = 2.998) has small effect size (d = 0.39). Among young people, male participants show a descriptively slightly higher overall mean score (M = 2.779) than female participants (M = 2.683). However, the effect size is so small (d = 0.097; p = 0.017) as to be practically negligible.

Strong correlation of pars-pro-toto indicator (WIA-Coherence) with WIA-Scale

Complementary to the WIA-Scale, a possible one-item indicator (pars-pro-toto) was developed (WIA-Coherence). For both learners and educators, strong correlations were found between WIA-Coherence and the WIA-Scale (rLearners = 0.717***; rEducators = 0.784***). In other terms, the WIA-Coherence item explains 51.4% of the variance in the WIA-Scale for learners and 61.5% for educators.

Strong correlation between WIA-Scale and motivation to contribute to sustainability

A second complementary item was used which asks for the degree to which learners and educators feel motivated by their daily experience at the educational organization to do more for sustainability (outcome: WIA-Sust-Motivation). Correlations with strong effect sizes were found between the WIA-Scale and the motivational effect of everyday life at the organization to contribute to sustainability for both learners (r = 0.644***) and educators (r = 0.665***). In other terms, the WIA-Scale explains 41.4% of the variance within the item on motivation to act for learners and 44.2% for educators.

Relationships between the WIA-Scale and other scales related to sustainability learning

Small or no correlations between WIA-Scale and problem awareness

To check for a possible effect of participants' problem awareness on their responses on the WIA-Scale, correlation tests were conducted with three separate scales on problem awareness: (1) a one-item scale on the perceived problem size of unsustainability (rLearners = − 0.052**, not significant for educators), (2) the expected future influence of unsustainability on one's own life (rLearners = 0.108***, not significant for educators) and (3) the perceived transgression of planetary boundaries (Steffen et al. 2015) for climate change (rEducators = 0.138**, not significant for learners) and biodiversity (rLearners = − 0.067**, not significant for educators). Across the three scales, correlations were either not significant or of small effect sizes with oppositional directions.

Positive correlations with perceived empowerment to make important contributions to the solving of sustainability issues, sustainable behavior, knowledge and well-being

The WIA-Scale strongly correlates with the perceived empowerment evoked by one´s educational institution to make important contributions to the solving of sustainability issues (r = 0.554***; for an overview of all significant correlations with r > 0.1***, see Fig. 3). Also, a medium to strong correlation was recorded between the WIA-Scale and self-reported sustainable behavior (rLearners = 0.295***, rEducators = 0.401***). We further found for both learners and educators that higher scores on the WIA-Scale are associated with higher self-judgments for subjective knowledge on SD (rLearners = 0.166***; rEducators = 0.240***) and higher perceived well-being (rLearners = 0.149***; rEducators = 0.187***).

Fig. 3
figure 3

Schematic overview of significant correlations between the WIA-Scale and other scales assessed as part of the explorative analysis with r > 0.1***. Constructs indicated with 1 were only assessed with one of the respective groups, i.e., learners or educators. SD Sustainable Development, ESD Education for Sustainable Development

Full size image

Experienced WIAs and educators: higher perceived relevance of ESD, more participation in ESD-training and fewer perceived barriers for ESD implementation

A correlation with moderate effect size was found between the WIA-Scale and the perceived relevance of ESD for educators (r = 0.342***). Also, correlations were recorded with the number of training sessions on ESD in which educators participated (r = 0.258***) and the responses on the scale were associated with higher self-judgment on subjective knowledge on ESD (rEducators = 0.353***) and fewer perceived barriers for ESD implementation (rEducators = − 0.244***).

Sustainable behavior: WIA-Scale is the strongest predictor in regression model

The final part of the analysis replicated the multiple regression analysis of Grund and Brock (2020). While Grund and Brock operationalized ESD implementation through two facets of ESD practice (holism and pluralism, see Boeve-de Pauw et al. 2015), this study used the WIA-Scale instead. We found that experienced WIA implementation was the strongest predictor for self-reported sustainable behavior of learners (b = 0.272) before sustainability-related emotions (b = 0.218), connectedness to nature (b = 0.174) and problem awareness (b = 0.110) (Fig. 4). While the explanatory power of all other predictors remains largely unchanged when using the WIA-Scale instead of the ESD (holism and pluralism) scale, the overall predictive power of the model increases from 23.8 to 28.7%. As some of the items within the behavior scale could be interpreted by default as being part of daily experience at the educational organization, the model was calculated again to include only items which overarchingly focused on activities outside the organization (informing others via social media, donating for charitable purposes, engagement in a group/organization for SD), resulting in an overall lower predictive power of the model (13,5%), in which the WIA-Scale remained the strongest predictor with an even higher b-value (b = 0.312).

Fig. 4
figure 4

Schematic representation of the multiple-regression analysis on sustainable behavior of learners (n = 2385, 14–24 years; replication of the analysis by Grund and Brock (2020), including the WIA-Scale as a predictor). The model reveals that the degree to which learners experience sustainability in line with a Whole Institution Approach (WIA) is the strongest of various predictors of how sustainable learners act. Other significant predictors include emotions regarding sustainability, connectedness with nature, and ecological problem awareness as well as attitudes regarding sustainability and age of respondents

Full size image

Discussion: Whole Institution Approach—measurable and effective for sustainability learning

A strong link between the extent to which sustainability is experienced on a day-to-day basis across all domains of educational institutions (WIA) and outcomes of sustainability learning has long been suspected (e.g., Buckler and Creech 2014; Kohl et al. 2022; McMillin and Dyball 2009; Sterling 2003; Tilbury and Galvin 2022). In this article, we presented the development, validation and first large-scale application of a quantitative WIA-Scale. The scale is able to systematically capture learners' and educators' perceptions of everyday sustainability in line with WIAs and can be used as a process indicator for high-quality sustainability learning. Among others, the findings provide strong indications that the degree to which a WIA to sustainability is experienced by learners and educators is indeed of substantial importance for effective sustainability learning. In the following, we

  1. (1)

    review the quality of the developed and validated WIA-Scale,

  2. (2)

    discuss the results of a first large-scale application in Germany,

  3. (3)

    reflect upon why WIAs are strongly related to effective sustainability learning and

  4. (4)

    consider opportunities for future use of the WIA-Scale in research and practice.

Measuring WIAs: scale development and validation

The WIA-Scale was developed based on a systematic review of the international literature on the characteristics of WIAs, which included an expert review on the relevance and comprehensiveness of the various facets (Holst 2023). High content validity of the scale can therefore be assumed. In several rounds of pretesting, the scale was refined to function both as a solid representation of WIAs (content quality) and as an easy-to-comprehend instrument for both learners from 14 to 24 years and educators across school education, vocational education and higher education. Its first large-scale application in Germany (n = 2,985) showed good item difficulty and high internal consistency. Also, good convergent, factorial and criterion validity were determined as part of the validation process. Overall, a high scale quality can be assumed on the basis of the presented first large-scale application. The observed skewness of the distribution (more individuals are at the low end of the scale than at the high end) can be seen as a strength of the scale, as it allows for high selectivity without ceiling effects, even if real WIA implementation is increased in the future. As the data was collected only once, test–retest reliability is still to be assessed.

Assessment of WIAs in Germany: first large-scale application of the WIA-Scale

With regard to the current implementation of WIAs in Germany, our results show that there are only very few learners (3.9%) and educators (8.9%) who, on average across the WIA-Scale, agree or strongly agree that sustainability is currently being practiced in their organization. In contrast, around one-fifth of the around 2,500 learners answered on average between “disagree” and “fully disagree”. This signals that at present, wide implementation of WIAs is rather an aspiration (e.g., Nationale Plattform BNE c/o BMBF 2017; Tilbury and Galvin 2022; UNESCO 2020, 2021) than lived practice. At the same time, the presented data indicates that different learners and educators already experience different degrees of sustainability in line with WIAs, which also underpins why it is important to approach the WIA as a continuous learning and development process rather than a specific status that is to be achieved (Bosevska and Kriewaldt 2020; Holst 2023; Mathar 2013; Shallcross et al. 2007). Also, each organization is likely to follow a very specific pathway toward a WIA (see Birney and Reed 2009; Bohunovsky et al. 2020; UNESCO 2012), which is why the WIA-Scale may, as an addition to critical assessments of sustainability across all activities of the organization, help to take stock and identify pathways for organizational development. Regarding differences between learners and educators, the analysis showed that educators tended to evaluate their educational organizations as more sustainable than learners. Although all data were controlled for social desirability, these differences were particularly present in areas of action where educators are directly involved (e.g., in curriculum and learning, community cooperation, operations and campus management).

In terms of the sampling strategy, the data presented is quasi-representative for Germany. While no norm data can be derived from the sample structure (by definition online access panels are not strictly representative for all subgroups of the population), the data is compiled along diversification criteria such as areas of education, federal states, age or job experience of teachers. One methodological limitation is that female learners are overrepresented due to the providers' panel composition. Interestingly, the medium to low mean of the overall scale in Germany does not substantially differ between any of these subgroups. This may be explained by both the overall low implementation status and the sample structure, in which no full organizations were assessed but only individuals from various organizations. While the scale may be used within single institutions to deepen the perspective on specific cases, cross-institutional sampling such as the data presented here can contribute to WIA-data on more aggregated levels such as regions, nations or even at the international level. This sampling strategy may also help to explain the divergence between our findings (e.g., on the relationship between WIAs and motivation, empowerment and behavior) and studies that have compared certified with non-certified organizations and often found limited effects (e.g., Boeve-de Pauw and van Petegem 2013; Olsson et al. 2019; Spínola 2015). While these studies offer information on certifications, they mostly cannot address correlations between experienced WIAs in general across diverse institutions and measures for effective sustainability learning. In other words, while certificates may be important for promoting organizational development, the possession of a certificate does not necessarily mean that sustainability is actually practiced and experienced more coherently. As another effect of the cross-institutional sampling, we did not find strong mean differences between the six areas of action, which is likely due to cross-institutional variance, and more such differences could be expected when the scale is used within individual organizations.

Effectiveness of WIAs for sustainability learning: findings and probable explanations

As part of the first large-scale application in Germany, we assessed the relationship between experienced sustainability across all domains of WIAs and measures related to effective sustainability learning. In the following, we summarize the core findings and subsequently discuss explanations for the (likely causal) links between perceived WIA implementation and effective sustainability learning.

As the results indicate, there is a strong relationship between how sustainable learners and educators perceive their organization and how motivated they feel to act for sustainability as a result of their daily experiences there. Also, learners who respond higher on the WIA-Scale feel considerably more empowered by their organization to contribute to solving the sustainability issues of our times. Further, learners and educators who perceive their organization as more sustainable view themselves as more knowledgeable on SD and describe overall higher well-being. With regard to educators, those who respond higher on the WIA-Scale also view the educational concept of ESD as more relevant, participate more frequently in ESD training and see fewer barriers for its practical implementation. Lastly, multiple regression analysis including various other measures such as connectedness to nature, emotions and attitudes regarding sustainability, problem awareness, age and income revealed that experienced sustainability at the educational organization in line with a WIA was the strongest predictor for how sustainably young people report to be acting. This was also found when only items specifically related to activities outside the institution were included, making a spillover effect seem very likely. Interestingly, only small or no correlations were observed between respondents' problem awareness with regard to (un)sustainability and their answers on the WIA-Scale, suggesting that the scale seems to function independently from the degree of problem perception. One explanation could be that higher problem awareness may both lead to sustainability efforts being noticed more and to them being judged more critically—two effects which may mutually balance each other out. While the results are correlational (no longitudinal analysis) for which causal relationships seem plausible (see below), some of the measures used included the direction of causality within the items (institution-induced motivation and empowerment). Overall, the findings provide strong indications that the extent to which sustainability is experienced on an everyday basis across all domains of WIAs is of substantial importance for effective sustainability learning.

For this suggested (causal) link, we see several probable explanations. First, the salience of social norms of sustainability in the educational institution is likely to influence what feels important to do and prioritize. This is in line with a wide field of psychological research on the influence of social norms on, for example, sustainable behavior (e.g., Cialdini and Jacobson 2021; Fritsche et al. 2018; Keizer and Schultz 2018). Transferred to the context of education, the socializing norm systems and rules-in-use that are experienced on a daily basis seem foundational to coherent and effective sustainability learning (see Barth et al. 2012; Holst 2023). This is consistent with insights from research on organizational cultures, in that the design of the surroundings and everyday practices (artifacts, symbols) always implicitly conveys meaningful information on underlying values and assumptions (Alvesson 2013; Schein and Schein 2017). A further explanation lies in the notion of “embodied” or “grounded” cognition (e.g., Barsalou 2008; Fugate et al. 2019), i.e., in approaching material environments as critical co-constituents of learning (Brock 2022). In linking the context to learning, the WIA therefore is in line with a reprioritization of the physical environment in its role for learning, making it not an external precondition for effective learning settings, but rather an important co-constituent of it (Brock 2022). Following that, the involvement of learners in reflecting on, and collectively designing learning environments would be conducive for effective learning. The importance of the way in which environment and rules become habits also ties into the findings of Kahneman (2003), who emphasized the importance of unreflective, habituated, more intuitive actions (automatic mode) compared to the comparatively few actions in everyday life that are based on more effortful cognition or deliberation. Also, a changed perception that structural changes toward more sustainability are possible through visible positive examples in one's own environment may counter hopelessness and disillusionment, which can be inhibitors for action (Grund and Brock 2019; Landry et al. 2018). Related to this, experiencing a WIA implies that learners experience (more) consistency between awareness and action, which may be associated with a reduction in internal dissonance and may also explain part of the observed positive correlations with well-being of learners and educators. Moreover, experiencing sustainability processes in which—ideally—own perspectives are relevant, heard and implemented as part of a collective effort may lead to an increased (self-)efficacy belief, both on the individual and collective level (e.g., Bandura 1997; Jugert et al. 2016). This may also lead to an increased sense of responsibility and agency (e.g., Emirbayer and Mische 1998), which can be viewed as a particular challenge in the field of sustainability given the complexity of sustainability challenges and tendencies toward pessimism about the future (Grund and Brock 2019; Ojala 2016). Interestingly, looking at the suggested link between experienced WIAs and self-described motivation and empowerment, a positive reinforcing feedback loop can be expected: motivated and empowered learners and educators can also be assumed to be more likely to make their institution a place where the WIA is implemented.

Future use of the WIA-Scale: indicator, monitoring and organizational development

Concerning future use, the WIA-Scale may be utilized by both researchers and practitioners, e.g., for monitoring purposes as a process indicator for consistent high-quality ESD or as part of organizational development processes (e.g., for critical self-evaluation). The data from Germany can serve as a comparison for future assessments within educational organizations or at a more aggregate level such as within communities, states, countries or larger regions. As the scale was pretested and refined as part of the German national monitoring on ESD, context-specific adjustments are recommended for its use in other contexts. In future studies, the scale could be complemented with open questions, e.g., on drivers of and barriers to organizational change toward sustainability. Also, further research could combine the WIA-Scale with conventional sustainability assessments to explore how the perceptions of learners and educators match with indicators commonly used in sustainability reporting. With regard to the two groups—learners and educators—we would generally recommend questioning both, as these different perspectives may be fruitful, e.g., in a process of organizational development. If used as an indicator for high-quality ESD, the perspective of learners may be considered even less susceptible to self-reporting biases. Moreover, this also allows one to directly grasp an output perspective on WIAs: how it is perceived by and how it motivates the “main target group” of educational institutions. Considering the high internal consistency of the scale, it is possible to use several variants depending on the use case. Here, we presented the WIA-Scale (13/15 items), while a shorter variant with 7/8 items that correlates very highly with the presented scale (rLearners = 0.962***; rEducators = 0.977***) is provided in the supplementary material (S2). Both encompass all areas of action of the WIA and can be used for independent assessments or self-evaluations. Further facets for an extended version (e.g., for in-depth self-assessments) can be found in the supplementary material to Holst (2023). Moreover, we suggested a one-item indicator (“WIA-Coherence”) that correlates highly with the WIA-Scale. This item may be useful when there is a strong limit to the number of items that can be assessed. For such standalone use, examples of areas of action (see the section on the conceptual basis of WIAs) should be provided in brackets. Also, further studies are required to test the robustness of the correlation between the one-item indicator with the overall WIA-Scale with respondents who have not answered the rest of the scale before, as the previously answered items and the response order might influence the answers.

Conclusion and outlook: toward high-quality ESD through WIAs

To foster coherent high-quality sustainability learning, the concept of WIAs calls for educational organizations to set the default modes of all activities and rules-in-use on sustainability (Buckler and Creech 2014; Holst 2023; Rieckmann et al. 2017; Wals and Mathie 2022). The first large-scale application of the introduced and validated WIA-Scale indicates that the degree to which sustainability is experienced on an everyday basis is of substantial relevance for effective sustainability learning. This effect can be assumed to exist beyond the large-scale sample in Germany. The results underline the importance of not just adding sustainability as another learning content to be conveyed, but to instead practice it on an everyday basis across the different domains of the educational institution. In other words, high-quality ESD is strongly linked to the realization of WIAs. Considering the importance of education for achieving the SDGs, we argue that WIAs to sustainability are a critical component of twenty-first century quality education in general (Laurie et al. 2016; also UNESCO 2020). Implementing WIAs in practice requires, among other things, the establishment of favorable policy frameworks that provide orientation as well as financial, material and personnel support for the collaborative development of educational institutions as learning and experience spaces for sustainable development (e.g., Tilbury and Galvin 2022; UNESCO 2020; Wals and Benavot 2017).