According to recent perspectives, burnout is a chronic feeling of exhaustion that appears in working or in occupational settings, and it is triggered by an extended exposure to work stressors (e.g., interpersonal stressors, work-content stressors, job-specific demands, etc.). Burnout can occur as a result of various types of ‘work’ including schoolwork, parental activities, or volunteering. This is possible because all these forms of work involve a personal effort to achieve a goal, which makes them similar to ‘classical’ work. The contemporary understanding of burnout focused on three components that may or co-occur:1) exhaustion (i.e., a feeling of chronic fatigue and stress), 2) cynicism (i.e., a detached approach towards people from work and work itself), and 3) lack of professional efficacy (i.e., concerns a feeling of reduced accomplishment and performance about one’s job) (Maslach & Leiter, 2016). These burnout manifestations lead to apathy towards work and even loss of meaning and interest regarding work.

More recently, burnout was acknowledged in the 11th Revision of the International Classification of Diseases (ICD-11, World Health Organization, 2018) in the section named “problems related to employment or unemployment”. In the ICD-11, burnout is described as a syndrome that occurs in the occupational context but is not classified as a medical condition. Instead, burnout is classified as a syndrome that results from unsuccessful management of chronic workplace stress and has three dimensions: (1) exhaustion; (2) mental distance from one’s job, or feeling cynical about one’s job; and (3) reduced professional efficacy. This ICD-11 definition developed from previous definitions of burnout and preserved the terminology and the three dimensions from Maslach’s inventory. The present ICD-11 definition is concurrent with Maslach’s understanding, but this time the target is put on the multi-factor syndrome and the possibility of triggering or aggravating each other. Hence, ICD-11 classifies burnout as a mental health problem that can occur both during unemployment and employment.

Although burnout is originally a work-related concept, studies investigated the occurrence of burnout in non-work settings such as school and university contexts (Law, 2007; Aguayo et al., 2019; Lin & Huang, 2014; Rosales-Ricardo et al., 2021; Shin et al., 2011) compared university activities with work activities and concluded that the two are similar. University students must attend classes, complete assignments, take tests, do presentations in front of the audience, and these activities can be regarded as “work”. Schaufeli and Taris (2005) reiterated the idea that burnout is not limited to human services and can occur in other general contexts other than work, such as school, volunteering, etc. Previous evidence suggested that, as compared with other education levels, higher education is more challenging and more exhausting for university students because the academic demands increase and generate academic pressure, assignments overload, and long work hours (Law, 2007; Aguayo et al., 2019). These challenges are generators of academic burnout and may raise emotional exhaustion, negative attitudes, and low personal accomplishment in students (Lin & Huang, 2014). In a recent meta-analysis, the prevalence of burnout in medical and non-medical university students was measured, resulting in a predominance of emotional exhaustion, cynicism, and reduced academic efficacy in all analyzed studies (Rosales-Ricardo et al., 2021).

Similarly, in a sample of school students, burnout was characterized as a reaction to exhaustive school demands and can generate a detached state of mind regarding school and feelings of inadequacy (Salmela-Aro et al., 2009). The most typical consequences of burnout syndrome in both school and university students are school dropout, decreased academic performance, lowered engagement, and low levels of academic achievement (Bask & Salmela-Aro, 2013; Caballero et al., 2015). In a recent meta-analysis, Madigan and Curran (2020) examined the relationship between school and university student burnout and academic achievement. Their results show a significant negative association between burnout and academic achievement, at different education levels (i.e., school, college, and university). Thus, burnout is experienced and occurs similarly for both school and university students. In university settings, given the increased academic demands, academic pressure will increase the risk of developing burnout symptoms in the case of university. Finally, Dyrbye and colleagues (2008) used a longitudinal approach to explore the relationship between suicidal ideation and burnout in medical university students and found that the suicidal ideation over the following year is predicted by the quality of life, depressive symptoms, and burnout level in the previous year. Given its high prevalence rates (Rosales-Ricardo et al., 2021) and its negative consequences, the assessment of university student burnout should be rigorous and similar to practice and have a constant involvement from mental health professionals and researchers.

Today, the Maslach Burnout Inventory (MBI – Maslach et al., 2017) is one of the most frequently used measures of burnout (Maricuțoiu et al., 2016), though recent authors (Schaufeli et al., 2020b) argued that the utilization of MBI has several conceptual, methodological, and practical disadvantages. A recent study developed a new burnout conceptualization to overcome some of the MBI limitations (Schaufeli et al., 2020b). In the present study, we investigated the factor structure and the validity of this new burnout questionnaire (i.e., the Burnout Assessment Tool - BAT; Schaufeli et al., 2020a, 2020b) and adapted it to a student population. Because the BAT was originally developed for the working population, little is known regarding its psychometric properties when it is used in a higher education context. Furthermore, through the present study, we added new evidence regarding the measurement validity of the BAT (Schaufeli et al., 2020a) by investigating its relationships with prospective appraisal tendencies and with coping strategies.

A new conceptualization of burnout

As mentioned earlier, the MBI (Maslach et al., 2017) is one of the most used self-reported measures for assessing burnout. However, recent developments in the field of burnout identified some limitations regarding how the MBI operationalized burnout (Schaufeli et al., 2020b). These limitations include conceptual, theoretic, psychometric, and technical issues regarding the use of the MBI and some practical difficulties in interpreting the MBI scores. First, Schaufeli and his colleagues (2020b) argued that MBI was developed only as a research instrument, without a clear theoretical conceptualization. Secondly, Schaufeli and his colleagues (2020b) observed that existing burnout components were outdated and did not include new symptoms, such as the cognitive impairments associated with the development of burnout. These cognitive impairments include diminished working memory and poor concentration capabilities, which are present from the early stages of burnout (Deligkaris et al., 2014). Therefore, the operationalization of the core of burnout should also include these new components. Thirdly, Schaufeli and his colleagues (2020b) argued that the MBI had some psychometric problems, such as reliability problems due to some item’s extreme wording that generated skewed response distributions. Finally, the MBI generates three separate scores for each subscale, not a global burnout score that can assess the characteristics of a syndrome (Schaufeli et al., 2020b). This creates ambiguity because of the disparity between the MBI manual and the actual instrument. For example, in the manual, burnout is described as a syndrome that contains a set of correlated symptoms. However, the instrument conceptualized burnout using only the three subscales, but not the total score. Briefly, the MBI does not have a total burnout score because it was not conceptualized as a syndrome.

Based on these observations, Schaufeli and his colleagues (2020b) developed a new instrument named Burnout Assessment Tool (BAT). BAT’s theoretical conceptualization of burnout advances a combination of unwillingness (i.e., lack of motivation) and inability (i.e., lack of energy) to perform the tasks required by one’s occupation. The BAT is a 23 items self-report questionnaire that measures the core symptoms of burnout as feelings of exhaustion, mental distancing, emotional and cognitive impairment. The exhaustion component is characterized by a severe loss of energy which results in feelings of tiredness and weakness (physical exhaustion) and feeling worn-out and drained of energy (mental exhaustion). More specific symptoms include an incapacity to relax after the work is completed, lack of energy for starting a new work, feeling drained after a full workday, feeling quickly tired though the work necessitated minimal effort. Mental distance is characteristic to a person that distances psychologically from work, indicating a marked aversion towards work. The consequences are expressed by mental and sometimes physical withdrawal from work: avoidance of contact with colleagues or clients, a cynical attitude, indifference, feelings of autopilot functioning, a lack or little interest or enthusiasm for work. Emotional impairment manifests in a feeling of being overwhelmed by one’s own emotions and intense emotional responses, with specific symptomatology surrounding feelings of being angry and frustrated at work, unable to control the emotions at work, overreacting, irritability, and a feeling of being upset for not knowing why. On the contrary, cognitive impairment manifests in cognitive deficiencies related to memory troubles, reduced cognitive performance, concentration, and attention deficits. The explicit symptomatology includes difficulties maintaining focus at work, concentration and attention shortages, absent-mindedness and forgetfulness, indecisiveness, and struggles with clear thinking and learning new information at work. These BAT scales were developed after interviewing several professionals who worked daily with burned-out employees. After the interviews, the cognitive and emotional impairment components were added as core burnout components, along with the “traditional” components (i.e., exhaustion, mental distancing, and professional efficacy). Because the third traditional component of the MBI was mentioned in the interviews, as suggested in the literature (which shows that professional efficacy does not correlate well with the other two components; see Worley et al., 2008), this component was not included in the BAT scales. Additionally, the BAT has ten additional items for assessing secondary burnout symptoms: psychological distress and psychosomatic complaints (Schaufeli et al., 2020b). These dimensions were selected because the lack of energy has effects on the burned-out individual and can lead to a reduced capacity in regulating the emotional and cognitive processes. The interviews with professionals provided additional information which materialized in the secondary symptoms of burnout. Psychological distress describes non-physical symptoms that are the result of some psychological problems as worrying, feeling anxious, tense, disturbed by crowds and noise, weight fluctuations, and sleep problems. Psychosomatic complaints refer to physical symptoms without a medical cause and cannot be explained by a physiological illness. These symptoms are generated or exacerbated by a psychological problem and are represented by headaches, muscle pains, chest pain, and palpitations, sickly or often getting sick, and gastrointestinal problems.

Existing evidence regarding the BAT

Despite its recency, the BAT was already adapted to European working respondents from Russia (Kolachev et al., 2019), the Dutch speakers from the Flemish region of Belgium (Schaufeli et al., 2020a), Italy (Spagnoli et al., 2021), and Romania (Oprea et al., 2021). Also, adaptations of the BAT were conducted in non-European cultures such as Japan (Sakakibara et al., 2020) or Ecuador (Vinueza-Solórzano et al., 2021).

De Beer et al. (2020) showed that the BAT has good cross-sectional validity across respondents of seven nationalities from European and Asian countries, namely Austria, Germany, Belgium (Flemish region), The Netherlands, Ireland, Finland, and Japan. Their findings indicated that assessing burnout with the BAT provides a theoretically and statistically stable conceptualization, which is comparable from one culture to another. In a similar vein, Hadžibajramović et al. (2020) analyzed BAT’s psychometric properties using the Rasch measurement model. These authors examined the construct validity applying Rasch analysis, combined the four subscales into a single score of burnout, and investigated the differentiation between item functioning concerning age, gender, and country. Therefore, the results indicated that the measurement criteria are met according to the Rasch model, and the overall score can be measured through the summarization of the items of the four subscales into one single burnout score (Hadžibajramović et al., 2020).

Despite its growing popularity in the working populations, there is little evidence regarding the psychometric properties of the BAT in the student population. Some evidence was initially provided by van de Weijer (2019), who asked 184 students to complete two burnout scales, the BAT scale (i.e., exhaustion and disengagement) and the student version of the MBI. Although van de Weijer (2019) reported strong correlations between the BAT and the MBI scales, the evidence is incomplete because (i) it did not include all BAT scales, and (ii) the author could not perform any evaluation of the psychometric properties of the BAT on the student population. Although BAT also has a context-free version in which there is no reference to work or a specific field, the present study adapted the original work version in the context of an educational sample (i.e., higher education students). Although students are not part of an actual work setting, we prefer not to use the context-free version because students are yet involved in a learning context, which is similar to an authentic working context (Schaufeli & Taris, 2005). Since both students and employees operate with similar activities like assignments, deadlines, evaluations, assignments overload, and long working hours, we adapted a student-specific version of the work version scale.

Validity of the BAT

Previous studies evaluated the convergent validity of the BAT by analyzing its correlations with the MBI. In the present contribution, we extend the evidence regarding the BAT validity by investigating the associations between the BAT scales and other variables known to be associated with burnout: depression, anxiety, and the level of psychological distress triggered by somatization, coping strategies, and anticipatory appraisal styles.

The associations between burnout and appraisal, or how a person evaluates an event and situation, can be linked to the question raised by Bühler and Land (2003, p. 5): “why under the same working conditions one individual burns out, whereas another shows no symptoms at all?“. Although two people share identical working environments and stressors, one can experience elevated levels of burnout, while the other experiences only moderate levels of stress. Every person experiences stress in a unique manner, depending on how they perceive the situation or their structure of thinking patterns called appraisal (Lazarus, 1991). As theoretically described in The Transactional Model of Stress (Lazarus & Folkman, 1984), one situation generates multiple interpretations or appraisals.

Starting from these observations, Lazarus (1991) classified appraisal into primary (or how relevant is that given situation for one’s needs) and secondary appraisals (or what can be done to deal with the problem). The differentiation of the primary appraisal involved four dimensions, as follows. The threat dimension has a negative valence, as the situation or stimulus is perceived negatively. It is based on the evaluation that the stimulus has the perceived capacity to harm or damage the organism, producing negative emotions such as worry, fear, and anxiety. However, when a person appraises the situation as challenging, that situation can be mastered or can have growth or gain potential, producing a feeling of exhilaration and excitement (Lazarus & Folkman, 1984). The appraisal dimension named control expectancy, which corresponds to secondary appraisal, is one of the most constant components employed in the traditional appraisal literature. In order to gain control over a situation or stimulus, the organism must quickly evaluate if it can cope with the stimulus or event and flexibly adapt to these changes (Scherer, 1982). Smith & Ellsworth (1985) viewed control as the perception of controllability and explored various control instances: if the control is in the hands of the person (personal power), another person, or other impersonal circumstances. The coping potential or coping competence refers to the individual ability and the personal resources to cope with a situation or stimulus. A study centered on occupational stress in university and college teachers found that the relationship between burnout and stress is partially mediated by cognitive appraisals (Gomes et al., 2013). Stress was negatively associated with a challenging perception of the situation, along with control perceptions, and positively associated with a threatening perception. The event perceptions produce emotional consequences that elicit different emotions in specific persons (Lazarus & Folkman, 1984). When the event is perceived negatively and threatens homeostasis, stress starts to rise, causing unsuccessful stress management. Thus, to understand why people react differently to a given situation or event, we need to explain the manner in which the person evaluates or perceives the situation, how they manage their abilities or resources to deal with the consequences of the situation, and the coping response to that stressful situation.

When a stressful situation triggers unpleasant emotions, the role of coping strategies is to manage these emotions and reduce stress. These coping strategies can take the form of behaviors or cognitions and can be social or individual (Carver et al., 1989). They can be grouped into four major coping approaches: problem-focused coping, emotion-focused coping, social support, and avoidant coping. Problem-focused coping represents an adapting coping approach, where the focalization is put on problem-solving. This coping mechanism may provide increased perceived control over the problem. Emotion-focused coping represents a maladaptive coping approach that is focalized on emotions and may lead to diminished perceived control over the problem. Social support coping (instrumental social support coping) describes one’s tendency to call for help, advice, and information (social support), and moral support, compassion, understanding from colleagues, friends, and relatives (emotional support). Avoidance coping is expressed by mental and behavioral deactivation, restrain, denial, and acceptance. These distracting strategies are used to reduce the concentration on the solution, avoid confrontation with the problematic situation, and reduce the effort, ignoring and abandoning finding solutions.

Currently, the development of burnout is seen as the result of a combination of high levels of stress with poor coping strategies; therefore the style in which a person copes with a situation is related to burnout levels (González-Morales et al., 2010). The results of a meta-analysis suggested that problem-focused coping is correlated only with personal accomplishment, while emotion-focused coping is correlated with exhaustion and with depersonalization (Shin et al., 2014). Religious copings, reappraisal, and social support were negatively correlated with the core burnout symptoms, while acceptance strategies were positively correlated with burnout symptoms (Shin et al., 2014). To conclude, psychologically healthy individuals suffer less frequently from burnout because they have higher adaptive or positive coping styles when facing chronic stressors.

Although burnout is strongly associated with depression and anxiety, the results of a systematic review and meta-analysis (Koutsimani et al., 2019) showed that they are not overlapping constructs, just interconnected ones, with different nosology and conceptualizations. However, in some circumstances, burnout and depression can overlap and change together over time (Ahola et al., 2014). For this to happen, burnout must be at a severe level and have relatively high frequencies to meet the criteria for depression (Bianchi et al., 2014). Furthermore, Schonfeld and Bianchi (2016) provided evidence for the overlapping symptomatology of burnout and depression. The psychological symptoms of major depressive disorder involve emotional problems such as persisting sadness and hopeless feelings, losing interest in past enjoyable activities, inability to feel pleasure (anhedonia), diminished concentration and ability to think, fatigability, changes in appetite, and sleep disturbances (American Psychiatric Association, 2013). The physical symptoms revolve around chronic pain and gastrointestinal issues (American Psychiatric Association, 2013). Hence, being burned-out or being depressed can share similar characteristics, like loss of energy or being drained out, depressed mood, impaired concentration and diminished ability to think, an inability to feel pleasure, and a tendency to function on autopilot or being absent-minded (Koutsimani et al., 2019). As a result of the different pervasion and contexts in which both appear, burnout and depression can be distinguished from one another and treated and formulated as different constructs. Depression is context-free, unrelated to any particular situation, and more pervasive, while burnout is work-specific.

General and specific anxiety (i.e., social anxiety) are also associated with burnout (Koutsimani et al., 2019; Sun et al., 2012). Additionally, in some studies, emotional exhaustion and cynicism were correlated strongly and positively with anxiety symptoms (Turnipseed, 1998). The interaction between work stressors and individual characteristics generates a state of anxiety that contributes to the onset of burnout (Turnipseed, 1998).

In addition to depression and anxiety, burnout is also associated with somatization distress. Based on a systematic literature review, Salvagioni and colleagues (2017) concluded that burnout was a significant predictor for physical complaints, such as headaches, gastrointestinal problems, respiratory problems, musculoskeletal pain, and prolonged fatigue. More precisely, burnout and psychosomatic symptoms were negatively correlated with job satisfaction and predicted 27% job satisfaction in a sample of Dutch nurses (Meeusen et al., 2010). Convergently, Piko (2006) reported that burnout predicted psychosomatic health problems and that emotional exhaustion is a predictor of psychosomatic symptoms.

The present study

This study aimed to investigate the psychometric properties and the validity of the S-BAT, a new measurement tool that assesses burnout in a university student population. This questionnaire has six scales grouped in two categories: core burnout symptoms (S-BAT-C, which includes feelings of exhaustion, mental distancing, cognitive impairment, and emotional impairment) and secondary symptoms of burnout (S-BAT-S, which includes psychosomatic complaints and psychological distress). In addition to the analysis of the internal validity of the S-BAT, we also collected data to assess the construct and the predictive validity of the S-BAT using variables that were not previously related to this scale.

Methods

Participants

The participants (N = 399, 60,70% female, mean age = 20.76 years old, SD = 4.62 years) were Romanian bachelor students from the Faculty of Psychology (57.10%) and Computer Science (42.90%). Most participants (i.e., 59.6%) have financial support from the Romanian Government, and 11.50% have failed exams from last semester. Regarding their occupation status, 7% of our participants have a full-time job, and 5.50% work a part-time job. An overview of the sample is presented in Table 1. Data was collected in the spring of 2021 and all measures were completed in a single session, when all students were at home and classes were held online because of the Sars-Cov-2 pandemic.

Table 1 Descriptive statistics of demographic variables
Full size table

Measures

We assessed student burnout using an adaptation of the original work-related version of the Burnout Assessment Tool (BAT; Schaufeli et al., 2020a, 2020b). The BAT has 23 items that assess the four core symptoms of burnout (BAT-C, with 23 items) and 10 items that investigate the two secondary symptoms of burnout (BAT-S). Both components, BAT-C and BAT-S, respectively, are rated on a five-point Likert scale, ranging from 1 (never) to 5 (always). The scale has multiple versions which are open access, including a Romanian translated version that is available on the BAT website (Schaufeli et al., 2020b). The BAT items were reformulated by replacing the word “work” with “school” to address the student’s specific educational environment. For example, an exhaustion item that was originally formulated as, After a day at work, I find it hard to recover my energy, was modified as follows: After a day at school, I find it hard to recover my energy.

We used The Primary and Secondary Appraisal scale (PASA; Gaab et al., 2005) for the evaluation of primary and secondary appraisals. The PASA has 16 items that differentiate the anticipatory cognitive evaluations of future tasks using two scales for assessing primary appraisals (i.e., the threat scale and the challenge scale) and another two scales for the secondary appraisals (i.e., self-concept of own abilities and control expectancy). The PASA (Gaab et al., 2005) is a self-reported questionnaire that requires respondents to rate themselves using a 6-point Likert scale (i.e., from 1 - strongly disagree to 6 - strongly agree). Following Gaab et al. (2005), we calculated an overall stress index by subtracting the primary appraisal (or initial assessment) from the secondary appraisal (or second evaluation).

Coping strategies were investigated using the Brief Coping Orientation to Problems Experienced (Brief Carver 1997). The Brief COPE is the short version of the COPE questionnaire (Carver et al., 1989), which has 28 items that address 14 coping strategies and four major coping approaches. The Romanian version of the COPE was adapted by Crașovan and Sava (2013). Respondents have to rate themselves using a Likert scale ranging from 0 (I have not been doing this at all) to 3 (I have been doing this a lot).

We assessed emotional distress with the 21-items version of The Depression, Anxiety, and Stress Scale (DASS-21; Lovibond & Lovibond, 1995). DASS-21 items assess the severity of the emotional distress along with three scales: depression (e.g., hopelessness, anhedonia), anxiety (e.g., the anticipation of negative events, subjective experience of anxious affect), and tension/stress (e.g., difficulty relaxing, easily agitated/irritable/over-reactive). Respondents have to rate themselves on a four-point rating scale, from 0 (did not apply to me at all) to 3 (applied to me very much, or most of the time).

Psychosomatic distress was measured using the somatization subscale of The Symptom Checklist 90 (SCL-90; Derogatis & Cleary, 1977). The subscale has 12 items, but for this study, we used only 11 items without the heavy arms/legs item. This item is marked by feelings of heavy or leaden arms and legs, known in psychiatry and clinical psychology as Leaden paralysis. The heavy arms/legs item was not selected because this symptom is often correlated and appears mostly in atypical depression (Posternak & Zimmerman, 2001) with symptoms like hypersomnia, increased appetite, and interpersonal rejection sensitivity, which will not be tackled in this present study. The statistical analyses were conducted for each item because they were considered independent symptoms of somatization. This subscale assessed how distressing or bothering the somatic symptoms are, and distress arose from dysfunction in bodily perceptions. The somatization subscale was correlated with body symptoms, hypochondrias, organic symptoms, and poor health (Derogatis & Melisaratos, 1983). The responses are rated on a four-point scale, varying from 0 (not at all) to 4 (extremely). This subscale was originally designed as a screening tool for the investigation of the physical complaints caused by psychological factors or stress. This distress arises from bodily perceptions and is focused on respiratory, cardiovascular, gastrointestinal, muscular complaints. On the contrary, the scale on psychosomatic complaints included in the BAT’s secondary symptoms measures the frequency of those complaints.

Statistical analyses

For adaptation purposes, we investigated the internal validity of S-BAT, alongside construct and convergent validity. The internal validity was examined using confirmatory factor analysis (CFA) with the maximum likelihood estimation (ML), under the assumption of normal distribution of the items. We used Pearson correlation analyses to assess construct and convergent validity. In the CFA, we estimated model fit using the χ2 (chi-square) index, the Root-Mean-Square Error of Approximation (RMSEA; Steiger, 1990), the Tucker-Lewis index (TLI; Tucker & Lewis, 1973), Comparative Fit Index (CFI; Bentler, 1990), and the Standardized Root Mean Square Residual (SRMR; Chen 2007). The cutoff criteria used in this study follow the indices accepted in the literature, specifically: values smaller than 0.05 for RMSEA showing an excellent model fit, and values greater than 0.08 indicate acceptable fit; values greater than 0.90 for RMSEA showing an excellent model fit, and values greater than 0.90 on TLI show acceptable fit; the values close to 0.95 on CFI show an excellent fit, and values greater than 90, good fit; for SRMR values smaller than 0.08 showing a good model fit (Hu & Bentler, 1999). We used the popular likelihood ratio test to compare nested models, while for comparing non-nested models we used the Akaike Information Criterion (AIC). To estimate the sample size required for this present study, we computed a sample size analysis using online software (Soper, 2022) The sample size was calculated by entering the required parameter values, using the anticipated effect size of 0.2, the desired statistical power level of 0.8, the number of latent variables being 6, and the number of observed variables being 33, followed by the value of probability level set at 0.05. The results indicated that the minimum sample size to detect an effect is N = 403, which is a value close to our sample size (i.e., N = 399). Finally, there were no missing values in the dataset since the number of cases with complete data (N = 399) corresponded to the sample size (N = 399).

Results

Investigation of the S-BAT internal validity

To investigate the internal validity, we tested the factor structure of the S-BAT scale using confirmatory factor analysis (CFA). To assess the model fit, we used the lavaan package in R studio (Rosseel, 2012), and we tested the following two models: a model that assumed that all items have a single latent factor, and another model that assumed that all answer variance is explained by six correlated factors. The fit indices presented in Table 2 indicated that the single factor model does not have adequate fit, whereas the 6-factor model had adequate values for all fit indices. The difference between these two models was statistically significant (Δχ2(15) = 1568.817, p < .001). We investigated the error covariance matrix of the 6-factor model, and we found that we can improve our model if we allow the covariance of errors in the case of three pairs of items that were developed for the same scale and had similar meaning (i.e., items 12 and 13, which belong to the Mental distancing scale; items 19 and 20, which belong to the Emotional impairment scale; items 27 and 29; see Table 3 for details regarding the item contents). Although the items 27 and 29 belonged to different scales (i.e., Psychological distress and Psychosomatic complaints), they both described subjective experiences that are specific to anxiety states. This improved model had the best fit indices (TLI = 0.926; CFI = 0.933; RMSEA = 0.052; SRMR = 0.048) and was significantly superior to the initial 6-factors model (Δχ2(3) = 125.029, p < .001). Next, we investigated the standardized factor loadings (presented in Table 3) to identify possible model misspecification. All items have standardized loadings with values higher than 0.50, which represents a value that is larger than the generally accepted cutoff value of 0.30.

Table 2 Fit index for the tested models on S-BAT (N = 399)
Full size table
Table 3 Standardized item loadings for the six factors correlated model
Full size table

We computed the internal consistency indices for each dimension of the S-BAT Romanian version. As shown in Table 4, internal consistency coefficients (i.e., McDonald’s omega-total coefficients) have values that range from 0.72 to 0.95. Because internal consistencies values greater or equal to 0.70 are recognized as acceptable and values greater or equal to 0.80 are interpreted as good (Nunnally & Bernstein, 1994), we can conclude that the S-BAT scales have good internal consistency levels. Furthermore, Table 11 in the Supplemental material displays the inter-item correlation matrix. Similar to the results reported by Schaufeli et al. (2020a, 2020b), all the correlations between the dimensions of the S-BAT (also presented in Table 4) were statistically significant and indicated moderate to strong associations.

Table 4 Descriptive data for BAT’s dimensions: mean, st andard deviation, Internal consistency indices, and correlations among dimensions
Full size table

In addition, we estimated a model with a higher-order factor and a bifactor model, to provide further insights regarding the multi-dimensionality of the S-BATs. The model with higher-order factor also included the covariances between the three pairs of items, as described in the improved 6 factors model. Surprisingly, both models had poorer fit indices and their AIC indices were larger than the same indices of the improved 6 factors model. As Burnham and Anderson (2004) explained, differences in AIC larger than 10 indicated that the models with larger AIC values should be considered less adequate. Also, the bifactor model was significantly less appropriate to describe our data, as indicated by the likelihood ratio test (Δχ2(9) = 145.645, p < .001). For the bifactor model, we calculated the explained common method variance (ECV) to assess how much of the total item variance is accounted by each factor. The ECV values presented in the Supplemental Material suggested that, in the bifactor model, 66.56% of explained item variance is accounted by the single factor. The remaining 33.44% of the variance is distributed between the six factors originally described by Schaufeli et al. (2020). In addition, we estimated the McDonald’s omega-hierarchical coefficients for the bifactor model, and we found that the single factor had excellent consistency (ω = 0.90), while the opposing factors had poor reliability indices (i.e., values between 0.22 and 0.36). To conclude, although the ECV values and the McDonald’s omega coefficients suggested that most of the S-BAT scales can be accounted by a single factor, both models that assumed the existence of higher-order factors had poor fit indices and could be considered inadequate to describe the dataset.

We also assessed the gender invariance of the six-factor model. When analyzing demographics data, their distribution in our sample did not allow for further invariance analyses (i.e., were unequally balanced). The gender invariance of the S-BAT was assessed using configural, weak, strong, and strict models. These results are illustrated in Table 5.

Table 5 Fitness indices for assessing the factorial invariance of the S-BAT.
Full size table

In conclusion, the S-BAT showed promising psychometric properties: adequate fit indices, good internal consistency, and inter-scale correlations that have similar values to the correlations originally reported by Schaufeli et al. (2020a).

Construct validity

Construct validity was assessed by calculating the correlations between S-BAT - DASS and S-BAT - SCL somatization subscale. Table 6 presents the Pearson correlations between the S-BAT and DASS scales, with correlation values ranging from 0.37 to 0.70. The smallest correlation values were between mental distance, anxiety (0.41), and stress (0.37), followed by cognitive impairment, and anxiety (47), stress (42). This suggests that the depression scale of the DASS and all S-BAT scales were adequately correlated, in line with previous studies about burnout and depression, validating the idea that the two concepts are associated. Despite the small correlation values of the two factors of S-BAT, the rest of the dimensions showed adequate correlation values, especially between core symptoms, and DASS’s anxiety (0.60), and stress (0.56) scales. Thus, burnout, anxiety, and stress were linked to overall burnout core symptoms, exhaustion, emotional impairment, psychological distress, and psychosomatic complaints. Taken together, the evidence presented in Table 6 suggests that the S-BAT scores are convergent with the DASS scores.

Table 6 Relationships between S-BAT and DASS
Full size table

The correlations values included in Table 7 indicated significant relationships between the scales of the S-BAT and SCL somatization items. In general, we found that the largest correlation values were between the scales Psychological distress (median value of the correlations = 0.40), Psychosomatic complaints (median value of the correlations = 0.56) and Exhaustion (median value of the correlations = 0.39). The remaining S-BAT scales were also significantly associated with the SCL-90 symptoms, but their correlation values were weaker (i.e., median values of the correlations between 0.25 and 0.30). However, only the psychosomatic complaints scale of S-BAT is significant when considering the convergent validity. These SCL-90 somatization symptoms coincide with the psychosomatic complaints of the S-BAT scale. But some of the psychosomatic symptoms (somatization) from the SCL-90 scale were not covered by the S-BAT somatization part (the psychosomatic complaints dimension). Only nausea (0.41) and lower back pain (0.47) symptoms showed a lower correlation with the psychosomatic complaints of S-BAT. Therefore, the evidence presented in Table 7 suggests that the secondary symptoms of S-BAT (psychosomatic complaints) are convergent with the SCL-90 somatization scale.

Table 7 Relationships between S-BAT and SCL-90
Full size table

Convergent validity

Convergent validity was assessed by calculating the correlations between S-BAT - PASA and S-BAT – COPE. The relationships between the prospective appraisal (as assessed by PASA) and S-BAT scales, presented in Table 8, are stronger than the relationships between coping and S-BAT scales. The strongest relationships were between anticipating threatening events and all S-BAT scales (median value of correlations = 0.40) and between negative perceptions of own abilities and all S-BAT scales (median value of correlations = 0.45).

Table 8 Relationships between S-BAT and prospective appraisal (PASA scale)
Full size table

Overall, the relationships between the S-BAT scales and coping strategies are presented in Table 9 suggested that all burnout scales are strongly associated with avoidant coping (median value of the correlations = 0.43) and, to some extent, with coping strategies based on seeking social support (median value of the correlations = 0.21). Interestingly, we did not find any significant correlation between BAT scales and emotion-focused coping, while the relationships between problem-focused coping and BAT scales were generally weak and positive.

Table 9 Relationships between subscales of the S-BAT and the Brief COPE
Full size table

Discussion

In the present study, we aimed to investigate the psychometric properties of the student version of the Burnout Assessment Tool on a Romanian student sample (S-BAT), to provide evidence regarding its convergent validity, and to extend the evidence regarding its convergent validity by correlating the S-BAT scales with anticipatory appraisal and with coping strategies. Despite burnout being experienced similarly by school and university students, in the literature, the term “student burnout” is used interchangeably referring to school/university students. When describing university student burnout, the term “academic burnout” is sometimes used, but it can also refer to university staff’s burnout, such as teachers, etc. To dilate this terminology confusion, the term “student burnout” was preferred when referring to university students. Using confirmatory factor analyses, we found that the initial, 6-factor structure of the S-BAT was supported in our student sample. This is particularly important because the scale was originally developed for assessing workplace burnout, not academic burnout. All items had high loadings on their designated factors, and the S-BAT scales showed good reliability. The psychometric characteristics are comparable to the values of the original scale presented in the manual (Schaufeli et al., 2020b), thus suggesting that the BAT is an adequate measure of burnout in a higher-education student population. Other psychometric properties of the S-BAT were analyzed, as the fitness indices for assessing the factorial invariance of the scale. The result showed invariance evidence for the configural, weak, strong, and strict models of the demographic variable “gender”. In addition to the 6 factors models, we tested two additional models: a model with a higher-order factor and a bifactor model. Both these additional models had poorer fit indices as compared with the 6 factors model. The bifactor model suggested that a single factor can account for most variance of the S-BAT items, but its overall fit indices indicated that it was not adequate to describe the observed data. Despite this, there was a strong single factor in the S-BAT, which is consistent with the idea that the scale measures only one construct (i.e., burnout syndrome) (Reise, 2012). Therefore, in the present study we could not formulate strong conclusions regarding the use of an overall score of the BAT on the student sample, therefore we used separate scores for each of its scales.

Construct validity was investigated through the associations within a nomological network that included the S-BAT scales and measures of depression, anxiety, stress, and somatization symptoms. We found strong correlations that are consistent with previous theories and empirical evidence. On the one hand, the S-BAT scales were positively correlated with all mental health scales (i.e., depression, anxiety, and stress), thus suggesting that student burnout measured with S-BAT is associated with poor mental health. In addition, all S-BAT scales were positively associated with all somatization symptoms from the somatization subscale of SCL-90, and these results are convergent with previous findings (Sun et al., 2012; Salvagioni et al., 2017). These relationships were stronger in the case of the secondary scales, as compared with the core burnout scales, and supported the distinction made by the S-BAT authors between core and secondary burnout symptoms.

Associations between the burnout symptoms and the prospective appraisal contained in the PASA (Gaab et al., 2005) provided support for the convergent validity of the S-BAT. Although classical stress theories (e.g., Lazarus & Folkman, 1984) emphasize the role of appraisal in the development of stress, most literature on burnout and stress overlooked this process and focused on coping strategies. Our results suggested that S-BAT scales are strongly associated with students’ perceptions of future academic tasks. Burned-out students generally perceive future academic events as threatening and have negative perceptions regarding their abilities to handle future difficulties. The relationship between burnout and the PASA Threat scale is particularly interesting because it confirmed the conclusions formulated by Guthier et al. (2020), who emphasized the strain-effect of burnout (i.e., the burnout level predicts evaluations of future job stressors). The relationships between S-BAT and the negative perceptions regarding own abilities to handle future difficulties are also in line with previous evidence regarding the relationship between burnout and self-efficacy (Maricuțoiu & Sulea, 2019). Finally, our results suggested that anticipatory appraisal variables (i.e., the PASA scales) have stronger relationships with student burnout as compared with the coping approaches. This is important because it confirmed the emphasis that Lazarus (1991) placed on the appraisal role in experiencing stress.

Finally, to collect further evidence regarding the convergent validity, we correlated our burnout scales with the coping strategies (Brief Carver, 1997). Similar to previous studies (Anderson, 2000; Stoeber & Rennert, 2008), we found that the use of avoidant coping strategies is strongly associated with all burnout symptoms. As individuals attempt to avoid dealing with their problems, these problems pile up and are generating increased stress levels. Also, in line with previous evidence, we found insignificant relationships between most S-BAT scales and problem-focused coping. In their meta-analysis, Shin et al. (2014) reported close-to-null values for the relationships between problem-focused coping and two of the burnout components (i.e., exhaustion and depersonalization), and our results are similar to their findings. Interestingly, we found weak positive associations between the use of social support and the S-BAT scales. This is surprising, as a previous meta-analysis reported negative relationships between burnout and seeking social support (Shin et al., 2014). This result can be explained by the fact that our students were studying online at the time we collected our data (i.e., due to the COVID-19 pandemic), which makes it difficult for one to seek social support because of the social isolation policies imposed by the University. The social alienation and discomfort associated with studying during the COVID-19 pandemic (Son et al., 2020) can have contributed to the reduction of instrumental social support and might have weakened the interpersonal connections and raised individualism in managing stressful situations (Irwin & Berge, 2006). Finally, we did not find any relationship between the BAT scales and emotion-focused coping, although Shin and colleagues (2014) reported consistent correlations between emotion-focused coping and burnout components (i.e., exhaustion and depersonalization). The absence of the relationship indicated that students engaged in emotion-focused coping do not have high or low levels of burnout. In some cases, engaging in emotion-focused coping at home (e.g., venting emotions or developing self-blame cognitions) could be a successful strategy to gain support or help from family members, while in other contexts, this could simply not work (e.g., the students do not live with their parents). Therefore, this unexpected result could be generated by the variables related to the social context of our respondents. Future studies should further investigate this relationship using the S-BAT to check whether the absence of a relationship between emotion-focused coping and BAT scales is specific to our sample.

The present study has several limitations that could be tackled by future studies. First, because in this study we used a clinical perspective on burnout, future studies could focus on variables included in other models that describe the development of burnout (e.g., the Study Demands-Resources Framework –Lesener et al., 2020). These variables could include study demands (e.g., time pressure) or study resources (e.g., support from students or from the teachers). The specificity of this sample, by the presence of only Psychology and Computer Science students, represents another limitation. Furthermore, because previous evidence suggested that students in different faculties could experience different levels of burnout (Aguayo et al., 2019; Caballero et al., 2015), future research could include samples of students from a larger variety of faculties and other areas of science, to improve the evidence regarding the generalizability of our findings. Next, future data collection should be extended over a longer period of time to strengthen the predictive validity evidence, using a longitudinal design. Another limitation of the present paper is that it did not investigate the discriminant validity of the student version of the BAT. To fill this gap, future studies that use the student version of the BAT could also include variables that are presumed to correlate negatively with students’ burnout (e.g., academic achievement). The overall fit of the S-BAT was acceptable for the model that assumed the existence of 6 correlated factors, but not excellent. We conducted exploratory model trimming (i.e., in the improved 6-factors model), and we found that the residual correlations of 3 pairs of items still had significant values. While this finding does not have an impact on our decision to accept the 6-factors structure of the S-BAT, the existence of these pairs of residual correlations can be used by future studies that could aim to find a shortened version of the S-BAT. A further limitation concerns the moment of data collection. The data collection was carried out during the coronavirus pandemic (COVID-19) when the courses were held online, which means that the students experienced this situation in a particular way. It is unlikely that the internal validity of CFAs to be affected, despite of this limitation, but it is possible that some correlations between the scales (BAT and coping/appraisal) to be specific to this sample. For example, we found a weak but positive association between the use of social support and the S-BAT scales. This surprising result is not in concordance with the previous studies that reported negative relationships between burnout and social support. Therefore, this negative association can be specific to the sample and explained by the fact that, in the online format, the students in this study sought more emotional support than informational support. Moreover, this emotional support (e.g., venting) may have worsened the burnout state by absorbing in emotional discussions that could deepen the self-perceived state of stress.

Conclusion

In conclusion, we found that the Burnout Assessment Tool (Schaufeli et al., 2020b) can be used on students enrolled in higher education institutions. Our confirmatory factor analyses indicated that the BAT structure is adequate in the student sample, and the internal consistencies of the scales were also excellent. In our student sample, the BAT scales were strongly associated with self-reported measures of depression, anxiety, stress, and psychosomatic symptoms, and all these results supported its construct validity. Finally, the relationships between BAT scales and self-regulatory mechanisms (i.e., appraisal and coping scales) also confirmed results from the previous studies, thus supporting the convergent validity of the BAT.