According to the International Telecommunication Union (ITU), approximately 5.3 billion people used the internet in 2022 (www.itu.int), which constitutes a majority of the global population. Furthermore, today information and communication technologies (ICT) are an essential backbone of a functioning economy, public administration, and society in general. The use of ICT brings significant benefits to individuals, organizations, and society. Examples include improved access to information, more and faster communication possibilities, as well as increased efficiency and productivity [1,2,3].

Despite these positive effects, the use and ubiquity of ICT can also have significant negative consequences. One major negative impact is digital stress. Both scientific research and anecdotal evidence indicate that human interaction with ICT, both in private (e.g., social media usage like Facebook or Instagram) and organizational (e.g., email, business application systems, social collaboration platforms like Jira or Teams) contexts, can lead to considerable stress perceptions among users [4, 5].

In the present article, we focus on the digital stress that users perceive in the workplace context due to the use and ubiquity of ICT. The phenomenon of digital stress was already discussed several decades ago, primarily as a result of the introduction of personal computers (PC) in companies. This discussion was predominantly based on the terms “technostress” [6] and “computer stress” [7]. However, in the past 15 years numerous scientific studies on ICT’s stress potential have been published [8]. Given the ongoing increase in digitalization in business, public administration, and society, it is expected that the phenomenon of digital stress will continue to be highly relevant in both science and practice.

Both in science and practice it is important to be able to reliably measure the phenomenon of digital stress. In addition to objective neurophysiological stress measurements [9,10,11], the use of questionnaire-based measurement instruments is often recommended in the academic literature, which can systematically capture subjective stress perceptions based on Likert scales [12]. In science, this type of stress assessment is essential because it allows the latent construct of “digital stress” to be measured empirically in theory-driven research. Furthermore, it is known that the detrimental effects of stress often also relate to negative subjective states that are objectively difficult to quantify [13]. In practice, the availability of a reliable and valid questionnaire-based measurement instrument is equally important so that organizations can assess the stress perceived by their employees. Based on such measurement in companies, effective measures for reducing or avoiding digital stress can be implemented. We observe that an increasing number of decision-makers in practice are beginning to address digital stress, as well as its causes and consequences. While traditionally managers from human resource (HR) departments were more concerned with this phenomenon, more and more top-level managers are now also paying attention to the topic. This is partly because this group is also affected by digital stress and partly because digital stress can adversely affect the performance and productivity of organizations, thus having direct business relevance [14].

Methodology

Data collection

For the validation of the instrument, we used an existing data set (n = 3333) that was collected in the German-speaking region (Germany, Austria, Switzerland) as part of the development of the original version of the Digital Stressors Scale (DSS). About one third of the data originates in each of the three countries and the participants are representative of the local employed population in terms of age and gender distributions (for details, please refer to [15]).

The questionnaire was provided online and for each question, a 7-point Likert scale was consistently used ranging from 0 “strongly disagree” to 6 “strongly agree.” Hence, the value “3” constitutes the neutral position on the scale. For each question, the participants also had the possibility to choose the option “Don’t know” and were instructed to do so if they were not sure about the meaning of the question or if they thought that it was not applicable to their situation.

Instrument validation

To create a short version of the original DSS, we focused on retaining at most three items per stressor category, in line with recommendations for the minimum number of items that is needed for reliability and factor analyses (e.g., [16]). We retained items that have shown strong loadings with their respective stressor categories in the original instrument and then tested the reliability of each of the 10 stressor dimensions with their reduced number of items. As can be seen in Table 1, all stressor categories show sufficient reliability with a Cronbach’s alpha of > 0.70. In addition, despite the fact that we substantially reduced the number of items for each stressor category (from five to three, and hence the total number of questions in the instrument was reduced from 50 to 30, a reduction of 40%), we can observe that for most categories of the short version instrument (denoted DSS[30]) reliability is comparable to the original version of the DSS and the criterion of Cronbach’s alpha > 0.70 was always met (denoted DSS[50]).

Table 1 Reliability of stressor categories in the 50-item and 30-item version of the Digital Stressors Scale (DSS)
Full size table

After this initial inspection of the reliability of our short version instrument, we then used the 30 items and 10 stressor categories as input for a confirmatory factor analysis to also ensure the validity of the new factor structure. The overall quality of the model can be inspected using fit indices, which show that the new factor structure works very well: Chi-Square = 2602.68, df = 360, p ≤ 0.0001; Root Mean Square Error of Approximation (RMSEA) = 0.045; Normed Fit Index (NFI) = 0.99; Comparative Fit Index (CFI) = 0.99; Standardized RMR = 0.034; Goodness of Fit Index (GFI) = 0.95 (see [17] for related cutoff values). In addition, Table 2 shows that each item loads strongly onto its dedicated stressor category, which indicates that the expected association between items and latent factors are supported.

Table 2 Stressor categories, items, and standardized factor loadings
Full size table

Applying the instrument

Recommendations for using the questionnaire can be made to help apply it effectively. For designing the questionnaire, it is recommended to make it available online, as it was during the instrument’s development, thereby facilitating the presentation of the items in random order. Additionally, there should be an introductory statement at the beginning, describing the types of technologies respondents should consider when answering the questionnaire. Based on Riedl et al. [15], we recommend the statement as summarized in Table 3.

Table 3 Recommended introductory statement
Full size table

Each item can be answered on a 7-point Likert scale, ranging from “strongly disagree” (value “0”) to “strongly agree” (value “6”). As a result, “neutral” corresponds to a value of “3” and represents the middle of the scale. The “Don’t know” option is also provided. Before beginning the data analysis following the survey, it is necessary to check for potential missing values. If respondents selected the “Don’t know” option or left an item unanswered, the missing value can be replaced with the median of the remaining values for that item. However, if the proportion of these cases is very high (e.g., > 10%), consideration should be given to excluding the respective item from further analysis.

To calculate values for the 10 stress categories and the overall digital stress score, the average of values per stress category (i.e., the average across the three items in each category) or the average across all items in the questionnaire (i.e., across the 30 items) is to be calculated. This procedure is possible because the DSS is a reflective questionnaire instrument. This means that the stress categories represent the perception of digital stress, and the items, in turn, represent their respective stress category. Consequently, the 10 stressor categories can also be used independently of each other. Therefore, for a specific scientific investigation or in a particular company, if not all stress categories are relevant or, in an extreme case, only one category is of interest, the relevant categories can be used independently. However, it is recommended to use the five-item version [18], especially when prioritizing a specific stress category or a few categories, to better mitigate the disruptive effects of missing values.

When interpreting the obtained stress values, the following should be considered. When calculating the total digital stress experienced by an individual (i.e., the average across all 30 questions), every value greater than zero already represents stress perceptions—the higher the value, the more pronounced the perceived stress. Moreover, our experiences of using the measurement instrument in corporate practice show considerable variations in values across the 10 stressor categories. The average value of the total perceived digital stress lies roughly in the middle of the scale and slightly below it for a majority of respondents. It is less common for many employees in a company to be far above the scale’s midpoint. However, this does not mean that such a situation is only associated with low to moderate strain levels among respondents. Rather, reports from corporate projects in German-speaking areas indicate that, at least currently, some stressor categories may have a rather low manifestation (e.g., stressor category III—Insecurity), while other categories can exhibit very high manifestations (e.g., stressor categories II—Conflicts and VII—Social environment). Such a constellation can be associated with high perceived strain, even with a moderate overall score for digital stress. In extreme cases, as per our experience from corporate projects, nine of the 10 categories may have a rather low manifestation, while one category has the maximum manifestation, leading to significant strain and burnout tendencies.

Furthermore, it should be noted that the interpretation of (digital) stress is ideally carried out using neurobiological methods and survey methods (such as the short or long version of the DSS). The main reason for this fact is that conscious stress perceptions in individuals that can be measured using survey methods often do not correlate with the typically unconscious increases in stress hormones and other stress parameters, such as increased blood pressure and reduced heart rate variability. This finding is reported in both the general stress literature and in the literature on digital stress (selected studies can be found in [10]). Therefore, in questionnaires, individuals may exhibit low values (i.e., they subjectively believe they have little stress) when asked about their perceived (digital) stress, even though neurobiological stress parameters already show increased or significantly changed values compared to baseline measurements. Despite this fact, psychometrically evaluated survey instruments for measuring digital stress, such as the DSS, are currently the central tool for stress assessment in a corporate context, particularly from the perspective of management and HR departments, even though wearables like smartwatches increasingly allow for a relatively straightforward physiological determination of stress parameters such as heart rate and heart rate variability [19].

Conclusion

Digital stress is a form of stress that is gaining increasing importance worldwide due to the widespread use of ICT. In this article, an English-language short version of the DSS was introduced based on the original English-language [18] and German-language [15] instruments, each of which conceptualizes digital stress along 50 questions across 10 stressor categories. The instrument presented in the current paper consists of 30 questions, and each stressor category is measured based on three questions. The psychometric evaluation presented here demonstrates the instrument’s reliability and validity, both regarding the total digital stress perceived by an individual in the workplace and for each of the 10 stressor categories. The instrument introduced here contributes to future research and the assessment of digital stress in businesses, providing a foundation for the development of effective coping strategies.