1 Introduction

How to integrate digital media in university is an important topic which is addressed both in research and is of practical relevance since Higher Education (HE) practitioners are struggling how to integrate digital media in study programs and infrastructure. This integration is assumed to offer innovative potential for teaching and learning at Higher Education Institutions (HEIs). However, the relatively new trend towards ‘Digitalisation’ in HE has not been sufficiently considered in the evaluation of student performance and academic success yet. For instance, the comprehensive multidimensional instrument to evaluate university teaching by Lemos, Queirós, Teixeira, and Menezes (2011) includes the dimension teaching methods but does not yet include digital media.

However, just as digital media is affecting everyday life, a change of academic studies and demands might be assumed, not least because skills concerning computers or digital media, in general, are more and more required in many occupational fields (see e.g. Ally and Prieto-Blázquez 2014), but also as the corona-crisis in 2020 and the associated rapid digitalisation of university teaching show very clearly. Notwithstanding the amount of existing research concerning media use of lecturers and students in recent years, for instance in Germany (Dolch and Zawacki-Richter 2018; Grosch 2012; Grosch and Gidion 2011; Müßig-Trapp and Willige 2006; Persike and Friedrich 2016; Schulmeister 2009; Vogel and Woisch 2013; Zawacki-Richter 2015; Zawacki-Richter et al. 2015, 2016, 2017) and internationally (Al-Husain and Hammo 2015; Dahlstrom and Bichsel 2014; Dahlstrom et al. 2011, 2013, 2015; Dahlstrom and Walker 2012; Rutherford and Standley 2016; Thompson 2013) both the use of digital media in HE settings as well as the impact of digital media on studying itself are still insufficiently investigated. With 2339 students in 2012 and 1327 students in 2015, from several HEI’s that offered online courses and study programs at the time, Zawacki-Richter et al. conducted surveys addressing digital media usage (Zawacki-Richter et al. 2017). In 2012 only 56% of the students owned a smartphone, 86% a laptop, and 9% a tablet and in 2015, already 91% owned a smartphone, 92% a laptop, and 40% a tablet (Zawacki-Richter et al. 2017). The EDUCAUSE Center for Analysis and Research (ECAR) has researched undergraduate students and IT between 2004 and 2015, based on 4123 in 2004 to 50,274 students in 2015 from HEI’s in the USA and up to 15 other countries (Dahlstrom et al. 2015). Those studies have shown a similar increase in the spread of technology and the use of mobile devices in both the private and academic sectors over time (Dahlstrom and Bichsel 2014).

This increase is only one indicator for the increasing relevance of digital media at HEIs and thereby the need for research on digital media behaviour of university students. In addition to further, mainly descriptive analyses of media use and distribution, Zawacki-Richter et al. (2015) established a media usage typology based on the 2012 survey of 2339 university students. In several subgroup analyses, they found, among other things, significant differences between male and female students (Zawacki-Richter et al. 2015). This corresponds to previous studies, such as by Huang, Hood, and Yoo (2013), who also investigated the use and acceptance of various (web 2.0) applications, for 432 college students, and found significant gender differences as well.

Above all, in the studies mentioned, factors such as underlying motivations, emotions, self-evaluations, or self-efficacy are hardly considered, and students’ social background is not taken into account either. A notable exception is a study by Horvitz et al. (2015) who examine faculty’s self-efficacy regarding online teaching, however, this study does not take the students’ view into account. Other studies examine the interplay in the context of school education (e.g. Li et al. 2019; Sangkawetai et al. 2018). A recent study by Nouri (2018) with about 500 students at a Swedish university investigated multimodal literacy and the learning design during self-studies. It finds that technology indeed changed university students’ self-studies and knowledge building, however, this study does not reveal how these multimodal learning practices affect academic success. Therefore, there is a need for a comprehensive evaluation instrument to explore the connection between students’ background factors and their study respectively media behaviour including the link with academic performance.

Research regarding students’ academic performance mainly focuses on individual characteristics, often using concepts from educational psychology but does not specifically address digital media in academic contexts. Concepts often used are for example the ‘expectancy-value theory of achievement motivation’ by Wigfield and Eccles (2000) or the framework of social-cognitive theory (SCT) and self-efficacy by Bandura (e.g. Bandura 1977). Most research is also located outside of Europe; in consequence, little is known about the transferability of this research to the European context. Also, as research is often characterized by low case numbers and a predominant focus on psychology students, results are often not generalizable respectively valid for other disciplines (see for example review studies by Bartimote-Aufflick et al. 2015; Honicke and Broadbent 2016). In addition to these short-comings, family background or other social and contextual factors are hardly taken into account in the aforementioned research strand. These factors, in turn, are addressed in social-science research. Due to increasing heterogeneity of students, not only at German HEI, such studies focus for example on the identification of groups with certain characteristics that are in some ways disadvantaged in academic studies (e.g. Röwert et al. 2017). In these studies, however, the important mechanisms and variables at the individual level, which would allow for further implications regarding possible interventions, are not considered. An integrative model considering social cognitive, individual characteristics as well as contextual or familial factors in terms of students’ performance and digital media behaviour at HEI is lacking so far.

Research shows that academic achievement varies between different social groups, such as migrants, students with children, or low socioeconomic status (SES) (Röwert et al. 2017). Often, this relation leads to lower academic achievement of those students whose parents are characterized by lower educational background. In addition to students’ socio-economic background, students’ self-efficacy expectations and motivation are related to their academic achievement and goal setting (e.g. Komarraju and Dial 2014; Pajares 1996; Putwain et al. 2013; Schunk and Pajares 2002; Zimmerman 2000a, 2000b; Zimmerman et al. 1992). Assuming a link between social backgrounds, e.g. parents’ educational background, certain self-efficacy expectations, and behaviour in academic settings in general (Zimmerman et al. 1992; Zimmerman 2000b), the same factors might be relevant in terms of students’ digital media behaviour, this needs to be further explored.

In sum, the introduced study aims to supplement current research in the field of digital media in Higher Education by developing a survey instrument that allows addressing the multi-faceted character of academic studies and digital media behaviour. Particularly, a new scale for digital media self-efficacy expectation (DMSE) is constructed to allow for a further examination of the determinants for observable media usage patterns and their potential links to students’ social backgrounds. Our instrument is designed to comprehensively capture the relevant individual, contextual, and social factors for academic performance and therefore, lead to a deeper understanding of the mechanism for the disadvantage of certain student groups, the relevance of digital media in HE and also further research on possible interventions at the same time. Thus, the developed evaluation instrument contributes to extending research on digital media in Higher Education. Furthermore, this focus on digital media in our research instrument also complements the study of Brahm and Jenert (2015) on university students and their attitudes towards studying, which is therefore partly replicated and also validated once more. However, this paper exclusively focuses on the development and validation of a survey instrument and some first descriptive insights, therefore, we do not present the results of the above mentioned potential analyses in the paper at hand. Nevertheless, we want to point out the possible applications of this instrument and promising starting points for further research. As a practical contribution, the instrument can also be used by other HEIs to evaluate their own digital media use and to determine in which ways their students are benefiting (or hindered) from using digital media, in particular concerning the disadvantage of certain groups of students.

The pre-test survey was conducted in December 2017 and the full data set was collected in summer 2018. In the following, the theoretical background of the evaluation instrument as well as the evaluation procedure is presented. In a second step, the results of validation procedures as well as first (descriptive) results regarding university students’ digital media attitudes and behaviour.

1.1 Theoretical Background

Bandura’s social cognitive theory (SCT) (e.g. Bandura 1977, 1986, 2011) offers a theoretical frame to analyse thoughts, motivation, and behaviour and therefore appears to be well suited to the aim of the study at hand. According to this theory, human behaviour, in general, is caused by personal, behavioural as well as environmental influences. In a reciprocal determinism, individuals interpret the results of their performance attainments in a certain way, which in turn informs and changes their environment and their self-beliefs. This again, informs and changes the subsequent behaviour. One central aspect of the SCT is self-efficacy which Bandura (1986, p. 391) defines as ‘people’s judgement of their capabilities to organize and execute courses of action required to attain designated types of performances’. The higher the self-efficacy belief, the higher the effort people will expend on an activity, the longer they will keep up when confronting obstacles, and the more resilient they will prove in the face of adverse situations (Pajares 1996, p. 544). In reference to the HE context, academic self-efficacy beliefs are based on students’ perceptions of their abilities to achieve a certain goal, e.g. to complete a course or to pass an exam. This may determine their learning effort that is spent on the activities to reach such goals.

Self-efficacy expectations and behaviour in academic settings may also be linked to students’ success of integration at a higher education institution (HEI). In line with the ‘model of institutional departure’ by Tinto (1993), the failure to become or remain incorporated in the intellectual and social life of the institution is one of three crucial factors for student dropout, in addition to academic difficulties and the inability of individuals to resolve their educational and occupational goals. While incorporation in intellectual life refers to integration into the academic system, incorporation in social life refers to students’ social integration. Both integration aspects depend on the terms determined by the HEI such as the course of studies as well as on external factors such as the social background. Although Tinto focusses on the identification of courses of action for HEI to reduce student dropout, the model and especially the aspect of integration may in combination with self-efficacy expectations and other non-cognitive factors, such as goal orientation, be appropriate to describe reasons for academic achievement and behaviour in academic settings as well.

Due to the general behaviour-determining influence of self-efficacy expectations, it can be assumed that students’ media behaviour is influenced by media-related self-efficacy expectations. For example, the willingness to deal with new technologies, to try out new applications or to try out digitally supported learning environments, and to stay on track even when facing difficulties, depends on how much a person relies on their skills and problem-solving abilities in dealing with these technologies, in other words: their media-related self-efficacy. However, research concerning media use in HE so far is limited to either the assessment of media applications in specific contexts (e.g. lectures, seminars) or analyses of media usage patterns for a rather broad student population (see above and the following section). In consequence, it has hardly been investigated whether, in addition to general academic behaviour, media use could also affect academic success. Furthermore, there is hardly any empirical evidence concerning the role of digital media self-efficacy for media-behaviour in academic contexts, and again its relevance for academic performance and the relationship with socio-economic backgrounds. Thus, the dual focus on both academic and digital media self-efficacy may be useful in terms of further examining students’ learning behaviour and digital media use. Also, since self-efficacy expectations depend on environmental aspects that are deemed highly relevant in SCT in general, it is important to take contextual as well as social factors into account which illustrates once more the relevance of a comprehensive survey instrument to analyse study behaviour in the digital era. This is especially true in times of rapid acceleration of digitalisation processes, such as during the corona crisis in 2020, where traditional models of academic behaviour may reach their limits.

1.2 State of Research

Research concerning the link between students’ self-efficacy expectations, motivation, and academic attainment (e.g. Komarraju and Dial 2014; Pajares 1996; Putwain et al. 2013; Schunk and Pajares 2002; Zimmerman et al. 1992) identifies self-efficacy expectations as an important predictor for academic goal setting and achievement.

For example, Bartimote-Aufflick et al. (2015) and Honicke and Broadbent (2016) found a connection between self-efficacy and study success. In line with the theoretical concept, i.e. reciprocal determinism (Bandura 1977), former experiences such as past grades in the academic context may influence subsequent self-efficacy expectations. This has also been shown in empirical studies (Klassen and Usher 2010; Lindsley et al. 1995; Talsma et al. 2018). However, for the context of physiotherapy education, Jones and Sheppard (2012) showed that previous experience was only related to self-efficacy in two distinct fields. Also motivation and goal orientation (Hsieh et al. 2007) because of their relevance for interest and self-regulation (Honicke and Broadbent 2016); emotions like anxiety (Hsieh et al. 2012); perceived control over actions and outcomes (Pekrun 2006), and certain personality traits like conscientiousness due to its link to self-discipline (Lievens et al. 2009) are relevant for the self-efficacy-achievement relation.

Furthermore, academic achievement varies between different social groups, such as migrants, students with children, or low socioeconomic status (SES) (Röwert et al. 2017). In this regard, research suggests that students’ SES may affect academic achievement via self-efficacy (Gecas and Schwalbe 1983; Weiser and Riggio 2010). Students stemming from lower socioeconomic backgrounds show higher academic performance when indicating higher self-efficacy; however, usually, such students are equipped with lower self-efficacy expectations (Weiser and Riggio 2010).

In our research, two instruments, in particular, have proven to be reliable and often used for investigating academic self-efficacy: The academic self-efficacy scale as designed by Jerusalem and Schwarzer (2002) and the scale used in the Motivated Strategies for Learning Questionnaire (Duncan et al. 2015). Since we focus on German university students and developed a survey instrument in the German language, the academic self-efficacy scale (Jerusalem and Schwarzer 2002) seems most appropriate to our intentions.

Recent research concerning students’ digital media use shows that students varying in e.g. age, family status, or ambitions show differing patterns of digital media use in academic settings (e.g. Grosch and Gidion 2011; Zawacki-Richter 2015; Zawacki-Richter et al. 2016). Since digital media are a global phenomenon and can have a positive impact on learning outcomes (Cavanaugh et al. 2009; Li and Ma 2010; Tienken and Wilson 2007), some relevance can also be assumed for HEI. Under the assumption that digital media behaviour is at least partly affected by self-efficacy expectations regarding digital media (applications), a closer look at digital media self-efficacy seems promising to analyse factors for study success.

To our knowledge, there is no up-to-date and suitable scale for assessing media-related self-efficacy. Possible scales are either outdated (Compeau and Higgins 1995) or rather specific, focusing on internet search (Eastin and LaRose 2000), social media (Hocevar 2013), information search (Vishwanath 2007), or communication. The scale for media self-efficacy (Hofstetter et al. 2009) on the contrary is too broad for our purposes. In consequence, we developed a scale for investigating DMSE that is not too specific, in order to address a broad range of different digital media and also not to wide-spread, to assure for validity and reliability of the scale.

2 Design and Sample

In order to empirically observe determinants of students’ academic behaviour, media use, and related attitudes, we developed a standardized questionnaire. Multiple instruments are arranged in three thematic blocks to capture self-efficacy expectations as well as emotions, motivation, media-usage behaviour, and socioeconomic factors. On that account, we chose instruments that are either research standards used in current research in the subject area or were constructed based on those standards, as is briefly described below.

2.1 Instrument Design

In Tables 1 and 2, a list of the scales addressing attitudes, motivation, and behaviour either in the general academic context (Table 1) or related to digital media (Table 2) and an example item for each of the scales is shown. We consistently used 7-point Likert scales for all of the psychometric measurements, e.g. ranging from ‘totally disagree’ to ‘totally agree’.

Table 1 Scales and example items for attitudes and study-related motives
Full size table
Table 2 Scales and example items for media behaviour and media-related attitudes
Full size table

Evaluation concerning studying (including emotions, motivation, and attitudes) is undertaken by partly adapting the scales for the ‘assessment of students’ attitudes towards studying’ (Brahm and Jenert 2015) and CHE-QuestFootnote 1 (Leichsenring 2011), which includes a scale for integration (Tinto 1993). The instruments used in Brahm and Jenert (2015) appear to be well suited to the present research project since they address attitudes towards the university as an institution and therefore the students’ social and contextual environment and attitudes towards studying (e.g. support from important people, emotions of anxiety and joy), while taking into account self-efficacy expectations and attitudes towards learning (e.g. autonomy in learning processes). Academic self-efficacy expectation (ASE) is measured with the corresponding instrument by Jerusalem and Schwarzer (2002). Additionally, instruments for intrinsic motivation and extrinsic goal orientation as well as perceived academic achievement are included from Brahm and Jenert 2015. To measure the big five personality traits, we as well included the BFI-10, a 10-item scale with two items each for the dimensions extraversion, agreeableness, conscientiousness, emotional stability, and openness (Rammstedt et al. 2013).

Aim and frequency of students’ media use and different attitudes regarding those media are questioned according to approved instruments by Grosch and Gidion (2011) and Zawacki-Richter (2015). Computers, tablets, and smartphones are seen as digital media equipment, while software tools (e.g. for text or spreadsheet processing, picture editing), research tools, search engines, and other online media tools, are summarized as digital media itself. The measurement of attitudes towards digital media is divided into several units, dealing with the overall evaluation for example of usefulness and concerns (e.g. privacy and data security concerns) and also the evaluation of usefulness with regard to academic studies.

Based on the general self-efficacy scale by Schwarzer and Jerusalem (2010), a scale for DMSE is newly constructed to capture students’ media-related self-efficacy. Self-assessed knowledge resp. skills regarding digital media applications are included as well.

Constructs and scales for age, educational qualification, nationality, occupational status, and income are based on a study conducted at a German university in 2014 (Lang and Hillmert 2014). In line with this study, parental characteristics (e.g. educational qualification) are measured as well. The data will allow classifying the respondents’ socioeconomic status according to the ‘International Socio-Economic Index of Occupational Status (ISEI)’ (Ganzeboom and Treiman 2003).

Besides, gender, subject of study and the number of semesters are included which will allow for subgroup analyses regarding gender, subjects and study experience, in addition to the four university contexts.

In summary, the questionnaire includes scales for self-efficacy, goal orientation, emotions associated with studying, media use, and related attitudes and demographic factors. All in all, this comprehensive instrument, thus, allows data to be collected for analyses of the complex relationships described above. Since the psychometric quality of the instrument must be guaranteed for these analyses, the instrument’s validity and reliability are the focus of the study at hand.

2.2 Data

To ensure the adequacy of the chosen scales and items, a first version of the developed instrument was given to experts in the field of educational research. These experts were three professors and four graduate students. After implementing the received feedback, the revised version of the online-questionnaire was given to two undergraduate students to finally check for wording, comprehensibility, and processing time.Footnote 2 Both the pre-test version and the final version of the questionnaire have been approved by the ethics committee for psychological research at the university Q.

We decided to recruit participants partly via mass emails to invite them to participate voluntarily.Footnote 3 Due to this nonprobability sampling method, the representativeness of the sample collected was not guaranteed. However, our goal was to reach a wide variety of participants from all faculties of the universities and not a representative sample. To attract more participants, we also used flyers and posters, information screens (e.g. in the library), and announcements in lectures. Also, we raffled vouchers and prices such as IPads or speakers.

The pre-test-survey was conducted using the online-questionnaire at three HEI in Switzerland and Germany at the end of 2017. In total, about 2000 students of different subjects received a link to the questionnaire. A total response of 171 cases was gained (response rate approximately 8.6%) of which 63 responded to every question of the survey.

The data of the main study was collected in the summer term 2018 at four German Universities X, Y, Z, and Q. In sum, 135,464 enrolled students were addressed and 3342 participated (response rate approx. 2.5%). The number of participants who completed the whole questionnaire is 1925 (response rate for complete questionnaires: 1.4%). Table 3 shows the absolute response number per HEI. The statistics reported below refer to the data of the main study but are roughly consistent with those found for the pre-test data unless otherwise stated.

Table 3 Higher Education Institutions of the participating students
Full size table

The proportion of female participants in the sample is 59.6%, 39.6% are male. The students’ mean age is M = 24.03 (SD = 4.01, min = 18, max = 59). As expected, our sample is not representative of the German students’ population in general since female students are slightly overrepresented in our sample.Footnote 4

We purposefully addressed students enrolled in a multitude of different subjects to reach enough variability in our sample to establish the instrument’s psychometric quality. In future studies, it would, therefore, be possible to analyse sub-populations in order to complement other studies (e.g. Zawacki-Richter et al. 2015) who for instance, did not include the humanities’ students. Also, in contrast to those studies, we focus on traditional students since our data was collected at universities without a considerable amount of distance or online education programs in 2018. Thus, the data allows exploring the digital media usage patterns of a more general, heterogeneous student population, while highlighting differences in study subjects and gender.

In our analyses (see below), we included only cases with at least 50 completed pages of the 119 pages of the questionnaire, resulting in 2039 cases. However, some analyses required complete cases so the number of cases was reduced due to missing values in some instances, as is mentioned below. To ensure construct validity and reliability, exploratory and confirmatory factor analyses, as well as correlation and internal consistency analyses, were applied on the pre-test data as well as on the full-scale data. The data analyses were conducted in R (R Core Team 2019).

3 Results

3.1 Internal Consistency Analyses

Each value of Cronbach’s Alpha for the main study data (see supplementary material for the results of the analyses for the pre-test) is above the common threshold of 0.70 (see Table 4). Any item showing an item-scale correlation lower than 0.50 was not part of the main survey resp. excluded afterward.

Table 4 Internal consistency measure by Cronbach's Alpha of the used scales
Full size table

Both self-efficacy scales show high values for Cronbach’s Alpha (ASE α = 0.92; DMSE α = 0.92) and also high item-scale correlations (r > 0.69) for each item, for the full-scale survey.Footnote 5 However, in order to further reduce the number of items, two items that referred to similar aspects of self-efficacy as two other remaining items (explicit problem-solving and reaction to surprising situations) were excluded for the revised instrument. An additional item was excluded from the main study data because of its low item-scale correlation (< 0.50). Each scale now consisted of seven instead of ten items. The mean scores of both self-efficacy dimensions show only a moderate correlation of r = 0.38. Moreover, a joint exploratory factor analysis of the 20 original items of both scales in the pre-test data clearly leads to the two dimensions of ASE and DMSE. Therefore, based on the pre-test data and also the full-scale data, the scales appear to provide valid and reliable measurements for both distinct self-efficacy dimensions.

3.2 Validity Analyses

3.2.1 Exploratory Factor Analyses

For the pre-test data, the attitude and motivation scales taken from Brahm and Jenert (2015), CHE, the two scales for ASE, and DMSE were tested for construct validity by conducting several exploratory factor analyses (EFA).Footnote 6 Due to a large number of expected factors, a single factor analysis containing all items was not useful. Instead, the analyses were carried out at the construct level. However, theoretically close constructs, e.g. intrinsic motivation and enjoyment in studying, were also checked in a comprehensive factor analysis including items of more than one construct.

First, each of the scales was evaluated in terms of the Kaiser–Meyer–Olkin (KMO) measure of sampling adequacy test (KMO > 0.60) and the Bartlett's test of sphericity. The factor models itself were evaluated in terms of Scree plots, factor loadings (higher than 0.30), parallel analyses (comparison to the structure of random data), and in accordance with theoretically expected factor structure and interpretability. For each scale, items that showed loadings greater than 0.30 on more than one factor are excluded for interpretation and also for the revised survey instrument, used for the full-scale data collection (see also supplementary material, Table A1). Still, it was ensured that there were at least three items for every uni-dimensional scale.

Most of the scales showed the expected one-factor structure. However, against our theoretical expectation, the factor structure of the original social integration scale (Leichsenring 2011) could not be replicated. In contrast to our theoretical expectation, two factors can be suggested in case of the original social integration scale (Leichsenring 2011), based on the pre-test data analysis. These might roughly be interpreted as social and academic integration since the items of one factor refer to private activities, partly with fellow students (e.g. private meetings, attendance of parties), and those of the other factor refer to learning activities with classmates (e.g. group learning activities). The scale was unchanged for the full-scale data collection and the exploratory factor analysis was repeated. For the full-scale data, a three factor solution seems most appropriate, based on the EFA. Those three factors distinguish between social and academic integration, as suggested above, but also free-time activities. As a consequence, only those items that could be uniquely defined as reflecting social integration, both theoretically and empirically, for the pre-test and also full-scale data, will be considered in further data analyses.

3.2.2 Confirmatory Factor Analyses

The final 11 factor-solution was validated for the full-scale data by conducting a confirmatory factor analysis (CFA). The results confirm a good fit of the model, based on comparative fit index (CFI > 0.95), Tucker Lewis Index (TLI > 0.95), root mean square error of approximation (RMSEA < 0.05), and standardized root mean square residuals (SMRM < 0.05) values (Brown 2015). The fit indices are reported in Table 5 (see also Output 1, supplementary material).

Table 5 Fit Indices for 11-factor CFA based on the full-scale data (n = 1457)
Full size table

3.2.3 Correlation Analyses

In Table 6, the Pearson product-moment correlation coefficients of test scores for the scales are given to check if these are in line with the reported results of Brahm and Jenert (2015).

Table 6 Pearson product-moment correlation coefficients (using pairwise exclusion of missing values) of test scores for each scale (n = 1931)
Full size table

The nomological validity of ASE is shown by expected correlations with other constructs (see Sect. 1.2), e.g. a negative correlation with anxiety when studying (r = − 0.52) or a positive correlation with motivation (r = 0.39). Also, motivation is positively correlated with enjoyment (r = 0.84), active participation in classes (r = 0.31) or identification with the university (r = 0.43). Students who are supported by their personal environment (subjective norm) in studying at their university also show a high identification with this institution (r = 0.45). Altogether, these results confirm the reported findings of Brahm and Jenert (2015).

Additional correlation analyses (not included in Table 6) of DMSE, self-assessed digital media knowledge/skills, and frequency of usage of certain media applications also appear plausible. Self-assessed skills in e-learning applications or programming are positively correlated with DMSE, in medium strength (r = 0.35 for e-learning; r = 0.37 for programming), whereas research in library catalogues or general online literature research are less strongly correlated with DMSE (r = 0.14 for library research; r = 0.20 for general literature research), to name only a few examples. Since the former presumably also require more elaborated skills in dealing with these media applications, a stronger correlation with a perceived high ability to deal with digital media in general (high DMSE) seems to be reasonable. In contrast, less elaborated skills are probably needed when dealing with literature databases or library online-services, so the ability to successfully face difficulties regarding digital media applications seems less relevant and the link between those is less strong. Similar results occur when analysing the relationship between DMSE and frequency of usage of certain media applications, such as cloud services (r = 0.29) or cooperation tools (r = 0.21) in comparison to no visible relationship at all between DMSE and usage of library services (r = 0.02) or the university webpages (r = 0.03). Again, this could be explained by the different demands on the skills required, with higher skills needed for online tools than for browsing through simple webpages. In sum, the relationship between DMSE and on the one hand self-assessed capabilities as well as on the other hand the frequency of use of certain media applications seems to vary depending on the requirement level of these applications, which seems plausible to us and thereby underlines the validity of the DMSE scale.

3.3 Distributions of Self-Efficacy Scales and Descriptive Results

In Fig. 1a, b, the distribution of the two self-efficacy dimensions is shown. In both cases, skewness differs from a normal distribution, slightly tending to the right. Still, a reasonable amount of variance is given in the data and the emphasis lies on the middle area of the Likert scale. Thus, multivariate analysis procedures such as regression analyses seem to be applicable.

Fig. 1
figure 1

Histograms of testscores (average itemscores) for a Academic Self-Efficacy and b Digital Media Self-Efficacy (n = 1955)

Full size image

Based on the pre-test and full-scale data, first descriptive results show the aims and frequency of students’ media use. While nearly each of the respondents owns either a laptop (94%) and a smartphone (96%) and has internet access (99%), a tablet-PC is owned by 45% of the students. According to the data, these mobile devices are mostly used on campus, for example, to look up something online, for online research (for study purposes), text messages, or e-mails to lecturers. The usage varies depending on the device type, for example, smartphones are used predominantly for communication and search purposes whereas laptops are used for access to university platforms, internet search, and also writing tasks (e.g. assignments). In contrast, location-based services, taking pictures, posting content, or communication on learning management systems (LMS) are rarely mentioned by the respondents. In terms of certain media applications, text-processing software, search engines, university e-mail account, chats, and e-books are used very frequently while e.g. massive open online courses (MOOCs), blogs, e-portfolios or twitter are seldom used, if at all. However, these many different media applications and differences in frequency of use as well as relatedness to study purposes allow for a more detailed analysis of media usage types and its link to study performance in subsequent investigations.

4 Discussion

Different instruments are available to assess student teaching and learning, however, instruments assessing the different facets of academic studies in the context of digitalisation are rare and currently more relevant than ever. In consequence, this study aimed to develop a comprehensive survey instrument that addresses the multi-faceted character of academic studies and digital media behaviour. Existing survey instruments such as ASAtS (Brahm and Jenert 2015), the CHE-Quest (Leichsenring 2011), and scales regarding students’ media use and attitudes (Grosch and Gidion 2011; Zawacki-Richter 2015) were combined to assess all aspects of students’ studying. In addition to these established scales, students’ self-efficacy expectations regarding digital media use have not been assessed up to now. In consequence, it was necessary to develop a new scale building upon the established self-efficacy scale by Jerusalem and Schwarzer (2002).

Both with pre-test data and the main study, the instrument proved to show valid and reliable results. In accordance with Bandura’s SCT (Bandura 2011) and also in line with other empirical research, the study showed positive correlations of students’ self-efficacy with motivation and goal orientation (Hsieh et al. 2007), interest (Honicke and Broadbent 2016) as well as study success (Bartimote-Aufflick et al. 2015; Honicke and Broadbent 2016). In contrast, a negative correlation with anxiety (Hsieh et al. 2012) could be confirmed. These correlations were shown both in the study by Brahm and Jenert (2015) as well as in our data, thus, indicating nomological validity of the survey instruments. However, we could not replicate the social integration scale according to (Leichsenring 2011), with our available data. Furthermore, the pre-test and main surveys led to different results. For future analyses, therefore, only those three items that can be clearly assigned to a scale for social integration should be considered.

In addition to this replication, a new scale for DMSE was developed and could be separated from the well-established construct of students’ academic self-efficacy (Jerusalem and Schwarzer 2002). Correlation analyses of the relationship between DMSE and self-assessed capabilities as well as the frequency of usage of certain media applications show a varying strength of the relationship with DMSE, depending on the requirement level of the application of consideration, which appears to be plausible. Our results, therefore, point to a valid scale for DMSE here as well.

Another important result concerns students’ aim and frequency of media use which matches those reported by Zawacki-Richter et al. (2017). We found a further increase in the dissemination of smartphones, laptops, and tablets, in comparison to the findings based on a survey in 2012 and 2015 (Grosch 2012; Zawacki-Richter 2015) with 96% of our student sample owning a smartphone, 94% a laptop, and 45% a tablet. This increase again highlights the increasing relevance of digital media at universities and thereby of research on digital media behaviour of university students.

4.1 Limitations

It is always challenging to obtain a sufficiently large sample. Of course, the use of large samples can lead to more reliable results and can further enhance instrument validity tests. This scale development is based on a small pre-test and a larger main study, thus, the validity and reliability of this newly developed scale should still be examined in further research. Results are already promising, however, the scale has up to now only been used in one Swiss and six German HEIs. Further research will be required to replicate and extend these findings in diverse contexts, i.e. different kinds of HEIs, such as universities of applied sciences or colleges in order to establish broad applicability of the scale. In addition to institutional diversity, the study could also be replicated outside the German-speaking higher education context.

The most notable limitation is the exclusive use of self-reported data in the questionnaire. In the meta-analysis by Kuncel, Credé, and Thomas (2005) the reliability of self-reported results is related to students’ school performance. This means that self-reported grades are appropriate measures of actual grades for students with good grades, but not for those with low grades. In consequence, other measures may be needed.

Furthermore, the validity of the instrument must be considered preliminary since this study focused on evaluating the factor structure, internal consistency, concurrent validity, and the divergent validity of the newly developed scale. Future research should include a more comprehensive evaluation of the scale, such as the test–retest reliability and criterion-related validity. Since convergent and discriminant validity could be established, criterion-related validity can be assumed, however, needs to be checked, ideally using longitudinal data.

4.2 Research and Practical Implications

Since the prevalent use of digital media for study purposes are confirmed in this study, it is necessary to extend existing instruments on teaching and learning (e.g. Lemos et al. 2011) by integrating the usage of digital media as well as self-efficacy beliefs concerning digital media. Furthermore, our study confirms the importance of our initial overarching questions. As we assume that students’ social backgrounds, e.g. parents’ educational background are linked to their DMSE and behaviour in academic settings in general (Zimmerman et al. 1992; Zimmerman 2000b), it should be investigated whether students equipped with higher DMSE can accomplish study demands better. Accordingly, research is needed on whether students’ individual and subject-specific academic and media behaviour depend upon other measured constructs such as motivation, attitudes, and their socio-economic background. Particularly about media behaviour, major subject-related differences, for example between engineering and humanities, can be expected, and especially regarding students’ self-efficacy expectations, and gender differences (Huang 2013; Pintrich and Schunk 2002; Schunk and Pajares 2002; Zawacki-Richter 2015).

Since the instrument has good psychometric properties, it can be recommended for application at other HEIs to examine students’ digital media usage and study behaviour. Also only selected parts of the instrument could be used, depending on the (research) question the applicants of the instruments have. In the summer semester 2020, for example, the instrument presented is used to conduct a longitudinal study on changes in media-related behaviour, attitudes, and self-efficacy as a result of the (at least partially) digitalised teaching during the corona-crisis. Besides, for example, the use and acceptance of learning platforms and campus information systems at universities could be evaluated to guide quality enhancement processes at HEIs. In this context, a study using the instrument could give insights into the purposes that widespread digital devices, such as smartphones and laptops, are actually used for. If a HEI, for instance, intends to increase the use of LMS as part of students’ learning environment, and if students often use the LMS with their smartphones, it is advisable to make them universally usable with smartphones. Based on the assumption that a high level of media-related and academic self-efficacy motivates students and promotes their goal orientation (van Dinther et al. 2011), interventions for enhancing the students’ (academic and digital media) self-efficacy expectations, potentially especially focusing on disadvantaged students, could be developed and evaluated, using the presented scales (see Brahm and Pumptow 2020).

In particular, the role of DMSE should be addressed in future research since digital media are prevalent in teaching and learning. A possible approach would be to model different clusters of media usage types, based on the data concerning digital media. This would allow for further analysis of students’ media behaviour in line with concepts resp. dimensions of media-usage by Johnsson-Smaragdi (2005; see also Zawacki-Richter, 2015). Such an analysis could provide insights into the determinants of digital media behaviour and also regarding its relevance for academic achievement.

5 Conclusion

Overall, this study contributes to further differentiation of established scales to evaluate students’ behaviour (e.g. Brahm and Jenert 2015; Lemos et al. 2011). Above all, it adds a new perspective to this research stream by developing a new scale to assess DMSE which seems to be an important construct to further explore students’ use of digital media in HEI. In this respect, the research also makes an initial contribution to extend Bandura’s SCT towards digital media usage in the context of HE: Academic self-efficacy and digital media self-efficacy can be conceptualized as two separate constructs. This distinction could, in turn, be used to make clearer differences between various student groups and better cater to their respective needs in teaching and learning. In summary, the instrument can be recommended for wider application at other HEIs to find out more about students’ digital media usage and its implications for other facets of learning.