1 Introduction

The ability to effectively use ICT (information and communication technologies) plays an important role in schools, many workplaces and in people's everyday lives (Ferrari, 2012; Fraillon et al., 2019). Therefore, researchers as well as organizations (e.g., European Commission, International Society for Technology in Education) have developed frameworks to promote ICT literacy by describing competencies ans skills that are considered important for the knowledge society (Siddiq et al., 2016; Voogt & Roblin, 2012). Additionally, many countries have integrated ICT into national school curricula (Fraillon et al., 2019). Against this backgorund, ICT literacy is understood as a meta-competence that helps people to acquire important competencies and skills for educational and work situations and to achieve private goals over the entire life span (Ferrari, 2012; van Laar et al., 2017). More recent conceptualizations of ICT literacy integrate technological and cognitive aspects to define ICT literacy. For example, the widely used definition of the Educational Testing Service (ETS) describes ICT literacy as “using digital technologies, communication tools and / or networks to access, manage, integrate, evaluate and create information in order to function in a knowledge society” (p. 16; see also Siddiq et al., 2016). This means that solving information-related tasks (e.g. searching for trustworthy information on the internet) requires both technical knowledge and cognitive skills (e.g. problem-solving skills) (Walraven et al., 2008).

Despite the importance of ICT literacy for successful participation in society and educational policy initiatives to implement the use of ICT in schools, little is known about how ICT knowledge and skills are acquired and which variables foster the acquisition of ICT literacy (Hatlevik et al., 2018; van Deursen & van Dijk, 2015). To answer these questions, several studies have investigated the power of different ICT-related person characteristics (e.g., self-efficacy, motivations) in predicting ICT literacy on standardized tests (Aesaert et al., 2015; Senkbeil, 2018; Claro et al., 2012; Hatlevik et al., 2018; Zylka et al., 2015). However, these studies either focused on one or very few ICT-related variables or examined the predictive power of ICT-related variables only cross-sectionally. Furthermore, important prerequisites for educational success such as intelligence or prior knowledge were not taken into account in these studies (Aesaert et al., 2015). Thus, the aim of the present study is to examine the incremental validity of various ICT-related person characteristics over and above intelligence and prior achievement when predicting ICT literacy.

The present study extends the field of research on ICT literacy in several aspects: First, numerous ICT-related constructs (e.g., trust, motivations, experience, frequency and breadth of use) were simultaneously examined as predictors of ICT competence. Furthermore, the relative importance of each predictor was determined by using the relative weights approach (Tonidandel & LeBreton, 2015). Second, this is one of the very first studies which uses a longitudinal approach, i.e., students' ICT competencies were measured twice. Therefore, in addition to ICT-related constructs, intelligence and prior achievement can be used simultaneously to predict ICT competence. This allows us to examine the extent to which ICT-related constructs predict change in ICT literacy. Third, we used data from the longitudinal National Educational Panel Study (NEPS) which analyzes educational processes and competencies of persons (e.g., ICT literacy) from different age cohorts in longitudinal studies by tracking fully representative samples (see Blossfeld et al., 2011). For this study, a representative sample of students in lower secondary education from Grade 9 in Germany was drawn (see also Kriegbaum et al., 2015 for a similar analysis for the domain of mathematics).

2 Theoretical background

2.1 Prior achievement and intelligence as predictors of academic achievement

Prior knowledge is probably the most important determinant of student achievement. It comprises all the relevant knowledge that the student has acquired in the past and positively influences both knowledge acquisition and the ability to apply higher-order cognitive problem-solving skills (Dochy et al., 1999). Although it forms the basis for understanding the new material and is therefore important for student achievement, most studies do not measure prior knowledge (Nachtigall et al., 2008). However, with longitudinal designs it is possible to use this information about the initial level of performance in "value-added" analyses. These allow for a more detailed analysis of student performance progress, as the effects of various factors influencing student progress can be disentangled and measured more accurately (Gustafsson, 2010).

Research has also shown that intelligence is a strong predictor of academic achievement. Correlations between measures of general intelligence and measures of educational achievement have been found to be around r = 0.50 (Gustafsson & Undheim, 1996) and between r = 0.43 and r = . 77, respectively (Deary et al., 2007). Since ICT literacy, as defined above, focuses on solving information-related tasks using ICT environments, intelligence (e.g., reasoning, critical thinking) should be an important prerequisite for this competence (ETS, 2002; Ferrari, 2012). For example, students with a higher analytic intelligence, i.e., the ability to deal with novelty and to adapt his or her thinking to a new cognitive problem (Carpenter et al., 1990), should also have a higher ability in dealing with ICT-related problems such as searching for trustworthy information on the internet, e.g., specifying search terms, judging search results and evaluating source and information (Walraven et al., 2008). Accordingly, various studies reported high correlations (0.43 ≤ r ≤ 0.69) between ICT literacy and general intelligence (Aesaert et al., 2015; Senkbeil et al., 2013a; Senkbeil, 2018).

2.2 ICT-related person characteristics as predictors of ICT literacy

Due to the rapidly changing technological environment and the fact that most of the young people acquire ICT skills on their own (Zhong, 2011), self-regulated and continuous life-long learning is a key factor for successfully keeping pace with recent developments in the area of ICT (Senkbeil, 2018; Fraillon et al., 2019; Zylka et al., 2015). Thus, it is assumed that ICT-related person characteristics such as motivations, attitudes, frequency of usage or years of experience using digital media are important prerequisites for acquring ICT literacy (Aesaert et al., 2015; Claro et al., 2012; Hatlevik et al., 2018).

2.2.1 ICT motivations: competence beliefs and usage motives

Recent research has emphasized the relevance of motivational factors (e.g., self-efficacy, self-confidence, usage motives) for ICT usage and ICT literacy (Christoph et al., 2015; Hatlevik et al., 2018; Moos & Azevedo, 2009; Senkbeil & Ihme, 2017a). Among motivational constructs, subjective self-beliefs about ICT skills (e.g., self-efficacy, self-confidence) have been identified as the main determinants of ICT literacy. Moderate correlations with ICT literacy were reported for ICT self-efficacy (0.26 ≤ r ≤ 0.44; Fraillon et al., 2019; Goldhammer et al., 2013: Senkbeil & Ihme, 2017a) as well as ICT confidence (0.20 ≤ r ≤ 0.35; Bradlow et al., 2002; Bunz et al., 2007; Katz & Macklin, 2007).

Both variables, self-efficacy beliefs and self-confidence, are also important for self-regulated learning, and are interrelated in many respects, as they both base on judgments of personal competence (Stankov et al., 2012). Self-efficacy refers to the perceived abilities of individuals and reflects what individuals believe they can do with the abilities they possess (Bandura, 1986), whereas self-confidence is a metacognitive and probabilistic judgment of performance that can be viewed as an indicator of metacognitive monitoring processes. For example, participants are asked to make a judgment about the number of correctly given answers after completing a test (Schraw, 2009; Stankov et al., 2012). Self-efficacy beliefs play a major role in self-regulation as they influence key components of the self-regulation process, e.g., effective learning strategies or performance monitoring and judgment. In turn, these self-regulatory activities affect self-efficacy. As individuals work on tasks, they monitor and judge their performances, and positive metacognitive judgments enhance self-efficacy and motivation (Schunk, 1996). Correspondingly, self-efficacy, and self-confidence are strongly correlated with each other (0.54 ≤ r =  ≤ 0.88; Chen, 2003; Morony et al., 2013).

Furthermore, ICT usage motives represent motivational incentives to satisfy certain needs by using digital media and form the basis for two overarching media orientations: an instrumental orientation and a social interaction orientation. An instrumental orientation involves media usage for goal-directed motives and activities such as information seeking, and learning and working (e.g., searching the internet for educational purposes). A social interaction orientation involves motives such as social exchange (e.g., chatting online), and self-presentation (Senkbeil, 2018). Instrumental and social interaction media orientations differ in their cognitive involvement. Cognitive involvement refers to the mental processes in acquisition and understanding media messages and is demonstrated in active participation of information processing. The mental processes include attention (selectivity of response that requires effort and allocation of cognitive capacity), recognition (comparing incoming information to known patterns in long-term memory), and elaboration (relating incoming information to existing knowledge) (Senkbeil & Ihme, 2017a; Perse, 1990). Research has shown that an instrumental media orientation is positively associated with higher cognitive involvement and greater knowledge of metacognitive strategies, whereas a social interaction media orientation is related to lower cognitive involvement and poorer knowledge of metacognitive strategies (Lee & Wu, 2013; Naumann, 2015). Correspondingly, an instrumental ICT orientation (motives and activities) is positively (0.15 ≤ r ≤ 0.30) and a social interaction ICT orientation is negatively associated (-0.25 ≤ r ≤ -0.15) with ICT literacy or related constructs such as digital reading (Senkbeil et al., 2013b; Lee & Wu, 2013; Naumann, 2015).

2.2.2 Other ICT-related person characteristics: access, experience, and usage

Additionally, it is argued that ICT-related person characteristics such as access, experience, usage at home, or usage at school should positively affect the acquisition of ICT literacy (Aesaert et al., 2015; Claro et al., 2012; Fraillon et al., 2019; Hatlevik et al., 2018). First, ICT access refers to the opportunities that parents create for their children to develop ICT competences by providing them with the necessary technological infrastructure (Aesaert et al., 2015). Second, students with longer use of digital media (ICT experience) have more oppurtunities to develop sufficient ICT skills in authentic contexts (Verhoeven et al., 2016). Furthermore, as they have been internet users for a longer period of time, they are expected to be better at finding information online because they have more experience to draw on (Hargittai, 2010).

Third, frequency of ICT usage at home refers to ICT literacy as a behavior-based competence and skill that requires practice with digital media (Aesaert et al., 2015; Fraillon et al., 2019). It is assumed that different activities using digital media (e.g., accessing information on the internet, playing video games, use of communication tools) may help to advance children ‘s knowledge in ICT literacy (Hargittai, 2010) because they are associated with different cognitive requirements (e.g., problem-solving skills when searching useful and trustworty information on the internet or working memory when playing video games; Walraven et al., 2008; Granic et al., 2014) so that they complement each other in building relevant ICT skills. Thus, a great breadth of ICT activities can therefore be understood as a larger number of learning opportunities to gain basic ICT skills (Claro et al., 2012; Hargittai, 2010).

Fourth, the use of ICT is increasingly becoming standard practice in school education and is therefore an important part of preparing young people for participation in modern society. The use of digital media in school should lead to an improvement in learning and the development of skills that enable people to get along in a changing global society (Fraillon et al., 2019; OECD, 2019). For example, they „allow greater ease in conducting research (due to the vast amount of information on the internet), more empirical investigation (as students used laptops or laptop-connected scientific probes to gather or analyse data in the classroom), and more opportunities for in-depth learning (classes could pursue subjects of investigation from multiple angles using computer- and internet-based resources)“ (Warschauer, 2008, p. 61). Accordingly, a number of studies have shown that the ICT-related person characteristics considered in this section (access, experience, use) are weakly positively associated with ICT literacy (0.10 ≤ r ≤ 0.20; Fraillon et al., 2019; Hargittai, 2010; Hatlevik et al., 2018; Senkbeil et al., 2013b). However, it should be noted that not all studies report significant associations for the aforementioned variables (e.g., use in school, experience: Claro et al., 2012; van Deursen & van Diepen, 2013).

In sum, cross-sectional research thus shows that primarily motivational constructs are moderately correlated with ICT literacy, while the other ICT-related variables are generally only weakly correlated or, in some studies, not correlated at all with ICT literacy.

3 Incremental power of ICT-related person characteristics in ICT literacy and predictors of change

So far, very few studies have examined whether different ICT-related person characteristics predict ICT competence over and above intelligence. For example, Aesaert et al. (2015) found that ICT self-efficacy and ICT-related instrumental activities (ICT use as an information tool) explained additional variance in ICT literacy over and above intelligence. Correspondingly, Senkbeil (2018) showed that instrumental and social interaction use motives predicted standardized test performance in ICILS (International Computer and Information Literacy Study) 2013 after controlling for intelligence (2.6% additional variance; Senkbeil, 2018). Consistent with the explanations in Sect. 2.1, intelligence had greater explanatory power than the ICT-related variables in both of these studies. Thus, intelligence should be the best predictor of ICT proficiency on standardized tests. The dominant role of intelligence is plausible because it encompasses the ability to reason, is strongly related to working memory, and is essentially correlated with achievement tests (Gustafsson & Undheim, 1996). On the other side, these results also suggest that ICT-related motivational constructs have incremental power in predicting ICT literacy. Since continuous, lifelong learning is a key factor in successfully keeping pace with current developments in ICT, it is also of great interest to know which variables predict changes in ICT competence. Therefore, further empirical work with longitudinal designs is needed to accurately determine the relative importance of different ICT-related person characteristics after controlling for not only intelligence but also for prior achievement.

4 Research goals and hypotheses

The present study aims to explore the incremental validity of different ICT-related person characteristics over and above intelligence and prior achievement when predicting ICT literacy. As research gives rise to doubts that ICT-related constructs such as ICT usage or psychological constructs, like motivation, can make an independent contribution to the prediction of ICT literacy over and above intelligence (Moehring et al., 2016), it is important to investigate the relative power of the ICT-related variables compared to intelligence as well as prior achievement (Senkbeil, 2018; Kriegbaum et al., 2015). Given the assumed issue of variable collinearity, i.e., the predictors (ICT-related constructs, intelligence, prior test performance) are correlated with one another, the relative importance of each predictor is determined by using the relative weights-approach (Tonidandel & LeBreton, 2015). Therefore, a four-study approach was implemented: First, we cross-sectionally examined the extent to which various ICT-related characteristics predict ICT literacy after controlling for intelligence (Hypothesis 1; H1). Second, the relative importance of each predictor was determined (Hypothesis 2; H2). Third, we examined longitudinally the extent to which different ICT-related characteristics predict ICT literacy after controlling for intelligence and prior achievement (Hypothesis 3, H3). Fourth, the relative importance of each predictor was determined (Hypothesis 4, H4).

The following hypotheses were investigated:

  • H1: It was expected that all ICT-related person charcteristics (confidence, usage motives, access, experience, usage) incrementally predict ICT literacy after controlling for intelligence.

  • H2: By regressing ICT literacy simultaneously on all variables, intelligence was expected to be the strongest predictor of ICT literacy followed by both ICT-related motivational constructs (confidence, usage motives).

  • H3: Longitudinally, both motivational constructs (confidence, usage motives) incrementally predict ICT literacy after controlling for intelligence and prior achievement.

  • H4: Longitudinally, by regressing ICT literacy simultaneously on all variables, prior achievement was expected to be the strongest predictor of ICT literacy, followed by intelligence and both motivational constructs (confidence, usage motives).

5 Method

5.1 Sample and procedure

The sample is part of the German National Educational Panel Study (NEPS) that tracks representative samples of students over their life course (see Blossfeld et al. 2011).

For this study, we analyzed data of Starting Cohort 4 (SC4). SC4 focuses on the pathways of Grade 9 students through higher secondary and vocational education tracks (see Aßmann et al., 2019 for details on the sampling procedure). In 2010, a representative sample of Grade 9 students (N = 14,436; 49.8% female) in 545 schools was drawn and tested within their schools. The mean age in Grade 9 (first measurement occasion, T1) was M = 14.73 (SD = 0.71) years. Thirty-five percent of the students attended an upper secondary school (academic track), 65% attended a basic or intermediate secondary school (vocational track). Three years later, in 2013, all students who agreed to further participation were retested (second measurement occasion, T2; N = 5,545; 53.9% female). Students who were still in school (i.e., 12th grade) were tested again within their schools (3,719 students), while students who had left school were tested individually at home (1,826 students).

5.2 Instruments

5.2.1 ICT literacy

The measurement of ICT literacy consists of two multiple-choice tests assessing technological as well as information-related knowledge: the Test of Technological and Information Literacy (TILT) for Grade 9 and Grade 12, respectively. The TILT is based on the definition of ICT literacy of the ETS (2002) and is operationalized according to aforementioned facets of technological and information literacy (accessing, creating, managing, and evaluating information; see Senkbeil et al., 2013b for details). The test-takers had to deal with realistic problems embedded in a range of authentic situations. To do so, most items used screenshots, for example, an internet browser, an electronic database, or a spreadsheet as prompts. To make the design as realistic as possible, screenshots are integrated into the task stimulus (e.g., of an internet browser). Each item required a multiple-choice response that asked test-takers to identify a correct solution from up to six response options (Senkbeil et al., 2017a; see Fig. 1 for an example item).

Fig. 1
figure 1

Copyright Leibniz Institute for Educational Trajectories (LIfBi). Reproduced with permission

Example item of the ICT literacy test.

Full size image

Different tests were given at both measurement occasions with a processing time of 30 min each (T1: 36 items, T2: 31 items). Both tests were scaled based on item response theory and could be linked by using link samples (so-called anchor-group design; Fischer et al., 2016; Pohl & Carstensen, 2013). Competence scores were estimated as weighted maximum likelihood estimates. The IRT-based reliabilities of the two tests were 0.83 and 0.74, indicating an acceptable to good reliability of measurement (see Senkbeil & Ihme, 2012, 2017b for further psychometric properties of the two tests).

5.2.2 Intelligence

Intelligence was measured in Grade 9 with the Raven-type matrices test NEPS-MAT that includes 12 items. In NEPS-MAT, each item consists of several horizontally and vertically arranged fields containing different geometric elements – with only one blank field. For each task, the logical rule underlying the pattern of geometric elements must be deduced. The testtaker has to choose the correct solution from various response options (Brunner et al., 2014). The number of correctly solved items provides an estimator of fluid intelligence that is closely related to general intelligence. The reliability of the scale was acceptable (Cronbach's α = 0.72).

5.2.3 ICT confidence

To assess ICT confidence in NEPS, the testtakers were asked to estimate their own performance after completing the test, i.e., they were were asked to estimate the number of correctly given answers. The estimated proportion of items solved correctly, served as an indicator of metacognitive judgment of performance (Schraw, 2009).

5.2.4 ICT usage motives

The concept of ICT usage motives was measured in Grade 9 with eleven items adapted from Senkbeil (2018) on four-point response scales (from 1 = not important to 4 = important). Following the classification scheme described above, ICT usage motives were conceptualized as a higher-order factor model with two second-order factors (instrumental, and social interaction motive factor) and four first-order factors (usage motives). The usage motives information seeking (three items) and learn and work (three items) were assigned to the instrumental factor, and the usage motives social exchange (two items) and self-presentation (three items) were assigned to the social interaction factor (Senkbeil, 2018). A confirmatory ordinal factor analysis using weighted least squares mean and variance (WLSMV) estimation revealed that the higher-order factor model fitted the data well, χ2 (41, N = 2075) = 1241.704, p < 0.001, RMSEA (Root Mean Square Error of Approximation) = 0.045; CFI (Confirmatory Fit Index) = . 982. Table 1 shows the items of the model and their descriptive statistics. The first- and second-order factor loadings were significant and equal to or above 0.49 with the majority of the loadings above 0.70 (see Fig. 2).

Table 1 Items, means (M), standard deviations (SD), and corrected item-total correlations (rit) of the higher-order factor ICT motivation scale
Full size table
Fig. 2
figure 2

Higher-order factor CFA model of the ICT usage motives

Full size image

5.2.5 Other ICT-related person characteristics

Regarding access to digital media, students were asked if they own a computer (desktop, notebook, or tablet) or if they could use a computer at home (0 = no, 1 = yes). ICT experience was measured with a single item that asked students how long they have been using digital devices (from 1 = never or less than one year to 5 = six years or more). The frequency of ICT usage at home and at school was assessed on six-point rating scales (from 1 = never to 6 = every day). The breadth of ICT usage was measured by six items (six-point rating scale from 1 = never to 6 = every day; Cronbach's α = 0.60) which were combined into one scale score. The six items represent different activities with digital media (using a spreadsheet to do calculations or plot graphs; searching the Internet to find information about activities to do or intersting things; playing role-playing or strategy games; watching downloaded videos or listening to downloaded music; writing or editing documents).

5.2.6 Social background variables

Students’ socio-economic status was denoted by parental occupation and then scored using the highest International Socioeconomic Index of occupational status (ISEI; Ganzeboom, 2010) of their parents. Student’s migration background was derived from the parents` place of birth. The student was classified as having a migration background if one parent or both parents were born outside of Germany (regardless of where the student was born).

5.3 Data analyses

Descriptive statistics (means, standard deviations, and internal consistencies of the study variables) were obtained using SPSS 26.0.

5.3.1 Structural equation models

To examine cross-sectionally as well as longitudinally the extent to which various ICT-related characteristics predict ICT literacy (Hypotheses 1 and 3), structural equation modeling techniques and path analyses with Mplus (Muthen & Muthen, 2012) were used. ICT-related variables that were measured with at least two items per scale were specified as latent variables. Saturated path analyses were conducted for the other ICT-related variables that were measured with a single item (self confidence, access, experience, frequency of usage at home and at school). First, separate analyses were conducted for each ICT-related variable both cross-sectionally (Hypothesis 1: Models 2a-2 h) after controlling for intelligence (Model 1) and longitudinally (Hypothesis 3: Models 4a-4 h) after controlling for intelligence and prior achievement (Model 3; see also Kriegbaum et al., 2015).

Different fit statistics were used to evaluate the estimated model, including the χ2-statistic. χ2 value strongly depends on the sample size and is highly sensitive in large samples, other commonly used fit indices were considered as well: the RMSEA (cut-off criterion: ≤ 0.08), and the CFI (cut-off criterion: ≥ 0.90) were used (Kline, 2016). As participants were sampled from schools, the nested structure of the data was taken into account, specifying school membership as the cluster variable (Mplus option TYPE = COMPLEX).

5.3.2 Handling missing data

For the missing data mechanism Missing at random (MAR), several established methods exist that yiled unbiased parameter estimates (Enders, 2010). One of these methods is the full-information maximum likelihood (FIML)-approach in Mplus that was used in this study. As the MAR mechanism is an unverifiable assumption that influences the accuracy of maximum likelihood analyses and because there was a dropout of 61% over the course of the two measurement occasions, data theorists recommend to include auxiliary variables. These are variables that are either correlates of missingness or correlates of an incomplete variable (Enders, 2010). Auxiliary variables will improve estimation by reducing estimation bias and „may effectively convert an MNAR (Missing not at random) situation to MAR “ (Schafer & Graham, 2002, p. 173). Thus, a selectivity analysis was examined to identify auxiliary variables. Social background and individual variables (gender, school type, socio-economic status, test mode, migration background, and age) were assumed as potential auxiliary variables (see Table 2).

Table 2 Means, standard deviations, and correlations of content variables with non-response and all study variables
Full size table

5.3.3 Relative weights

The relative weights approach is a way to quantify the relative importance of correlated predictor variables in regression analysis (Tonidandel & LeBreton, 2015). Relative weights analyses were conducted in this study to determine the relative contribution of each predictor towards explaining variance in ICT literacy (Hypotheses 2 and 4). According to Johnson (2000, p. 1) relative weight can be defined „as the proportionate contribution each predictor makes to R2, considering both its unique contribution and its contribution when combined with other variables “. Since predictors in regression analyses are often correlated, relative weights analyses can be used to transform the correlated predictors into new variables that are uncorrelated with each other (Tonidandel & LeBreton, 2015) so that correct estimates are provided. These analyses were conducted separately cross-sectionally and longitudinally by using the software RWA-Web Form to estimate relative weights. The relative weights (RW) computed by the software provide estimates of variable importance using the metric of relative effect sizes representing a decomposition of the total R2 (see second and fourth column in Table 6). The percentage of explained criterion variance (%) of each variable is obtained by dividing each relative weight by the model R2 (see third and fifth column in Table 6). These rescaled weights provide estimates of the relative importance of each variable (Tonidandel & LeBreton, 2015).

6 Results

6.1 Descriptives and intercorrelations

Means (M), standard deviations (SD), and intercorrelations among all study variables are presented in Table 3. The correlations among all study variables were inspected for multicollinearity. As expected, substantial correlations can be found between some variables. First, ICT literacy showed high stability over three years (r = 0.68, p < 0.001). In addition, intelligence was highly positively correlated with ICT literacy at both measurement points (T1: r = 0.49 and T2: r = 0.46, respectively). On the other hand, the ICT-related variables were only weakly to moderately correlated with ICT literacy (|.05 |≤ r ≤|.27|) and intelligence (|.00 |≤ r ≤|.21|). This also applies for the correlations among the ICT-related variables. They ranged from insignificant (r = 0.00 for instrumental and social interaction usage motives) to moderately positive (r = 0.35 for social interaction usage motive and breadth of ICT usage).

Table 3 Means (M), standard deviations (SD), and intercorrelations among all study variables
Full size table

6.2 Selectivity analyses

There was a substantial dropout of 61% of the students over the two measurement occasions. As it was assumed that the students` dropout (i.e. refusal of further participation in the panel) was not completely random, the dropout process was examined in more detail using a selectivity analysis (Zinn & Gnambs, 2018). To do so, we regressed a non-response indicator (coded 0 = participation and 1 = refusal at the second measurement occasion) on the study variables (ICT literacy in Grade 9, intelligence, ICT-related person characteristics). Furthermore, social background and individual variables were added that were assumed to influence the dropout process (Zinn & Gnambs, 2018).

The selectivity analysis (see Table 4) showed that students that had left school and were individually tested at home were significantly more likely to refuse further participation than Grade 12 students who still attended school (test mode: B = 3.33, p < 0.001). Furthermore, proportionally more dropout was observed for students who attended basic or intermediate secondary school in Grade 9 (B = 0.38, p > 0.001), boys (B = 0.10, p < 0.05), and students with lower ICT literacy in Grade 9 (B = -0.07, p < 0.05). However, students` age, migration background, socio-economic status, intelligence, and all ICT-related variables did not influence participation or refusal at the second measurement occasion. In summary, the dropout process at the second measurement time point (T2) was at least partly due to a missing-at-random process. Consequently, the variables driving the selection process (test mode, school type, and gender) were included as auxiliary variables in structure equation models and path analyses (see also Zinn & Gnambs, 2018).

Table 4 Logit regression analysis for non-response (coded as 1 = non-response and 0 = participation) at the second measurement point
Full size table

6.3 Predicting ICT literacy cross-sectionally

Table 5 shows the results of the cross-sectional structural equation and path models predicting students' ICT literacy. All structural equation models had an acceptable or good model fit (0.053 ≤ RMSEA ≤ 0.065; 0.854 ≤ CFI ≤ 0.981), the saturated path models fitted perfectly. In accordance with Hypothesis 1, all ICT-related variables incrementally predicted students' ICT literacy over and above intelligence. As expected, the highest proportions of variance in ICT literacy were explained when intelligence and one of the motivational constructs (ICT confidence, ICT usage motives; Models 2f–h) were entered into the regression analysis together as predictors (0.26 ≤ R2 ≤ 0.28).

Table 5 Predicting ICT literacy. Results from structural equation modeling (cross-sectional) with model parameters (β, SE) and fit indices
Full size table

In line with Hypothesis 2, relative weights analyses showed that intelligence explained most of the variance in students' mathematical competence (59.3% of explained criterion variance), followed by social interaction ICT usage motives (14.1%), ICT confidence (11.9%), and instrumental ICT usage motives (8.7%). The other ICT-related person characteristics each contributed less than 3% of explained variance in ICT literacy (see Table 7).

6.4 Predicting ICT literacy longitudinally

Table 6 shows the results of the longitudinal structural equation and path models predicting students` ICT literacy. Again, the fit of structural equation models ranged from acceptable to good (0.047 ≤ RMSEA ≤ 0.068; 0.898 ≤ CFI ≤ 0.986), while the saturated path models fitted perfectly. In accordance with Hypothesis 3, when predicting ICT literacy longitudinally, the motivational constructs (confidence, instrumental and social interaction usage motives) incrementally predicted ICT literacy after controlling for intelligence and prior achievement. Additionally, this also applied for breadth of ICT usage, ICT experience, and ICT usage at school. Neither ICT access nor ICT usage at home incrementally predicted students' ICT literacy at the second measurement occasion (T2) after controlling for prior achievement and intelligence. The highest proportion of variance (R2 = 0.49) was explained by Models 4 g and 4 h using ICT literacy at T1, intelligence, and social interaction ICT usage motives (Model 4 g) or ICT confidence (Model 4 h) as predictors.

Table 6 Predicting ICT literacy. Results from structural equation modeling (longitudinal) with model parameters (β, SE) and fit indices
Full size table

As expected and in line with Hypothesis 4, prior achievement in ICT literacy was, by far, the most powerful predictor of ICT literacy at T2 (60.2% of explained criterion variance), followed by intelligence (19.2%). For the ICT-related person charcteristics, social interaction usage motives explained the highest portion of variance (8.1%) in ICT literacy, followed by ICT confidence (6.0%) and instrumental ICT usage motives (3.4%). The other ICT-related variables (i.e., breadth of usage, experience, usage at home and at school, access) each contributed less than 2% of explained variance in ICT literacy (see Table 7).

Table 7 Relative weights (RW) and percentage of explained criterion variance (%) for all predictors, both cross-sectionally and longitudinally
Full size table

7 Discussion

The present study aimed to explore the incremental validity of different ICT-related person characteristics over and above intelligence and prior achievement when predicting ICT literacy. For this purpose, the relative weights approach was used to obtain correct estimates for the variables taken into account when predicting ICT literacy. Previous research has mostly examined the influence of relatively few ICT-related variables (e.g. self-efficacy, usage at home) on ICT literacy in cross-sectional studies, and important determinants of student achievement (e.g., cognitive abilities) were only very rarely taken into account (Aesaert et al., 2015; Hatlevik et al., 2018). Thus, the present study extended previous findings by focusing on important determinants (prior achievement, intelligence) of student achievement in a longitudinal design and taking into account a relatively large number of ICT-related predictors simultaneously. In the following, we discuss the results in terms of our hypotheses.

7.1 ICT-related variables and intelligence prediciting ICT literacy cross-sectionally

In accordance with Hypothesis 1, ICT-related person characteristics incrementally predicted students` ICT literacy cross-sectionally after controlling for intelligence. As expected, intelligence was the most powerful predictor of ICT literacy, which is consistent with previous research findings (Aesaert et al., 2015; Senkbeil et al., 2013a; Senkbeil, 2018). The analysis of the relative weights showed that ICT-related motivations, i.e., ICT confidence and instrumental and social interaction use motives were also very powerful predictors. Taken together, the motivational constructs accounted for more than one-third of the explained variance (34.7%). Two aspects are noteworthy with regard to this finding. First, it emphasizes the importance of self-ratings of ICT skills (e.g., self-efficacy, confidence) in predicting ICT literacy and is also consistent with previous studies (Senkbeil & Ihme, 2017a,  b; Goldhammer et al., 2013; Hargittai, 2005; Hatlevik et al., 2018).

Second, it is important to note that ICT usage motives contributed more explained variance (22.8%) than ICT confidence (11.9%). This finding might be explained by the fact that instrumental and hedonic media orientations differ in their cognitive involvement (e.g., metacognitive strategies) so that they have a strong influence on the acquisition of ICT skills. Instrumental ICT activities (e.g., information seeking, preparing texts or presentations for school) often require higher-order thinking skills (e.g., metacognition, problem-solving skills) as well as adequate access behavior when searching information on the internet (e.g., carefully evaluating search results, integrating contents of accessed documents into a coherent mental model; Naumann, 2015). Thus, they enhance the acquisition of ICT literacy. Conversely, social interaction ICT activities (e.g., chatting online, posting images or videos in social networks) do not require extensive higher-order thinking skills or information-related knowledge and skills and are associated with lower knowledge of metacognitive strategies (Lee & Wu, 2013; Naumann, 2015). For example, Naumann (2015) found that while using social online media, “students do not regularly cognitively engage in demanding tasks involving the thorough evaluation of hyperlinks as to their potential relevance for the task at hand. Thus, using social online media most likely does not result in developing the.skills needed for task-adaptive navigation “ (p. 275). Moreover, online social media (e.g., using social networking sites or reading emails) are often used in parallel with another task such as searching for information for a school assignment (Kirschner & Karpinski, 2010). However, multitasking is bound to impair comprehension and learning (Sana et al., 2013), which in turn leads to poorer ICT literacy (see also Senkbeil & Ihme, 2017a; Senkbeil, 2018).

The differential associations between the ICT usage motives and ICT literacy underline that some ICT-related activities, in particular social interaction activities, can have a negative impact on the development of ICT literacy. On the other hand, in accordance with previous research findings, it can be assumed that experience with information-related tasks (e.g., information seeking for school or work, evaluating information, creating information products such as presentations) is an important prerequisite to acquire functional ICT knowledge and skills (e.g., Senkbeil, 2018; Lee & Wu, 2013).

7.2 ICT-related variables, intelligence, and prior achievement prediciting ICT literacy longitudinally

In accordance with the third hypothesis, the motivational constructs (ICT confidence, instrumental and social interaction ICT usage motives) predicted subsequent ICT literacy after controlling for prior achievement and intelligence. Furthermore, this also applied for breadth of ICT usage, ICT experience, and ICT usage at school. However, one has to keep in mind that the effects of ICT experience (β = 0.03) and ICT usage at school (β = 0.02) on ICT literacy at the second measurement occasion were very small as well as the relative weights, i.e., the percentages of predicted criterion variance (0.3% each). Furthermore, neither ICT access nor ICT usage at home predicted students' subsequent ICT literacy.

The small contributions of these variables (access, experience, usage at school and at home) can be explained by the fact that they do not contribute directly to further learning on the basis of already acquired knowledge, but rather represent important prerequisites in the sense of the necessary material resources and learning opportunities that enable a person to acquire basic skills, for example, when exploring digital devices and applications in childhood (Zaman & Mifsud, 2017; see also Sect. 7.3). The results therefore indicate that the contributions of ICT access, ICT use at home and at school and ICT experience do not, or only marginally, exceed the explanatory power of prior achievement. The relative weights analysis showed that prior achievement explained most of the variance in ICT literacy, followed by intelligence, which is in line with previous findings in other domains such as mathematics (e.g., Kriegbaum et al., 2015).

Since students' ICT literacy was highly stable over time and intelligence is a strong predictor of academic achievement, it was difficult for other variables to incrementally contribute to the prediction of ICT literacy. However, after controlling for both variables, ICT-related motivational constructs (confidence, usage motives) made significant and important contributions (17.5% of the explained variance). Thus, these findings suggest that motivation has an influence on the development of ICT literacy that is independent of prior achievement and intelligence.

7.3 Proximal and distal factors of educational productivity

The finding that the relative power of ICT-related motivational constructs is cross-sectionally (34.7%) as well as longitudinally (17.5%) much greater than the predictive power of the other ICT-related variables (6.0% and 3.0%, respectively) can be explained by Walberg’s (2006) model of educational productivity. This model describes so-called proximal and distal productivity factors that influence educational achievement. Proximal productivity factors that are closest to the learning experiences of students, such as students` cognitive abilities, prior achievement or motivation, exert more influence than distal variables (e.g., mass media exposure, use of ICT applications) that are rather removed from students’ learning experiences (e.g., Walberg, 2006). According to this classification, ICT confidence as well as instrumental and social interaction ICT usage motives can be regarded as proximal productivity factors whereas the other ICT-related variables (access, usage at home and at school, experience, breadth of usage) can be regarded as distal productivity factors because they do not necessarily contribute to students’ learning in a direct way. This conclusion is important not only for a better theoretical understanding of the interplay of proximal and distal productivity factors, but also for pedagogical practice in terms of which pedagogical tools are most effective in promoting ICT-related competencies. For example, promoting computer use at home should not be the first priority (Senkbeil & Wittwer, 2013c).

Furthermore, when examining ICT-related person characteristics as potential causes for a high level of ICT literacy, it is necessary to specify the psychological mechanisms by which ICT-related person characteristics foster the development of ICT literacy. Mentioning that, for example, access to digital media provides opportunities to develop ICT skills is not a fully satisfactory explanation because it does not sufficiently specify the psychological mechanisms for developing ICT skills over a longer period of time (see also Senkbeil & Wittwer, 2013c). As pointed out in the theoretical background, while psychological mechanisms could be given for most of the ICT variables taken into account, the most elaborated and convincing psychological mechanisms and explanations could be specified for the ICT-related motivations. In this respect, the reported results are in line with the level of elaboration of the theoretical explanations.

This argument applies in particular to the longitudinal analysis that examined the extent to which the change in ICT literacy was predicted by ICT-related constructs. As there is only little regular computer use in German schools (Drossel et al., 2017), young people have to acquire ICT skills and knowledge on their own and in a self-directed way (Zhong, 2011). Thus, it seems more than plausible that self-confidence and instrumental usage motives that are important determinants of self-regulation play a larger role for the development of ICT literacy from grades 9 to 12 than the other ICT-related variables. Although these variables (e.g., access, experience, usage at home) represent important prerequisites (e.g., necessary material resources, learning opportunities) for the acquisition of basic ICT skills (Aesaert et al., 2015; Fraillon et al., 2019), they only play a minor role for the process of life-long learning, i.e. the continuous development of ICT literacy on the basis of previous knowledge. The results underline the importance of motivations because, compared to intelligence and other psychological constructs (e.g., personality), they are more malleable via educational processes, i.e. characteristics of learning contexts. However, a necessary prerequisite for implementation is that information-related and strategic ICT knowledge and skills should be included as standard components of the educational curriculum (e.g., Drossel et al., 2017; van Deursen & van Diepen, 2013).

7.4 Limitations and further research

Limitations of this study might further highlight possible directions for future research and should be considered when interpreting the results. First, even though longitudinal research provides a better basis for inference about causality than do cross-sectional designs, our analyses were limited to two measurement time points. In order to draw causal conclusions about the influence of ICT-related variables, in particular motivational constructs on ICT literacy further research should have at least three measurement time points and cross-lagged analyses (Gustafsson, 2010). Moreover, our sample only included adolescents between 13 and 19 years of age. Although previous studies document the predictive power of ICT-related motivations on ICT literacy for different age groups and did not detect age differences (Senkbeil & Ihme, 2017a; Senkbeil, 2018; Christoph et al., 2015; Hatlevik et al., 2018), further research should test to what extent our findings can be generalized over a broader range of ages. Second, intelligence was assessed with a rather short test. This can lead to an underestimation of the predictive power of the construct. Future studies should include different facets of intelligence (e.g., fluid and crystallized intelligence; Moehring et al., 2016). Third, the results of the present study do not address the processes underlying the relationship between motivation and development in ICT literacy. Previous studies indicate that this relationship could be mediated by ICT engagement or the use of effective metacognitive strategies (Lee & Wu, 2013; Naumann, 2015). But additional empirical work is needed to examine how motivation leads to increased achievement (Putman et al., 2020). Fourth, in the present study, ICT literacy was used as an indicator for standardized test achievement. German NEPS also measures additional competencies, namely students' competence in mathematics, science and reading. Against the background of numerous cross-sectional studies that examine the impact of ICT-related variables (e.g., usage at home, attitude, self-efficacy) on students` achievement in mathematics, science or reading (e.g., Hu et al., 2018), future research should include these competencies in longitudinal studies to investigate the incremental power of ICT-related variables in predicting standardized test achievement in other domains.