Keywords

1 Introduction

The use of new technologies and the acquisition of technology-related competencies have been widely considered relevant or even indispensable for the participation in a modern knowledge and information society, regarding both social and professional aspects [1,2,3]. The recent developments have led to an emergence of new challenges for schools and school systems, causing “the need for students to develop new forms of relevant skills like digital literacy or computer and information literacy” to be constantly growing [4]. The necessity to meet these challenges sees tablet computers as providing a variety of opportunities as mobile learning devices. International studies based on an increasing dissemination of tablet computers around the world have found tablet computers to have a positive effect at various levels of student learning, the latter serving as the ultimate point of reference here [5,6,7,8,9,10]. A meta-analysis of 23 tablet-related studies [11] revealed that a majority of research reports positive effects of tablet use on student learning outcomes, however, others indicate that tablet use may equally have no effect or even negative consequences. The authors conclude that “the fragmented nature of the current knowledge base, and the scarcity of rigorous studies, make it difficult to draw firm conclusions”. In-depth investigations are advised (ibid.). While research on tablet computers for educational purposes is thus available, the frequency of use by school subjects as well as the relationship between the use of tablet computers and skills like digital literacy or computer and information literacy have so far not been a key research interest. This research gap is closed using data from the new, so-called TiGer project at a German upper secondary school, which in this respect provides current in-depth data from a unique control group design. The aim of the present contribution is to answer the question of how often students use tablet computers in class, what level of computer literacy they have and which role the tablet computers play with regards to the students’ computer literacy.

Theoretical allocation of this research and the relationship between the use of new technologies and outcome variables is described in many theoretical models (e.g. [12, 13]). One elaborated and current model concerning Computer and Information Literacy (CIL) as an outcome is the ICILS 2013 framework [14]. The use of new technologies is a factor at the process level involving school and classroom factors. The other construct, namely CIL is modelled as an outcome. Contrarily to the established model of school quality and school effectiveness, the relationship between the use of ICT for learning and CIL is correlative.

2 Current State of Research

The current state of research will be presented in three sections: First of all, the use of new technologies at school with a focus on tablet computers is presented. In a second step, the students’ computer literacy will be focused, before the current state of research regarding the relationship of computer use and CL will be addressed.

2.1 The Use of New Technologies at School

The fact that current research literature on mobile new technologies in education is still rather scarce reveals a first research gap that is addressed by the present contribution. In the following, new technologies in general will therefore be assessed with regards to the findings of previous research. It is particularly striking that in Germany, a traditional concept of media infrastructure using static computers in computer rooms is still common practice. The equipment infrastructure with regards to mobile devices, however, can be considered distinctly below average in comparison to Germany’s neighboring countries participating in the study [15]. According to ICILS 2013, tablet computers were available to 6.5% of Grade 8 students in Germany only (ibid.). The equipment with tablet computers at German schools is therefore significantly below the EU average of 15.9%.

The international comparison further showed big differences in the use of computers at school. While 81% of students in Australia report using the computer at least once a week, only 31 percent of students in Germany report doing so [15]. Taking a closer look at the individual main subjects (including the test language (Language Arts: test language), mathematics and foreign languages (Language Arts: foreign)), it becomes evident that only three to four percent of students in Germany report a regular, i.e. at least weekly computer use in class. By international comparison, Germany therefore comes in last [15]. Data on the frequency of tablet use in different school subjects is therefore currently not available for Germany with only a few exemplary studies and pilot projects examining and evaluating tablet use in general and giving recommendations (cf. [16,17,18]).

2.2 Students and Their Computer Literacy

Studies around the world have investigated ICT-related competencies of students using different methods. Hakkarainen et al. [19] in Finland, for instance, used a design that relies on the students’ self-reported competencies. Likewise, Osunwusi and Abifarin [20] base their comparative analyses on questionnaires administered to participants in Nigeria. For Chilean students, Claro et al. [21] used a performance-based assessment design – the ICTSfL test. The results merely show that the students’ skills related to the use of ICT as consumers exceeded the skills of becoming producers using ICT (ibid.). Alternative designs in the qualitative domain include interviews of Indian students as conducted by Sampath Kumar et al. [22]. Results comprise rather specific findings based on the answers given by the interviewees. While the afore-mentioned studies tend to have a closer look at the data gathered, other studies from Australia have computed an overall ICT Literacy Scale using achievement tests, from which proficiency levels regarding computer literacy could be deduced (cf. [23,24,25]). These can then serve the purpose of being included in further analyses. While the Australian studies mentioned above display a scope limited to the Australian states and territories, international comparisons of the CIL students have at their disposal as shown by the IEA study of ICILS 2013 help to gain further insight into the competence levels of German students. According to ICILS 2013, Grade 8 students in Germany reach an average performance of 523 scale pointsFootnote 1, similar to the EU average (525 scale points) but significantly above the OECD average (516 scale points) and the international average of 500 scale points [15].

2.3 The Relationship Between the Use of New Technologies and CIL

Meta-analyses [26, 27] suggest there is a positive relationship between ICT use and student achievement in different subject areas. While ICT has been examined with respect to its influence on subject-related achievement such as mathematics and science, the use of new technologies and its relationship with computer literacy has thus far been investigated rather marginally. Only very recently, Rohatgi et al. [28] were able to show for data gathered in Norway that the use of ICT has an indirect impact on the CIL achievement of students, the intermediate construct being ICT self-efficacy. Sung et al. [29] in their meta-analysis have further found mobile devices to be more effective than desktop computers or laptops when it comes to students’ learning performance. Focusing on direct effects again, the research interest of this contribution comprises both the teachers’ and the students’ use of the afore-mentioned technologies. As far as the teachers’ use of computers is concerned, the international comparison shows a positive effect on CIL in Germany only, contrarily to Australia, Norway and the Czech Republic [30]. Regarding the school-related use of computers by students, the international comparison revealed consistently positive relations. Warschauer [31], for instance, was able to demonstrate that students attending notebook schools (including primary and secondary schools) learn to access, process and use information much better while also acquiring skills to integrate this information into written and multimedia presentations. Likewise, an Australian study conducted by the National Assessment Program ICT Literacy pointed out that the school-related use of computers shows a positive correlation with the students’ ICT-literacy competencies [25]. The ICILS 2013 international comparison equally emphasized that the school-related computer use in most participating countries correlates positively with CIL. Only three countries, including Germany, showed a negative effect [32]. While some authors even seem to consider the literacy developed through the use of tablet computers to be a new, distinct literacy called tablet computer literacy (cf. [33]), the relationship between a school-related use of tablet computers and computer literacy has thus far not been addressed.

3 Research Questions

The following research questions can be deduced from the research desiderata described above:

  1. I.

    How often do students use tablets in school? Is there a difference between students in tablet classes and the control group?

  2. II.

    What level of computer literacy do students have at their disposal? Is there a difference between students in tablet classes and the control group?

  3. III.

    Which role does the use of tablets play in this context? Is there a relationship between the use of tablets and computer literacy?

  4. IV.

    To what extent do teachers experience a development regarding the tablet class students’ computer literacy?

4 Methodology

4.1 Sample

In order to answer the research questions above, the present contribution uses the data provided by the TiGer project (Tablets im Gymnasium evaluieren und reflektieren). This project used scholarly guidance in the process of introducing tablet computers to Grade 7 students at an upper secondary school (Gymnasium) in the German federal state of North-Rhine Westphalia. From the beginning of the school year 2014/2015, tablet computers were used in Grade 7 classes. These mobile devices are owned by the students who could therefore also use them outside school. Two out of five parallel classes were fully equipped with tablet computers, while the remaining three classes continued without tablets, hence serving as control groups. The concomitant research involves an elaborate triangulation design of multiple perspectives. The focus of this contribution is placed on a quantitative student survey, gathering data both at the beginning of the school year when the students did not yet possess any tablet computers (Time of Measurement 1; ToM 1) and at the end of the 2014/15 school year, with tablet class students having worked with tablet computers while students from the control group have not (Time of Measurement 2; ToM 2). This quantitative data is subsequently complemented by qualitative teacher data, which was gathered in the form of open questionnaires.

4.2 Instruments and Methods

Composition of Longitudinal Section.

In order to model the longitudinal section, all students who only participated at one time of measurement were excluded from the analyses. The original 119 participants were hence reduced to 105 students that constitute the sample. 43 of these were taught in tablet classes, whereas the remaining 62 attended parallel classes in which no tablet computers were used and that can hence serve as a control group.

Tablet use.

The use of tablets was assessed in a subject-specific approach for all school subjects. In the following, the frequency of tablet use is reported for the main subjects of German, Mathematics and English. The five-tier response format ranged from never (1) to every day (5). At that, the use of tablet computers is regarded both subject-specifically and as a latent construct used for further analyses. The exploratory factor analysis of the subject-specific tablet use reveals that the three selected items show factor loadings on one factor with a very good reliability for both groups of students at both times of measurement.

Computer Literacy.

For the purpose of assessing the students‘computer literacy, the National Educational Panel Study’s (NEPS) computer literacy test for Grade 9 was used [34]. Due to the fact that a few tasks of the Grade 9 NEPS test were no longer up to date at the time of measurement – which can be attributed to the fast-paced development of new technologies – these tasks were replaced by corresponding items from the Grade 6 NEPS test. Example items can be found in Senkbeil et al. [35]. The paper-pencil test consisting of 36 tasks (including 30 simple multiple choice and 6 complex multiple choice items) depicts realistic problems embedded in a range of authentic situations using screenshots.

In order to estimate item and person parameters for computer literacy, a partial credit model was used and estimated in ConQuest [36]. In a first step, ability estimates were estimated as weighted maximum likelihood estimates (WLEs, [37]) with a metric of 500 and a SE of 100. As the same computer literacy tests were used at both times of measurement, the item parameters of ToM 1 are implemented to estimate the item difficulty of ToM 2. The results show for both times of measurement that the items exhibited good item fit. Moreover, the test showed a high reliability (.932/.865) and the different comprehension requirements foster a unidimensional construct. The correlation of the two WLEs amounts to .685 for the tablet classes and .678 for the control group. This suggests that the computer literacy construct remains relatively stable over time. In addition to measuring the students’ competencies, open, qualitative questionnaires were handed out to the teachers teaching the tablet classes (N = 6). This complementary instrument results in a multi-perspective, triangulative design of the study, which is expected to contribute to the second research question of this paper.

5 Findings

5.1 Tablet Use

At the beginning of the school year (ToM 1), neither the students in tablet classes nor the students of the control group used tablet computers in their main subjects. The two groups therefore do not show significant differences (Table 1). It can hence be established that the starting situation is the same for all students. At ToM 2 – after one year – the results show that tablet class students use tablet computers at least once a week but not every day. No significant differences between the main subjects could be observed. According to expectation, the control group does not show any changes with respect to the first time of measurement – the use of tablet computers can still be considered an exception here. The differences between students from tablet classes and students from the control group were all found to be significant.

Table 1. Students’ tablet use differentiated by tablet class and control group at ToM 1 and ToM 2
Full size table

5.2 Computer Literacy

The computer literacy that students have at their disposal and the extent to which students from the tablet classes and from the control group show differences in their performance will be subsequently discussed. The average performance of the students taught in tablet classes before obtaining the devices is at 472 scale points, thereby displaying significantly lower scores than their fellow students from the control group (520 scale points) (Table 2). This result remains in force even after the introduction of tablet computers and repeated computer literacy measurement procedures after the school year. Both the control group and the tablet students attain – on average – the same competencies as they did at ToM 1, resulting in the fact that tablet students show a significantly lower average level of achievement regarding computer literacy yet again.

Table 2. Students’ performance indexes (WLE) in computer literacy differentiated by tablet classes and control group at ToM 1 and ToM 2.
Full size table

5.3 Relationship Between Computer Use and Computer Literacy – Students Achievement

As explained by means of the theoretical framework model, a correlation between the use of tablet computers and computer literacy can be assumed (Table 3). At ToM 1, this correlation is not significant for the tablet classes (at .207 with p = .225), but proves to be significant for the control group (at .364 with p = .007). The second time of measurement shows a substantially smaller correlation that is not significant in either case (−.011 for tablet classes; .183 for the control group).

Table 3. Correlations between the use of tablet computers and computer literacy differentiated by tablet classes and control group at ToM 1 and ToM 2
Full size table

5.4 Relationship Between Computer Use and Computer Literacy – Teacher’s Perspective

In contrary to the findings from this performance test, the teachers’ impressions as reflected by the qualitative data gathered from an open questionnaire do point to an improvement in the students’ CIL. One teacher, for instance, found that “the students use the tablet more like a tool while the ‘play instinct’ decreases constantly. The exhausting of all of the tablets’ functions is impressive. Information research, the preparing of presentations, drafting of e-mails and saving documents in clouds has become a routine for most [students]. The targeted use of apps also becomes more and more reflected.” (Mrs Schmidt, 35 years old, teacher of Arts and Spanish). This reflects the experience of all six teachers, especially with regards to the more confident and reflected use of the new technologies. Individual teachers, however, also point to “alarming” cases, where “unconfident girls” are still showing difficulties in using basic applications after 1.5 years of using the tablet computers.

6 Discussion and Outlook

First of all, the analyses have shown that according to expectation, the tablets were used on a regular basis in the main subjects after their introduction. At that, students in tablet classes use tablet computers on an at least weekly basis, while the frequency of tablet use in the parallel classes without tablets did not change over time and still corresponds essentially to the category of “never”. Hence, it is not surprising that students in tablet classes use tablets in class significantly more often than the students in the control group. This result primarily emphasizes the importance of material resources available to students that constitutes a prerequisite for the implementation of new technologies.

With regards to the second research question, it can be stated that the average computer literacy – as tested with the TILT test – was substantially lower among tablet students than among students from the control group both prior to and after the introduction of tablet computers. While the tablet students’ level of CIL amounts to 472 scale points (i.e. below average), the control group’s CIL is above-average (520 scale points). It further becomes evident that no improvement could be detected in either group as far as the students’ performance is concerned. These findings were not to be expected, however, they do appear rather plausible when taking a closer look at the instrument with which computer literacy was tested – and its theoretical conception. In contrary to the findings from this performance test, the teachers’ impressions as reflected by the qualitative data gathered from an open questionnaire do point to an improvement in the students’ CIL. According to preliminary findings (teacher, student, and principal perceptions), the software applications used there did not correspond to those that were put to use by the tablet classes. For instance, students did not send e-mails with their tablets but communicated via specially-designed communication applications. Further differences were found with respect to the process components of the theoretical framework model of computer literacy. The way in which tablet students manage information, for example, is completely different from the way it is operationalized in the TILT test. The students used the tablets’ cameras to digitalize and save documents – a competency that is not assessed by the instrument. It is, however, worth noting that for the German-speaking world, there is currently no valid and openly accessible instrument available to account for these diverse potential applications that are unique to tablet computers. With the development in this area being rather fast-paced – both with respect to software applications and hardware – the designing of a suitable and valid instrument does constitute a big challenge. A further possible explanation for the persisting difference in CIL could lie in the fact that students from the control group acquired their skills outside of the classroom. This remains to be verified and could constitute the focus of further analyses.

A third result of this contribution reveals that the assumed correlation between the use of tablet computers and computer literacy as depicted by the theoretical framework model [14] cannot be maintained. The same is true if these results are compared to multiple research findings that have asserted a direct relationship between the two constructs [19,20,21,22]. At first glance, this result therefore seems unexpected, while it does support the assumption that the TILT instrument could not measure the competencies as they were acquired by the students through the use of tablet computers.

From the teacher interviews (question 4), it becomes apparent that there are indeed positive effects that go along with the use of tablet computers (such as an increase in motivation, an improvement of organization and communication as well as a compensation for cancelled lessons). These, however, do not constitute the focus of this paper.

While the data of the study are not representative neither for Germany nor for the focused school form, it seems to be helpful to implement larger studies (e.g. large-scale assessments) in order to draw representative conclusions about the use of tablets.