Introduction

The use of technology in mathematics education has been highlighted by researchers over the past two decades. In the US, the National Council of Teachers of Mathematics (2000) considered technology as “essential in teaching and learning mathematics” (p. 3). Scotland’s Curriculum for Excellence (Education Scotland n.d., p. 40) noted that “use of technology in appropriate and effective ways” allows for learning experiences that promote enjoyment of mathematics. In a survey of mathematics research in the United States over the past 30 years, Cheung and Slavin (2013) found that technology produced a positive effect on students’ achievement in comparison to traditional methods. However, they also indicated that effects varied by the type of educational technology used.

Mobile technologies have been gaining wider acceptance in education in recent years. School and government level initiatives have rolled out these technologies in the classroom (West 2012). Potential benefits of using mobile technologies for learning include: facilitating learning across contexts, facilitating contextual learning, and providing personalisation in both personal and collaborative environments (Cochrane 2010). These potentials make mobile technology seem an ideal tool for learning mathematics.

Literature on mobile learning and mathematics

Mobile technologies have provided various approaches to learning: engaging learners in contextualised learning environments using mobile devices’ built-in sensors (Tangney et al. 2010); using the mobile phone to journal math learning (Project Tomorrow 2011); and connecting learners through mobile phones and social media (Roberts and Butcher 2009). Crompton and Burke’s (2015) survey of mobile learning in mathematics showed that there is a growing interest in mobile technology effectiveness, with 75% of 48 studies reporting positive learning outcomes. Similarly, in Fabian et al. (2016) review of mobile learning studies in mathematics, 77% of 31 studies reported that mobile technologies improved students’ achievement. But while there are more reports of successful mobile learning studies in mathematics, there are also studies that found contrary results. For example, Miller and Robertson’s (2011) randomised controlled trial of mobile game-based learning found no significant difference between students who played mobile games and those who don’t. Carr (2012) who also did a random assignment between the experimental and control group also found no significant difference in the test performance between groups. Roberts and Vänskä (2011), who implemented a mobile tutorial service over 24 weeks found that students had a decline in their test scores over the course of the intervention. Similarly, Perry and Steck’s (2015) quasi-experimental study also found that students who used the iPads as virtual manipulatives had a decline in their test scores by twice as much as those who were taught the traditional method. This shows that the use of mobile technologies does not always result to positive outcomes. While overall, mobile learning studies in mathematics have shown a positive effect (Sung et al. 2016), the nature of the intervention and the study design affects the success or failure of mobile learning interventions.

Pollara and Broussard (2011) noted that the majority of studies on mobile learning reported positive student perceptions of mobile use in the classroom. Mobile learning studies on mathematics yielded the same results. Students found the use of mobile technologies engaging and useful (Baya’a and Daher 2009; Lai et al. 2012). Baya’a and Daher reported that students saw mobile technologies as useful mathematics tools because they facilitated visualisation, encouraged collaborative learning and enabled exploration of mathematics in an outdoor environment. While student perception towards mobile technology use is positive, studies that report on how these positive attitudes are affecting their attitudes to mathematics is limited and sometimes contrasting results. However, like the studies noted above, many of the current studies on mobile learning focus were over short periods, the majority less than a month long (Sung et al. 2016), therefore, how students perceive the use of mobile without regard to novelty value is not clear.

Student attitude, achievement and the relationship between the two is a well-researched area of mathematics (Zan et al. 2006), but literature on effect of technology use on attitudes towards mathematics is limited, with few quantitative studies (Li and Ma 2010). This is also the case for math and mobile learning studies. A literature search on math and mobile learning studies between 2003 to 2015 yielded only seven quantitative studies—the results and the programme length of these studies varied. Three studies found a positive change in student self-concept and attitudes to maths (Main and O’Rourke 2011; Riconscente 2013; Wu et al. 2006) and three studies found no significant improvement in students’ attitudes (Jaciw et al. 2012; Miller and Robertson 2010, 2011). Bray and Tangney (2016) found different results for various scales of maths attitudes: an improvement in students’ affective engagement and attitudes towards technology, but no significant difference in students’ behavioural engagement, mathematical confidence and confidence with technology. The length of implementation of mobile learning activities varied for the seven studies, ranging from 1 day to 1 year. The limited and differentiated results of effects in attitudes to mathematics studies is a gap in mobile learning literature and one that this current study addresses.

Design framework

Mobile learning, being a relatively new field, is short on theory in the same way that elearning theories had been sparse during the first decade of its introduction into schools. Mayes and De Freitas (2004) noted that “there are really no models of e-learning per se—only e-enhancements of models of learning (p. 4)” and this is currently the same for mobile learning theories. Some of the theories underpinning mobile learning studies include: behaviourism, cognitivism, constructivism, situated learning, collaborative learning, socio-cultural theory and many others (Keskin and Metcalf 2011). These learning theories are not exclusive to mobile learning but have been discussed in terms of how these theories were applied or adapted in light of the mobility of the devices and mobility of the learner.

Constructivist learning is one of the typical approaches adapted in math studies that employ technology use (Li and Ma 2010). Its application to mobile learning literature is just as prominent as it is in maths learning literature. Mobile technologies support constructivist learning through active learning activities (Wijers et al. 2010), immersion in authentic environments (Sommerauer and Müller 2014), and learner-generated context (Bray et al. 2013). Moreover, mobile devices are “inherently social collaboration and communication devices that provide powerful tools for enabling social constructivist pedagogy (Cochrane 2014, p. 72)”. Effective technology integration requires consideration of the technology and underpinning pedagogy. Drijvers (2012) pointed out that pedagogical design, the role of the teacher, and educational context are crucial elements in integrating technology for mathematics.

One of the noted advantages of mobile technology over traditional computing is its capacity to support learners in a variety of contexts (Tangney et al. 2010). Through the ubiquitous learning environments that mobile technologies support, learners are afforded the chance to learn in situ (Baya’a and Daher 2009). In these learning environments, students have found mobile devices helpful in terms of facilitating visualisation of maths concepts. Most of these studies, however, have been exploratory and qualitative in design. Studies that provide evidence in terms of improvement in student performance are few (Wu et al. 2006; Hwang et al. 2015). The current study, in addressing this gap, aims to provide some quantitative evidence on the effects of using mobile technologies in ubiquitous learning environments.

Research questions

Given the limited literature that explores the effects of using mobile technologies in mathematics, this research aims to investigate the effects of using mobile technologies on students’ attitude and achievement. Specifically, this study aims to answer the following research questions:

  1. (a)

    What are the students’ views on the use of mobile technology for learning mathematics?

  2. (b)

    Is there a change in attitude towards mathematics when mobile technology is used for learning maths?

  3. (c)

    Is there a change in attitudes concerning technology when mobile technology is used for learning mathematics?

  4. (d)

    Is there a change in students’ performance in a mathematics test when using mobile-supported math learning activities?

Methodology

Research design and nature of the intervention

The study adopted a quasi-experimental mixed methods design with the experimental group working on mobile learning activities and the control group following their normal curriculum. Evaluation was carried out at three levels: Micro, Meso and Macro-evaluation (M3) using Vavoula and Sharples (2011) evaluation framework. The M3 evaluation framework provides a “structured format to assess usability, educational and organisational impact and their inter-relationships (p. 12).” Table 1 outlines the research design following the M3 level framework.

Table 1 M3 Level evaluation framework

Students participated in mobile-supported, collaborative learning activities carried out in indoor and outdoor learning environments. Topics covered include geometry and information handling. An overview of the mobile learning activities is listed in Table 2. For comparison, activities used to teach these topics, as identified from Scottish Primary 6/7 textbooks, are also listed in Table 2. Lesson plans for the mobile learning activities were provided to teachers before the start of the intervention. The rationale for the design of the activities carried out in this study was to allow students to connect mathematics concepts to their environment, taking advantage of the different features of mobile devices like camera, built-in sensors and network connectivity. For example, in one of the outdoor activities, students worked in pairs to take pictures of symmetrical objects in their environment. An application called SnapShot Bingo provided them with a list of the specific symmetrical conditions that they had to find and organised the images that they captured. Using the networked capability of the tablets, all works were saved in a shared workspace to allow them to access others’ work and discuss in bigger groups.

Table 2 List of activities carried out

Participants

The participants were obtained by soliciting teachers from within one local authority in Scotland. Two teachers from the same primary school agreed to participate in the research. One of these teachers adopted the use of mobile technologies in his class (experimental group) while the other teacher taught her math classes as usual (control group). The teachers decided upon themselves about the group assignment. A total of 52 primary 6 and 7 students aged between 9 and 11 years old participated in the study, with 24 students in the experimental group (11 boys and 13 girls) and 28 students in the control group (14 boys and 14 girls). The experimental group is an inclusive composite Primary 6/7 class with two students identified as having additional support needs. The control group students are all in Primary 7. The school, as described by an Education Scotland report, had students receiving free school meals around 40% less than the national average and pupils’ absences roughly 10% below the national average.

Instruments and measures

Technology used

Mobile devices used in the study were 7–8 inch Android 4.2 tablets costing less than £100 each. The rationale for the choice was the low cost and the higher portability of smaller tablets over bigger 10-inch screens. Several activities were carried out while students moved around. The small form factor made the tablets easier to hold while the students were mobile, but big enough to allow two users to conveniently share a screen. These devices were provided to the experimental group for the duration of the research project.

Maths attitude inventory (MAI)

The maths attitude inventory used was adapted from two maths inventories: Lim and Chapman’s (2013) Attitudes Towards Mathematics Inventory and that of Pierce et al. (2007). Three subscales were adopted from Lim and Chapman with the following reported Cronbach alpha values for reliability: enjoyment of mathematics (0.90), self-confidence (0.94) value of mathematics (0.91). Two subscales from Pierce et al. were added to measure student attitudes to mathematics in relation to technology, with the following reported reliability scores: confidence with technology (0.79) and learning mathematics with mobile technology (0.89). As the instruments were used on a different population group, the reliability scores were recomputed and were as follows: enjoyment of mathematics (0.83), self-confidence (0.83), value of mathematics (0.61), confidence with technology (0.68) and learning mathematics with mobile technology (0.61). Admittedly, some of these new reliability scores are lower than acceptable and this is considered in the data analysis and discussion.

Maths test (MT)

The math test has a total score of 40. The questions comprised of topics discussed in the intervention: symmetry, angles, area and perimeter and information handling. These items were from practice exercises in Primary 6 and 7 mathematics textbooks used in Scotland (Heinemann Maths and TeeJay CfE Maths).

End activity evaluation

The End Activity Evaluation was adapted from two established usability questionnaires. Eighteen adjectives from the Microsoft Desirability Toolkit (Benedek and Miner 2002) were arranged on a semantic differential scale five units apart with two opposite adjectives at each end. This resulted in nine adjective pairs, but to allow students to rate the tablet and the activity separately, these pairs were repeated. Additionally two questions from the Lewis (1991) After Scenario Questionnaire were added to the instrument. The resulting questionnaire thus consisted of 20 questions with three factors of usability: usefulness, ease of use and user satisfaction (Lund 2001).

Interviews

Group interviews were designed to elicit student feedback about the activities which might have been missed in the end activity survey. Students reflected upon the activities that they had completed and were asked to explain which of the activities they liked and disliked. Their opinion on the advantages and disadvantages of doing these types of activities were sought. Students also related the challenges they had experienced with the activities. Discussions were audio recorded and transcribed.

A semi-structured teacher interview was also conducted. Questions asked include the teacher’s view on the mobile learning activities, observations on how the activities affected the students and perceived advantages and disadvantages of mobile learning. The interview was audio recorded and transcribed.

Procedure

A pre-test of MT and MAI was completed by the experimental and control group at the start of the intervention. Following the tests, an introductory session with the experimental group was conducted to brief the participants about the nature of the activities to be carried out. The experimental group participated in 8 h-long sessions of mobile learning activities spread over a period of three months. A variety of mobile-supported collaborative learning activities were carried out within and outside the classroom as listed in Table 2. Students completed an End Activity Evaluation questionnaire at the end of an activity. Midway through the programme, the experimental group completed the MAI test again and afterwards went on a two-week spring break. After the break, they continued with the intervention for four more sessions. At the end of the programme, both groups took the MT and MAI which is identical to the one carried out at pre-test. An interview with the teacher and student participants of the experimental group was also carried out at the end.

Data analysis

The ratings on the end activity evaluation were grouped into three categories of usability: usefulness, ease of use and user satisfaction. These were further grouped into tablet and activity ratings. Each subscale score was the average of the item ratings for that category, thus giving a range score between 0 and 5. A high subscale score indicated a good usability rating and vice versa. The tablets’ usability was analysed using descriptive statistics. Gender difference in student ratings was analysed using independent t-test of the usability scores.

Student perceptions about the learning experience were analysed from the group interview. Student responses to questions raised in the group interviews were analysed into themes: (1) student perception of the tablet activities, (2) advantages/disadvantages of using the tablets, (3) opinions on group/paired work, and (4) issues and challenges in using tablet devices. Teacher perceptions about the mobile learning activity were used to give further information.

An independent t-test of the MAI, MT and gains score was used to compare the experimental and control groups before and after the intervention. Paired t-test for the MAI scores was conducted to detect change in MAI scores over time. Gender difference in gains in MT score of the tablet and control group was also examined using independent t-test of the gains score.

Results

This section is divided into the three evaluations carried out in this study: (1) micro-evaluation that corresponds to the tablets’ usability test; (2) meso-evaluation which represents the student evaluation of the learning activities; (3) macro-evaluation which covers the quantitatively measured effects of using the tablets on students’ attitudes and achievement

Micro-evaluation

A graph of the semantic differential ratings for the evaluated sessions is shown in Fig. 1. A higher score means agreement with the positive statement while a lower score means otherwise. For example, in the activity with areas and perimeters, an average rating of 4.0 on the item irrelevant vs useful meant that students found the activity more useful rather than irrelevant. From the graph, it can be observed that most of the activity ratings fall within the positive ratings with a range between 2.6 and 4.8. This shows that majority of students evaluated the activities positively. From the graph, location does not appear to be a factor in the student evaluation as the angles session carried out outdoors had higher rating than the one carried out indoor but the information handling activity carried out indoors had higher score than the one carried out outside the classroom.

Fig. 1
figure 1

End activity evaluations

The end activity ratings in Fig. 1 were grouped into three subscales of usability: usefulness, user satisfaction and ease of use; and further grouped into tablet ratings and activity ratings. The tablet rating and the activity rating for each subscale all had significant positive correlations. The activity ratings were divided into male and female ratings as shown in Table 3. An independent t-test of the ratings categorised by gender showed no significant gender difference in all categories.

Table 3 Summary of activity ratings grouped by gender

Meso-evaluation

Student perception of the tablet activities

The whole experimental group participated in the group interviews, except for two students who were not available for interview at the time. Students were not required to answer all the questions and they could opt not to participate. Of that sample, 80% gave positive feedback, 15% gave mixed feedback and just 5% gave negative feedback. Students found the use of the tablets fun (n = 11), made learning easier (n = 5), were useful (n = 3) and better than their usual maths lesson (n = 3).

I found that I understood it more. When the teacher describes something to you, I don’t really get it, but with the tablets I understood it more and it was really fun. (Female student)

Students with mixed feedback (n = 3) found the activities fun but the technical problems encountered lessened the positive experience they had. For example, a female student explained that “it was good, and it was fun but sometimes the app didn’t work and then you got annoyed.”

Students who scored low on the maths pre-test found the activities fun (n = 2) and better than their usual maths lesson (n = 2). Similarly, students who scored high on the maths pre-test found the activities fun (n = 4), more engaging (n = 3), helpful, easier and better than their usual maths (n = 7).

It makes you look forward to maths like I know that every 11 o’clock every Thursday I was gonna get good maths. No, not good maths… I mean more fun maths… better maths. (Male student)

Perceived advantages/disadvantages of doing tablet activities

Students were asked what they thought were the advantages of the tablet and most of the responses were in relation to how they would normally do mathematics without these tools. Students felt that it was better than their usual maths lesson because it was more fun (n = 4), engaging (n = 5), made learning mathematics easier (n = 2), and more active (n = 2), with the activities allowing them to move about and engage with their environment. Students also thought that it was a good opportunity to be able to use technology while learning maths (n = 4).

It was really fun and also when I use the tablets I understood it more when I kept using it but when I just write it on a jotter, I still don’t know what’s happening. (Female student)

When asked for disadvantages, only two answers were given: one student highlighted that the main disadvantage of using the tablets was the cost involved, while another student replied that it was more challenging because they had to remember more.

Working in pairs and in groups

Overall response to the group work conducted was positive (n = 15). Students felt that being able to work in pairs made the task easier. A female student, for example, explained, “I find it easier to work in pairs so you could discuss it with your partner.” There were, however, cases where some students considered that working in pairs using the tablet was not always good because it led to some discussions over control of the device (n = 7).

Sometimes people can do all of it and not give the other person a shot and it could also be the opposite way, if you really like it, you’d be bad at not giving the other person a shot. Sometimes when you’re with someone who doesn’t understand then you’d want to do it all and if you’re with someone that does it all then it annoys you as well. (Female student)

Several students (n = 6) who scored low in confidence with technology made more reference to the negative aspects of doing collaborative work with mobile technology than those who scored high in this subscale. While students understand the value of collaborative work, there were cases where students who were more competent with technology took over control of the device.

Sometimes my partner takes over and I had nothing to do and sometimes I wanted to have a go and she’s like sometimes, just wait a minute but that’s the disadvantages with working with a partner but the advantages they know what they’re doing and you don’t and sometimes you know what you’re doing and they don’t but they’re not letting you have a go at it. (Female student)

Students who had low self-confidence scores at pre-test found working in pairs advantageous in making the mobile learning activity easier.

It was much easier because if you had ideas to reflect off your partner… if you get stuck you’re not sitting there like (not knowing what to do) and you’ll be more like… how do you do this. (Female student)

Students with high self-confidence scores at pre-test found working in groups better than working on their own (n = 9). There was, however, negative feedback (n = 5) about group work from students who had higher self-confidence scores. Some students said that their partners could get in the way at times.

Challenges encountered

Several challenges were encountered during the intervention including internet, software, and battery issues. The tablets were connected to a mi-fi 4G device rather than the school’s network but this type of connectivity is not very stable and so sometimes caused tablets to be disconnected from the network. One student commented: “a couple of the tablets won’t link. I can’t send it to the teacher. I’m sitting next to him and I can’t send it.” Students reflected that these technology issues were the downside of doing the tablet activities. They explained that sometimes the applications on the tablet would close for no apparent reason. Students dealt with this issue by relaunching the application and starting over, which solved the issue most of the time, but sometimes the instability of the application left students frustrated as they lost work that they did for the session.

Teacher interview

A semi-structured teacher interview was conducted at the end of the session. Like the students, the teacher was also positive about the intervention. The teacher describes the mobile learning sessions as fun and was able to set a reference on how to use the school’s mobile devices not just for maths but for other subjects as well. The teacher confirms that the students were positive about the mobile learning session and would look forward to these sessions. He notes that the students already know how to use the tablets “so it’s really just a case of pointing them to the tools that help them learn and ensure that they are interesting and in a collaborative way.” From the teacher’s perspective, the benefits of using the tablets included: engagement with learning and practice with transferable technology skills. For example, he notes that at the start of the program, one of the girls wouldn’t engage with technology but through the intervention, this student “had been more willing to have a go with technology” which is something that the student herself has confirmed during the student interviews. The teacher also observed that students were able to see how they could use the applications used in the maths intervention into other areas of the curriculum.

My class has already been down to Primary 1 and was showing how they can use mobile technology to support French and the Primary 3 class helped my class to set up all the auras [tagged information using the application Aurasmas] for the parents night. It’s kind of putting the whole school on board.

In terms of the tablet’s usefulness to learning maths, the teacher notes that there’s the element of ownership of learning.

when they were doing the task on angle, it’s one that [they] found, [one] that they considered following certain criteria for being a certain angle, and one that they found rather than, you know, the textbook identified… here they were able to find their own one.

He adds that assessment also had a big role in the activities as he could see straight away what the students are doing and provide guidance where necessary.

The sessions had its own set of technical glitches but this is something that the teacher has anticipated. For example, the sharing facility of the tablets using Dropbox did not initially work but the teacher opted to connect the tablets to a bigger display manually (via wired connectors) to facilitate sharing. This was an alternative solution which the teacher was able to implement straight away and also shows the flexibility of the teacher in using technology. He notes: “you expect certain things to not run smoothly, then you troubleshoot when it happens… it’s not one that actually bothers me that much because the advantages outweigh the disadvantages.” This statement also shows the positive outlook of the teacher in terms of handling technology issues.

There is also the need to adapt technology use rather than take it as an out of the box experience. He notes that “there is a need to identify right at the very start to identify how are these technologies going to enrich the lesson.” This was a step he took at the start of the research project. For example, there was no differentiation offered in the lesson plans provided for the intervention and all students were expected to do the same thing. These lessons had to be tweaked a bit to accommodate the differences in class. The activities provided to the teacher became the core learning session but the teacher implemented it in such a way that challenges both the advanced students and those that struggle a bit.

Macro-evaluation

Difference between groups in MT

A t-test of the gain scores between groups showed a significant difference, t(43) = 4.57, p = 0.005, ES = 0.89, in favour of the experimental group. Figure 2 shows a graph of the gain scores of the experimental and control group.

Fig. 2
figure 2

Gain scores of experimental and control group

Gender differences within the experimental group was also examined. There was no significant difference in the scores of male and female students in the experimental group before and after the intervention, t(22) = 0.118, p = 0.907, d = − 0.05 and t(22) = − 0.874, p = 0.391, d = − 0.36. There was no significant difference in the gain scores, t(22) = − 1.567, d = − 0.64 but there was a medium effect size in favour of female students. An independent t-test of the gains in maths test score between male students in the tablet and male students in the control group found no significant difference, t(23) = 1.441, p = 0.163, d = 0.58 but found a significant difference between female students of the experimental group and the girls in the control group, t(25) = 3.098, p = 0.005, d = 1.19. A graph of these differences in gain scores by gender and grouping is shown in Fig. 3.

Fig. 3
figure 3

Gain scores grouped by gender

Mathematics attitude inventory

A paired t-test to compare the groups’ scores at pre-test and post-test was conducted followed by an independent t-test to compare the gain scores on the MAI inventory. The descriptive statistics for Mathematics Attitudes Inventory (MAI) and its subscales are shown in Table 4. The paired sample t-test for pre-test and post-test scores also showed no significant difference in all subscales except for enjoyment, where there was a significant decrease in score for control group, t(24) = − 2.680, p = 0.013, ES = − 0.55. An independent t-test of the MAI gains between the experimental and control group resulted in no significant difference in all subscales of the MAI (see Table 4). There was, however, a small effect in the subscale self-confidence (ES = 0.21). An independent t-test of the male and female students score at varying time points per group indicated that there is no significant difference in all the five subscales being investigated except for the experimental group’s VMT pre-test score where boys scored significantly higher than girls, t(22) = 3.130, p = 0.005, d = 1.28. This difference, however, did not manifest in the subsequent tests.

Table 4 MAI scores of experimental and control group

Discussion

Interpretation of findings

Here we discuss the responses to the research questions using the results from the micro, meso and macro-evaluations and interpret these findings in relation to existing literature.

Students’ views on the use of mobile technology

In general, students had a positive view about the use of mobile technologies and they found the learning activities fun, engaging and useful. This positive result in students’ perception of mobile technologies is consistent with other mobile learning studies (Baya’a and Daher 2009; Lai et al. 2012). Previous studies that report student perspectives on mobile use do not separate evaluation of the activity from tablet usability. For example, in Lai et al. (2012), the students found the use of the tablets easy and fun but it was unclear whether students were also referring to the learning strategy embedded in the technology use. While there was an attempt to separate the ratings between technology and activity, the positive correlation between the two suggests that students might not have objectively evaluated the two separately. For example, students mentioned difficulties with the mobile device in the interviews but a triangulation of these responses with the end activity evaluation shows that students still rated the both the tablet use and the activity positively. As with other mobile learning studies (Baya’a and Daher 2009; Lai et al. 2012), novelty was a contributing factor to student satisfaction. Students contrasted the mobile learning activities with what they usually did and although challenging to some students, student satisfaction with the activities remained positive.

Student attitudes to mathematics and mobile technology

There was a slight positive change in students’ enjoyment, self-confidence and value of mathematics a month after the intervention has started. Wu et al. (2006), utilising a day-long session, had the same findings. These gains, however, had reversed in direction by the end of the intervention. One possible reason is that the increase in scores during the early days of the intervention was mainly due to novelty effects. As students progressed through the intervention, and having taken the same instrument third time in 3 months, they might have become more reflective in their attitudes towards mathematics. It is also possible that some of the novelty wore off.

Students described the tablet activities as fun and engaging but this did not result to a significant change in their EN scores. A comparison of the individual interview data with EN scores showed that the interview results corroborated the individual EN findings with some students but not all. For instance, several students explained that they found the activities fun and this was evidenced by the gains in EN score. However, there were cases where the students explained that it was more fun and interesting but this did not translate into gains in EN scores. Admittedly, this result is not entirely encouraging, but this can be interpreted in a positive light given that the control group had a significant decline in their enjoyment of mathematics.

Evidence of self-confidence is apparent in some of the student narratives as they explained how the use of the tablets helped them understand abstract concepts, however, the test scores did not show a significant difference for both control and experimental group. The lack of statistically significant improvement in attitudes was probably due to the program frequency and duration. The weekly mobile learning sessions over 3 months might not have been enough to yield a permanent change. As with Miller and Robertson’s (2010) findings, 3 months of mobile learning did not result to a change in attitudes towards mathematics. Glimpses of negative attitudes towards mathematics are noted in the interviews as students repeatedly compared the tablet activities with their usual maths class, often quoting their usual maths session as boring and the tablets better. It could be that while students found the once a week activity with the tablet interesting, this did not outweigh their daily experience with maths.

There was a slight increase in students’ confidence with technology and value of mobile technology but these increases were not statistically significant. There is very limited study that explores this area (Li and Ma 2010). A more recent study by Bray and Tangney (2016) considered this change and found no significant difference in students’ confidence with technology, but found a significant improvement in attitudes towards technology. The timings and implementation, however, were different in Bray and Tangney’s three interventions (one with 2 h a day over a week, another with 6 h over 2 days and the third a 2-h afternoon session) which may explain why the same gains were not shown in this study. While the quantitative data showed no significant improvement in student scores, the teacher testimony that students had been using the technology skills that they had learned in other subjects is a positive outcome.

Mathematics achievement

Students explained that the activities made them recall the topics better and helped them visualise the concepts being learned as was the case in other mobile learning studies (Baya’a and Daher 2009; Spikol and Eliasson 2010). Videos, animations and math manipulatives are typical mediums that are used to help visualise math concepts both in mobile learning environments and computer-based environments. However, with mobile devices, an additional medium for visualisation is the learners’ environment, facilitating a connection between the abstract math concepts to real world. Some students felt that this new way of doing maths had helped them grasp abstract maths concepts and as a result helped them remember better. These narratives were supported by a significant improvement in MT scores and further supported by the significantly higher gains of the experimental group in comparison to the control group. Findings in relation to maths performance were consistent with other mobile learning studies (Hwang et al. 2015; Riconscente 2013).

One of the highlights of mobile learning research is the context and setting of the learning environment. For example, Frohberg et al. (2009) review of state-of-the-art mobile learning studies included mobile learning activities in both formal and informal learning environments like museum, rivers, forests, towns, among many others. As discussed previously, mobile devices have the ability to capture data from the environment with its built-in sensors, camera and communication tools and these features help facilitate learning activities designed with the situated learning framework by allowing learning to take place in authentic context. For this study, it is difficult to ascertain how the incorporation of the outdoor space, the collaborative nature of the activity or the students’ perception of the activities contributed to the difference in the gains between the experimental and control group but it is also worth noting that these are exactly the potential of mobile technologies: to facilitate learning across context and provide personal and collaborative learning environments (Cochrane 2010).

Gender differences in perceptions and performance

There is a significant amount of literature that suggests gender difference in attitudes towards technology use. Male students tend to have higher perceptions about the value of technology in comparison to female students (Barkatsas et al. 2009; Reed et al. 2010) and this was also the case at the start of this study. Boys initially had higher perceptions about the usefulness of mobile technology (VMT). Throughout the end activity evaluations and the VMT test at the end, no gender difference was found. This result suggests that male and female students’ perceptions of mobile technology use did not vary. This finding is consistent with other mobile learning studies in mathematics (Tsuei et al. 2013; Deater-Deckard et al. 2014) where gender does is not a contributing factor to students’ evaluation of mobile learning activities.

As for the maths test, there was no significant gender difference found in the gain scores of either the experimental or control group. This suggests that the mobile learning intervention has been effective for either boys and girls but so does the traditional model followed by the control group. Older literature suggests that there are gender gaps in mathematics in favour of male students (Else-Quest et al. 2010) but the findings of this study suggests otherwise, which is more in keeping with recent meta-analysis (Voyer and Voyer 2014). A cross comparison of the scores by group and gender found a significant difference in the gain scores of female students in the experimental group in comparison with the female students in the control group. This trend in gain scores is consistent with other mobile learning studies on maths (Schacter and Jo 2016; Shin et al. 2012) where the effect of the intervention was greater for female students. It is worth noting, however, that results of this study could have been confounded by other variables such as novelty and research design. The next section discusses these limitations further.

Limitations of the study

Dependence of data on self-reports given the age of the participants was a shortcoming. This was minimised by repeatedly discussing with the students what the words meant whenever they had to evaluate the session. Students were also encouraged to ask for clarification whenever they were not sure what the question meant, which was particularly an issue for double negative statements.

Other threats to validity are in relation to sample size, research design, instrument reliability and novelty. The small sample is a threat to the validity of the results, so effect sizes were also provided to give an idea of the magnitude of the difference between the groups investigated. The no-treatment control group is a limitation of the design. While this was not the original intention of this study, it had been particularly difficult to recruit schools willing to involve a control group who would follow a similar programme to the experimental group. Another threat to validity was the low reliability scores for some scales of the mathematics attitudes test and the end activity evaluation. Nielsen (1993) quotes that “reliability of usability tests is a problem because of the huge individual differences between users (p. 166).” Novelty is also a possible threat to validity. While 3 months of intervention is longer than other mobile learning studies, this is still relatively short compared to other interventions that use more established technologies. Still, the use of the M3-Level evaluation framework has been valuable for data validation and triangulation. It is difficult to know which of the results can be attributed to novelty, to the nature of the activities or to the use of the mobile device.

Conclusions

This study set out to investigate the effects of using tablet devices for mathematics learning in terms of student attitudes, perception and their achievement. The design of the activities carried out in this study featured mobile technologies being used for active, collaborative learning activities. It also illustrated how these technologies can be used to allow students to engage with their environment as they explore maths concepts. We found a modest difference in students’ performance in a maths test but the weekly use of the tablets did not show a positive increase in students’ overall attitudes towards mathematics.

The findings suggest that the use of mobile technologies elicit positive responses from students, however, technical issues can be disruptive of the learning activities. While there are advantages in adopting these technologies in the classroom, it is worth emphasising how the design of the activities, the technical breakdowns and learner characteristics can make a difference to results. Just as it is important to consider the functionalities of the device and how it can be used to integrate into existing curriculum, it is also important to consider how the design of the activities fit with learner characteristics.

Ultimately, it is the teacher not just the technology that drives the change in the classroom. Technical breakdowns, student characteristics and learning design are all seen as teacher responsibilities and it would be worthwhile addressing how teachers are being trained to target those issues as well as being trained to use new technologies. Future research needs to address the role of the teacher in implementation.

Implications for future practice, policy and research

This study yielded a mixture of positive and negative results in using mobile technologies for mathematics. The variation in results means that while mobile learning could be beneficial in some situations, further research is needed to help identify which situations and learning environments are most suitable. The non-significant findings on effects on attitudes calls for further research with longer implementation times. This study was implemented as a once a week activity over a period of 3 months, and adopted active, collaborative learning activities. If the timeframe and frequency has been changed, what effect would this have on students’ attitudes? If the activities were different, would students still find it better than their regular maths?

In the current study, we opted to use various brands of more affordable tablets and provided our own wireless network. This raises financial and other practical concerns. Should the schools invest in these technologies or should schools utilise a Bring-Your-Own-Device policy? Where devices vary, what technical skills should teachers have to help students carry on with tasks if technical issues arise? Schools might have existing wireless connectivity, but could it cope with an influx of wireless devices? The questions above are just a few from the list of technical considerations that need to be addressed before implementing these technologies in the classroom. The field of mobile learning is new and the technology that supports it is changing very fast. The issues and questions raised here are among the many questions in this rapidly evolving field.