Advertisement

Journal of Computers in Education

, Volume 5, Issue 2, pp 133–148 | Cite as

Technology effectiveness in the mathematics classroom: a systematic review of meta-analytic research

  • Jamaal Young
  • Faith Gorumek
  • Christina Hamilton
Article

Abstract

The purpose of this systematic review was to examine trends in prior meta-analytic research to provide recommendations for future mathematics education research and instructional praxis. The current study aims to contextualize the effects of technology-enhanced instruction in the mathematics classroom. The researchers conducted a comprehensive literature search of articles written between 1980 and 2015. The final pool of studies comprised 18 meta-analyses inclusive of studies conducted between 1986 and 2014, representing 1193 independent effect sizes. The results suggest that the effects of technology on mathematics achievement range from small to large. Results suggest that researchers and educators should consider grade level, duration, and the instructional role of technology as key components when incorporating technology in the mathematics classroom. Results also suggest that race, socioeconomic status (SES), and gender did not moderate the effects of technology integration, although they were examined less frequently across studies. Implications are provided for practice, and research related to these results. Because of the chosen research approach, the research results provide relevant and practical implications to support classroom teaching with technology. This study contributes to the literature on technology-enhanced mathematics instruction by providing synthesis of 30 years of meta-analytic research.

Keywords

Technology-enhanced instruction Mathematics Systematic review Moderators 

Introduction

Digital technologies are exciting pedagogical tools that can enhance the delivery, clarity, and precision of mathematics instruction. Incorporating technology in the classroom makes an essential contribution to student success in mathematics (Nepo, 2017). Based on this trend, research examining effective use of technology in the mathematics classroom has grown exponentially. Over the last three decades, numerous meta-analytic studies have investigated technology’s effects on mathematics achievement and the factors that mediate these effects (Chan and Leung 2014; Li and Ma 2010). These meta-analyses provide summary effect size estimates, as well as moderators of the effect sizes across studies. Summary effect sizes are often the focus of traditional meta-analysis, while less emphasis is placed on the moderators of these effects.

Effect size reporting and its role in meta-analytic thinking are considered significant concerns in effective mathematics education research consumption and reporting. The American Psychological Association (APA 2010) and the American Educational Research Association (AERA 2006) regularly advocate for the reporting of effect sizes and more recently, considered meta-analytic thinking an extension to previous reporting practices. Numerous mathematics education scholars cite the benefits of effect size reporting and meta-analytic thinking through the presentation and interpretation of confidence intervals (Young et al. 2013; Young and Young 2016; Cumming 2012; Zientek et al. 2008). Effect sizes and confidence intervals are organic elements of meta-analytic research and represent metrics for comparison and summarization of effects across studies. Therefore, reviewing the trends in previous meta-analytic research on the moderators of the effects of technology integration on mathematics achievement is vital to the fidelity of technology integration research in the mathematics classroom.

However, to enhance the teaching and learning of mathematics with technology, researchers must refine theoretical constructs through empirical specification, which can, and should guide classroom applications. Moderators are often directly related to classroom implementation, and can be used to refine theoretical constructs thereby supporting empirical specification. Unfortunately, moderators of effect sizes are rarely synthesized in the empirical literature. Synthesizing the moderators of effect sizes across prior meta-analyses has empirical and practical importance to effective implementation of technology-enhanced teaching in the mathematics classroom.

The purpose of this systematic review was to examine the moderator analysis results for prior meta-analytic research to identify trends in empirical research and practice. It is our hope that results of this study provide recommendations for future research and instructional praxis. These results are relevant because they demonstrate how the expansion of meta-analytic thinking supports effective classroom teaching with technology.

Literature review

Prior syntheses and meta-analyses combine knowledge from individual studies to inform the teaching and learning practice with technology. Recent syntheses have examined the influence of technology-enhanced instruction on learning across a multitude of disciplines and contexts (Chang et al. 2018; Fu and Hwang 2018; Wang et al. 2017), however few studies have systematically reviewed prior meta-analyses to synthesize the results across first-order meta-analysis (Young et al. 2018; Gurevitch et al. 2018; Tamim et al. 2011). Within mathematics education research, numerous studies have examined the unique influences of specific technology integration on the teaching and learning of mathematics. Several studies have examined the relationship between teacher pedagogical beliefs and their use of technology in the mathematics classroom. The majority of prior meta-analytic research has focused on the unique effects of integrating specific technological tools in the mathematics classroom. In the sections that follow, the researchers review the effects of several common classroom technologies on student achievement in mathematics.

Computer-assisted instruction

The use of computers to guide and enhance mathematics learning is well documented. Two of the most common applications of computers in the mathematics classroom are computer-assisted instruction (CAI) and computer-based instruction (CBI). CAI and CBI are similar applications of computers in the classroom, but their instructional purposes are nuanced. Computer-assisted instruction (CAI) is a more precise term, often referring to the use of computers in drill and practice, tutorials, or simulation activities offered in substitution or as a supplement to traditional, teacher-directed instruction (Hicks and Holden 2007), while computer-based instruction (CBI) is broadly defined as the use of computers in the delivery of instruction (Kulik 1983). The effects of CAI and CBI on student achievement in general and in mathematics education specifically have been examined across a multitude of grade levels and diverse contexts (Yung and Paas 2015). Despite their nuances, CAI and CBI are often operationalized as learning delivered primarily by means of the computer, which typically incorporate drill and practice, simulations, and well-defined feedback mechanisms. CBI and CAI have been used interchangeably within prior meta-analyses in mathematics education research, and thus, they are discussed as one in the same here.

The results of prior meta-analyses have suggested that the effects of CBI/CAI on mathematics achievement vary from small to medium based on effect size benchmarks (Cohen 1992). Prior meta-analyses were conducted across grade levels and various types of mathematics content (Chadwick 1997; Chen 1994; Hsu 2003; Larwin and Larwin 2011; Lee 1990). CBI/CAI studies consistently conclude that duration and mode of instructional use were statistically significant moderators of study effects. These results are particularly pertinent as they relate to the length of treatment and the instructional modality necessary to enhance mathematics teaching and learning. Calculator use has a rich tradition within mathematics education, and unlike CAI or CBI, calculators are viewed as a more content specific instructional technology.

Calculators

Many mathematics educators continue to debate when to use calculators in the mathematics classroom within research and policy documents. The affordances of calculators as pedagogical tools cannot be denied. The variety of hand-held calculators continues to evolve. Today, calculators range from simple arithmetic calculators to scientific calculators, graphing calculators, and symbolic calculators with a variety of calculating modes, including algebraic systems and spreadsheets (Close et al. 2012). The National Council of Teachers of Mathematics (NCTM) contends that calculators are fundamental technologies in mathematics classrooms that enrich student understanding (NCTM 2000). Given the multiple perspectives on the use of calculators in the mathematics classroom, the results of a prior meta-analysis on the effects of calculators on mathematics achievement were instrumental to the acceptance of calculators as pedagogically meaningful tools.

The results of prior meta-analysis investigating calculator use and mathematics achievement tend to converge at the moderate level of effectiveness. Statistically, the significant moderators of calculator effects on mathematics achievement are grade level and assessment type (Ellington 2006; Hembree and Dessart 1986; Nikolaou 2001; Tokpah 2008). This is not surprising given that the grade level remains a point of contention. Many concerns remain regarding early exposure to calculators in the mathematics classroom, due in part to the inconsistencies in access during examinations. For instance, the results of the 2009 National Assessment of Educational Progress (NAEP) indicate that 66% of fourth graders claimed they never used a calculator for exams or quizzes, compared to only 28% of eighth graders surveyed (Planty et al. 2009). These results are further substantiated by trends observed in prior meta-analyses. Hembree and Dessart (1986) conclude, “average students (except fourth grade) who use calculators in concert with traditional mathematics instruction improve their basic skills with paper–pencil tasks, both in computational operations and in problem-solving” (p. 96). Therefore, assessment and grade require additional pedagogical consideration.

Mathematics software and emerging trends in mobile technology

Mathematics software applications vary from general to specific forms, such as digital geometry software (DGS), and virtual manipulatives. Compared to CAI/CBI and calculator use in the mathematics classroom these tools are relatively under-researched. Thus fewer meta-analyses exist. The overall effect sizes for mathematics software applications range from 0.09 to 1.02 (Chan and Leung 2014; Cheung and Slavin 2013; Moyer-Packenham and Westenskow 2013; Steenbergen-Hu and Cooper 2013). The consistent statistically significant moderators of effect sizes observed in the literature are grade level, duration, and mathematics subject matter (algebra, geometry, etc.). This indicates that the divergence in effect sizes across these studies may be attributed to these aforementioned moderators.

Emerging research trends tend to focus on the effects of mobile technology on the teaching and learning of mathematics. For example, Bano et al. (2018) identified three themes within the pedagogical approaches present in the mathematics and science instruction with mobile devices literature. These approaches were collaboration, inquiry-based learning, and realistic learning. Fabian et al. (2016) found the overall mean effect of mobile technology on achievement in elementary mathematics was .48. The researchers also found that the results of studies in middle grades classrooms were positive overall, but the effects on high school environments were mixed. Given that the results of prior meta-analysis provide credence to the use of technology-enhanced teaching methods in the mathematics classroom, but lack overarching prescriptive conclusions for general praxis with technology, a summary of effects across prior meta-analysis is warranted.

Moderators and meta-analytic thinking

A meta-analytic lens may serve as the most suitable empirical tool to identify the best practices with technology in the mathematics classroom. Meta-analysis is a research synthesis tool that uses summaries of effect sizes to generate empirical conclusions from ostensibly similar studies. Meta-analysis involves (1) summarizing several studies regarding effect sizes, and (2) combining the results to make summative inferences (Cooper 2016). This process involves calculating the average effect size, testing for homogeneity, detecting moderators, and explaining any heterogeneity (Hunter and Schmidt 2004). The detection of moderators is the critical feature of any meta-analytic study; because differences in strength and direction in effect sizes are identified here. Rosenthal (1991) argues, “The search for moderators is not only an exciting intellectual enterprise but indeed…it is the very heart of scientific enterprise” (p. 447). Moderators offer conditions for effects that are theorized, thus informing researchers of the circumstances in which the effects under investigation can be reliable (Schmidt and Hunter 2014). This information is vital to successful implementation of technology in the mathematics classroom across instructional contexts.

Using the lens of meta-analytic thinking, researchers can make better decisions about technology integration in the mathematics classroom. Meta-analysis can help researchers find specific variables that account for the variance in the effectiveness of technology integration in the mathematics classroom. Moderators quantify qualitative variables that influence the strength or direction of relationships in meta-analytic research (Steel et al. 2002). Moderators are also important because they identify statistical interactions, which do not imply causation but rather add context to effect size results (Cooper and Patall 2009). Given the distinctions among the associations they identify, moderators are consistently placed in three categories. Moderators are categorized as either: (1) methodological variations, (2) theoretical constructs, or (3) study characteristics (DeCoster 2004).

Methodological variations refer to components of the experimental design such as sample size, random assignment, or treatment duration. Theoretical constructs are moderators grounded in theory or based on the application of recognized theoretical trends. The final category of moderator variables includes study related artifacts, such as publication status or publication year. Moderators are recognized for their ability to enhance theory development and increase the general richness of empirical work (Aguinis et al. 2011). Given the empirical merit of meta-analytic research and the contextualization offered by moderator analysis, systematically summarizing results across studies is practically and scientifically necessary. To examine the effect of technology on achievement in mathematics across multiple contexts, therefore, a literature survey was conducted to identify and characterize pertinent moderators of effect size. The present study was guided by the following research question:

1. How are the moderators of effect size characterized in prior meta-analyses of technology-enhanced mathematics instruction?

Method

The current systematic review utilized the Preferred Reporting Items for Systematic Reviews and Meta-analysis (PRISMA) protocol. According to Moher, Liberati, Tetzlaff, and Altman Moher et al. (2009), PRISMA represents a set of evidence-based items that represent accepted practices for conducting systematic reviews and meta-analysis. Eligible studies were limited to meta-analyses written between 1980 and 2015. Due to the focus on meta-analyses, systematic reviews, literature synthesis, and traditional qualitative or quantitative studies were not included.

Data sources were electronic databases covering education, psychology, and social sciences. The specific databases included JSTOR, ERIC, EBSCO, Pych INFO, and ProQuest. In each database, an initial search was performed against the abstracts using the Boolean search term “meta-analysis OR systematic reviewANDmathematics OR STEMANDtechnology OR digital”. Whenever possible, search limiters were used to align the initial search results more closely with the eligibility criteria. For example, most databases allow limiting the search to a specific date range. The search was concluded in January of 2016.

Screening process

Figure 1 presents the complete study inclusion and exclusion process. The screening criteria shown in Table 1 guided the selection of articles from the initial pool. First, study titles and abstracts were screened for relevance to the research question and study topic of interest. Then, the remaining studies were screened against the criteria provided in Table 1. The initial pool of 42 studies was systematically screened using this process and reduced to a final pool of 18. As shown in Fig. 1, most of studies were removed for lack of effect size reporting and the absence of a digital technology focus. Pertinent data related to the research questions were extracted from the remaining studies.
Fig. 1

Technology meta-analysis study inclusion flowchart

Table 1

Inclusion Screening Process

Criterion

Include

Exclude

Publication Year

1980-2015

Before 1980

Language

English

Non-English

Context

Classroom settings

Context other than classroom settings

Research Design

Meta-analysis

Other empirical research (quantitative, qualitative, mixed-methods), secondary data analysis, and systematic reviews

Participants

K-12 and post-secondary students

Non-student populations

Relevance

Examined the effects of technology-enhanced instruction on mathematics achievement

Did not study the effects of technology enhanced instruction on mathematics achievement

Data collection and analysis

The extraction protocol presented in Table 2 guided data extraction from the retained articles. Data extracts included citation, purpose, mean effect sizes, number of independent effect sizes, moderators, and key findings. Extracted data were stored in a database indexed by article. In addition, Results, Discussion, and Conclusion sections of each article were extracted and stored in a database for critical analysis.
Table 2

Data Extraction Protocol

Extract

Description

Citation

Author(s) and publication date

Source

Article, conference proceeding, or dissertation

Purpose

Purpose, objectives, research questions

K

Number of independent effect sizes included in meta-analysis

ES

Mean effect size

Moderators

Detailed description of moderators (operationalization, statistical significance, etc.)

Key findings

Summary of main findings and conclusions

To examine the moderators affecting the strength and direction of the results, each meta-analysis’ methodological, theoretical, and study characteristic moderators, the researchers used a semi-structured coding protocol based on an adapted list of features and trends found in the systematic review. Moderators were coded verbatim initially, and then coded categorically after all studies were reviewed. Moderator categories were based on operational definitions that emerged during the coding discussions and data extraction process. The researchers assessed coding reliability by comparing the independent coding results from the studies. The initial inter-rater agreement was 95%, and we met to resolve the remaining inconsistencies in the coding results.

Data were analyzed descriptively to best characterize the trends in moderator influence on effect size variability. Frequency counts for each moderator were recorded along with the Q B statistics, and p-values. Moderators were assigned a rating of high, medium, or low based on the ratio between the frequencies of statistically significant observation compared to the total number of observations for that particular moderator. These data represent a measure of the impact of each moderator across the studies reviewed in the current study.

Results

The final pool of studies comprised 18 meta-analyses inclusive of studies conducted between 1986 and 2014, representing 1193 independent effect sizes (Table 3). The median year of publication was 2007 and the range for the year of publication was 28 years. A complete list of study characteristics is presented in Table 1, which shows that the majority of the meta-analyses were journal articles (10 out of 18) and the remaining meta-analyses were dissertation studies. All studies except one included either an overall mean effect size or sufficient data to calculate the overall effect size. Only one study reported an overall negative effect size, and the overall effect sizes ranged from − .11 to 1.02 in magnitude.
Table 3

Description of Included Meta-Analysis

Citation

Purpose

Source

k

ES

Hembree and Dessart (1986)

Integrate the findings of the research on effects on students of using calculators in learning mathematics in Grades K-12

Journal

29

.64

Lee (1990)

Determine the Effectiveness of CAI in elementary and secondary instruction

Dissertation

243

.38

Chen (1994)

Synthesize and extract the main findings from studies on computer-based instruction (CBI) in mathematics education

Dissertation

76

.50

Chadwick (1997)

Examine the effects of CAI in the secondary mathematics classroom on cognitive and affective outcomes

Dissertation

41

.51

King (1997)

Determine the effect of computer-enhanced instruction (CEI) on college level mathematics

Dissertation

30

.20

Nickolau (2001)

Synthesize the effects of hand-held calculators on K-12 mathematics achievement

Dissertation

24

.54

Hsu (2003)

Examined the effectiveness of Computer-Assisted Instruction (CAI) instruction in statistics education

Dissertation

25

.43

Ellington (2006)

Examined the effects of calculator use on student achievement and attitude levels

Journal

54

NA

Schenker (2007)

Examine the effectiveness of using technology to enhance statistics instruction

Dissertation

117

.24

Tokpah 2008)

Examined the Effects of Computer Algebra systems (CAS) on mathematics achievement

Dissertation

102

0.38

Rosen and Salomon (2007)

Examined the effectiveness of constructivist technology intensive learning environments versus traditional learning environments

Journal

32

0.46

Wang et al. (2007)

Examined the effect of testing mode (computer vs. paper and pencil) on mathematics achievement

Journal

14

− .11

Li and Ma (2010)

Examined the effects on computer Technology on mathematics achievement in K-12

Journal

46

0.28

Larwin and Larwin (2011)

Determine the effectiveness of CAI student mathematics achievement in post-secondary statistics courses

Journal

219

.57

Cheung and Slavin (2013)

Examined the effects of educational technology on mathematics achievement in K-12 settings

Journal

74

0.16

Steenbergen-Hu and Cooper (2013)

Examined the effects of intelligent tutoring systems on K-12 mathematics achievement

Journal

26

0.09

Moyer-Packenham and Westenskow (2013)

Synthesize the findings examining the effects of virtual manipulatives on student achievement

Journal

32

.35

Chan and Leung (2014)

Evaluate the effects of digital geometry software on mathematics achievement

Journal

9

1.02

To answer the research question, the researchers identified 17 specific moderators by screening the frequency data using IBM SPSS Statistics 20; then, based on characterizations and definitions presented in prior research narrowed the list of moderators. The researchers operationalized each moderator for fidelity. Table 4 presents the operational definition of each moderator, frequency of investigation, and impact based on the ratio between the number of times the moderator was a statistically significant predictor and the moderator identification frequency. Next, the researchers ranked the moderator impact as either (low < .50), (medium ≈ .50), or (high > .50) based on the data observed. The researchers listed the moderators in descending order of frequency of observation for explanatory transparency.
Table 4

Moderator Operational Definitions, Frequencies, and Impact

Moderator

Operationalization

f

Rank

Grade level

Grade spans and other ordinal descriptions of student progress within educational systems (elementary, secondary, freshman, etc.)

12

High

Role

Instructional role of technology. Examples substitute, supplement, etc

10

High

Duration

Length of treatment reported in days, weeks, or months

9

High

Ability

Non-exceptionality classification of students based on prior achievement

8

Medium

Mode

Instructional modality of technology use in the classroom. ex. drill/practice, tutorial, simulation, problem-solving, etc

8

High

Assessment

Describes the role of technology in the delivery and completion of mathematics formative and summative assessment. ex. computer-based assessments, access to calculators during exams, etc

7

High

SES

Socioeconomic Status (SES), the social status or class of an individual or group. Operationalized as low and high based on receipt of free or reduced lunch

6

Low

Subject-matter

Mathematics content strands—Algebra, geometry, fractions, statistics, calculus, etc

6

High

Gender

Reported dichotomous factors. male and female

5

Low

Concentration

Time per session expressed in minutes or hours

5

High

Technology type

Specific category or type of technological tool – graphing vs. basic calculator, virtual manipulative, computer-assisted instruction, etc

5

Low

Differentiation

Student receiving specialized instruction – Special education, ESL, ELL, GT, etc

4

Medium

Race

Student self-reported background – Asian, Black, Latino, White, Other

3

Low

Community

Urban, rural, or suburban described by population density

3

Low

Teacher

Teacher’s positionality as either facilitator or instructor regarding technology integration

3

High

Organization

Describes whether treatment was delivered individually, in pairs, or in groups of three or more

3

Low

Access

Accessibility of technology reported as low, medium, or high. Pertains to students’ ability to use technology at will

2

Medium

Grade level was the most frequently observed moderator while access was the least frequently observed. The remaining moderators in the upper quartile, listed in descending order, were role, duration, ability, and mode. The lower quartiles of moderators in ascending order were organization, teacher, community, and race. Results in Table 4 suggest that four out of the five moderators identified in the upper quartile of frequency had a high impact on the variability of meta-analysis results. Most of the lower quartile moderators have a low impact on the variability of effect sizes. The teacher was the only moderator in the lower quartile that had a high impact on the variability of meta-analysis results. In summary, eight moderators ranked as high, three as medium, and six as low. In the discussion section that follows, the researchers provide substantive conclusions and implications for teachers, administrators, and researchers based on these results.

Discussion

The purpose of this systematic review was to examine the moderator analysis results for prior meta-analytic research to identify trends in empirical research and practice. The analysis of this research compared different conceptualizations of learning with technology examined measurement in mathematics classrooms, and identified common and generalizable findings across the meta-analyses regarding the moderators of the effectiveness of technology integration in mathematics classrooms. The results suggest that the effects of technology integration on mathematics achievement vary from negligible to large, but are consistently small. However, given the practical significance of a small effect size in the mathematics classroom this finding has educative merit for teachers and administrators (Hill et al. 2008).

Moderators are important tools to use when evaluating and planning technology-mediated learning in the mathematics classroom. Thus, the focus of this study was on moderators of the effects of meta-analyses. The 17 moderators investigated in this study varied in frequency of investigation and impact on effect size differences. Grade level, role, and duration were the three most investigated moderators, and all had a high impact on effect size variation across the results of the meta-analyses examined. In the 12 studies examining the moderator variable grade level, 66% found that grade level had a statistically significant influence on the student achievement effects variability when technology was integrated in the mathematics classroom. Many included studies favored technology in middle and high school classrooms as opposed to early elementary settings. This conclusion is consistent with prior studies that found that technology is utilized less in elementary schools compared to high school classrooms (Brown et al. 2007). Researchers should continue to assess this phenomenon, and teachers and administrators should examine the grade level implications of technology integration closely when designing and applying interventions.

The results of the ten studies examining the instructional role of technology suggest that technology was a statistically significant moderator of effect sizes in 90% of the meta-analysis examined. Only one study concluded that the instructional role of technology was not a statistically significant moderator of effect sizes, but the findings from the other nine studies concluded that effect sizes were larger when technology was used to supplement or augment instead of substitute or replace traditional instruction in the classroom. In addition to traditional instructional tools such as software resources and standard tools such as graphing calculators, educators are exploiting a variety of technological tools in mathematics instruction, including cell phones and other mobile technologies (Gay and Burbridge 2016; Young and Young 2012; Davis 2010; Valk et al. 2010). As studies continue to use these tools, it will be important to revisit the effects of instructor role on achievement.

Third, duration was assessed in nine studies and found to be a significant moderator of effect sizes in 89% of the studies investigated. Most studies found that at approximately three weeks the effect of a technology intervention weans. Thus, researchers and educators need to be cognizant of overexposure and the novelty factor. Other considerable instructional findings were that mode of instruction, assessment, subject matter, concentration, and teacher instructional orientation all statistically significantly influenced the variability of effect sizes in the meta-analysis; however, these moderators were investigated less often across studies. Such student demographic variables such as race, gender, SES, and community were not consistent moderators of the effects of technology on mathematics achievement.

Limitations

Summarizing effects of moderators on the effect sizes across meta-analyses has several limitations. First, much of the data pertinent to each moderator resides at the individual study level. This is problematic because a precise estimation of the exact influences of all moderators assessed in prior meta-analyses would be difficult to feasibly examine even through second-order meta-analysis (Young 2017). Thus, a representative sample of moderators that could be assessed at the meta-analysis rather than at study level was selected for systematic review in the present study.

Conclusion

This study provides a comprehensive systematic review and literature survey of research conducted from 1985 until 2015. Based on the summary of almost 30 years of research, this study provides important conclusions related to the effectiveness and moderators of technology integration in mathematics classrooms. In conclusion, the results of this systematic review indicate that technology integration supports mathematics achievement across prior meta-analytic research. However, the statistically significant moderators of the effects vary across studies.

Based on these results, the researchers recommend that teachers and researchers continue to implement technology in the mathematics classroom, but emphasize optimizing the effects grade level, role of technology, and duration. The researchers also recommend further research into demographic variables, which were investigated less frequently across studies. In addition, more research is necessary to capture the unique influences of teachers on the effects of technology integration in the mathematics classroom. Finally, the researchers recommend further investigation into variables such as student access to technology at home and the effects of instructional context regarding the duration effects of technology integration. Armed with these recommendations, researchers and educators are better equipped to make informed decisions concerning the when, where, and how of integrating technology in the mathematics classroom.

Notes

Compliance with ethical standards

Conflict of interest

All authors declared that they have no conflict of interest

Ethical approval

No animals were involved in this project. All procedures performed in studies involving human participants were in accordance with the ethical standards of the institutional and/or national research committee and with the 1964 Helsinki declaration and its later amendments or comparable ethical standards.

Informed consent

Informed consent was obtained from all individual participants included in the study.

References

  1. Aguinis, H., Gottfredson, R. K., & Wright, T. A. (2011). Best-practice recommendations for estimating interaction effects using meta-analysis. Journal of Organizational Behavior, 32(8), 1033–1043.CrossRefGoogle Scholar
  2. American Educational Research Association. (2006). Standards for reporting on empirical social science research in AERA publications. Educational Researcher, 35(6), 33–40.CrossRefGoogle Scholar
  3. American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.Google Scholar
  4. Bano, M., Zowghi, D., Kearney, M., Schuck, S., & Aubusson, P. (2018). Mobile learning for science and mathematics school education: A systematic review of empirical evidence. Computers & Education, 121, 30–58.CrossRefGoogle Scholar
  5. Brown, E. T., Karp, K., Petrosko, J. M., Jones, J., Beswick, G., Howe, C., et al. (2007). Crutch or catalyst: Teachers’ beliefs and practices regarding calculator use in mathematics instruction. School Science and Mathematics, 107(3), 102–116.CrossRefGoogle Scholar
  6. Chadwick, D. K. H. (1997). Computer-assisted instruction in secondary mathematics classrooms: A meta-analysis (Ed.D.). Drake University, United States-Iowa. Retrieved from http://search.proquest.com/pqdtglobal/docview/304389928/abstract/9D8837D0D8EB4A14PQ/11. Accessed 10 Sept 2015.
  7. Chan, K. K., & Leung, S. W. (2014). Dynamic geometry software improves mathematical achievement: Systematic review and meta-Analysis. Journal of Educational Computing Research, 51(3), 311–325.CrossRefGoogle Scholar
  8. Chang, C. Y., Lai, C. L., & Hwang, G. J. (2018). Trends and research issues of mobile learning studies in nursing education: A review of academic publications from 1971 to 2016. Computers & Education, 116, 28–48.CrossRefGoogle Scholar
  9. Chen, T. Y. (1994). A meta-analysis of effectiveness of computer-based instruction in mathematics (Ph.D.). The University of Oklahoma, United States - Oklahoma. Retrieved from http://search.proquest.com/pqdtglobal/docview/304105627/abstract/9D8837D0D8EB4A14PQ/5. Accessed 10 Sept 2015.
  10. Cheung, A. C. K., & Slavin, R. E. (2013). The effectiveness of educational technology applications for enhancing mathematics achievement in K-12 classrooms: A meta-analysis. Educational Research Review, 9, 88–113.CrossRefGoogle Scholar
  11. Close, S., Oldham, E., Shiel, G., Dooley, T., & O’Leary, M. (2012). Effects of calculators on mathematics achievement and attitudes of ninth-grade students. Journal of Educational Research, 105(6), 377–390.CrossRefGoogle Scholar
  12. Cohen, J. (1992). A power primer. Psychological Bulletin, 112(1), 155–159.CrossRefGoogle Scholar
  13. Cooper, H. (2016). Research synthesis and meta-analysis: A step-by-step approach (Vol. 2). Thousand Oaks: Sage publications.Google Scholar
  14. Cooper, H., & Patall, E. A. (2009). The relative benefits of meta-analysis conducted with individual participant data versus aggregated data. Psychological Methods, 14(2), 165–176.CrossRefGoogle Scholar
  15. Cumming, G. (2012). Understanding the New Statistics: Effect Sizes, Confidence Intervals, and Meta-analysis. Routledge: Routledge publications.Google Scholar
  16. Davis, M. R. (2010). Solving algebra on smartphones. Technology Counts, 29(26), 20–23.Google Scholar
  17. DeCoster, J. (2004). Meta-analysis notes. Retrieved from http://www.stathelp.com/Meta%20analysis%202009-06-01.pdf. Accessed 7 Nov 2015.
  18. Ellington, A. J. (2006). The effects of non-CAS graphing calculators on student achievement and attitude levels in mathematics: A meta-analysis. School Science and Mathematics, 106(1), 16–23.CrossRefGoogle Scholar
  19. Fabian, K., Topping, K. J., & Barron, I. G. (2016). Mobile technology and mathematics: Effects on students’ attitudes, engagement, and achievement. Journal of Computers in Education, 3(1), 77–104.CrossRefGoogle Scholar
  20. Fu, Q. K., & Hwang, G. J. (2018). Trends in mobile technology-supported collaborative learning: A systematic review of journal publications from 2007 to 2016. Computers & Education, 119, 129–143.CrossRefGoogle Scholar
  21. Gay, A. S., & Burbridge, L. (2016). “Bring Your Own Device” for Formative Assessment. Mathematics Teacher, 110(4), 310–313.CrossRefGoogle Scholar
  22. Gurevitch, J.,Koricheva, J.,Nakagawa, S., & Stewart, G. (2018). Meta-analysis and the science of research synthesis. Nature, 555 (7695), Retrieved from https://www.nature.com/articles/nature25753.pdf. Accessed 15 Mar 2018.
  23. Hembree, R., & Dessart, D. J. (1986). Effects of hand-held calculators in precollege mathematics education: A meta-analysis. Journal for Research in Mathematics Education, 17, 83–99.CrossRefGoogle Scholar
  24. Hicks, D., & Holden, C. (2007). Teaching the global dimension: Key principles and effective practice. London: Routledge.Google Scholar
  25. Hill, C. J., Bloom, H. S., Black, A. R., & Lipsey, M. W. (2008). Empirical benchmarks for interpreting effect sizes in research. Child Development Perspectives, 2(3), 172–177.CrossRefGoogle Scholar
  26. Hsu, Y. (2003). The effectiveness of computer-assisted instruction in statistics education: A meta-analysis (Ph.D.). The University of Arizona, United States - Arizona. Retrieved from http://search.proquest.com/pqdtglobal/docview/305338759/abstract/9D8837D0D8EB4A14PQ/29. Accessed 1 Aug 2015.
  27. Hunter, J. E., & Schmidt, F. L. (2004). Methods of Meta-Analysis: Correcting Error and Bias in Research Findings (2nd ed.). Thousand Oaks: Sage.CrossRefGoogle Scholar
  28. King H. J. (1997). Effects of computer-enhanced instruction in college-level mathematics as determined by a meta-analysis (vol. 59, p. 114A). United States: The University of Tennessee.Google Scholar
  29. Kulik, J. A. (1983). Synthesis of research on computer-based instruction. Educational Leadership, 41(1), 19–21.Google Scholar
  30. Larwin, K., & Larwin, D. (2011). A meta-Analysis examining the impact of computer-assisted instruction on postsecondary statistics education: 40 years of research. Journal of Research on Technology in Education, 43(3), 253–278.CrossRefGoogle Scholar
  31. Lee, W. C. (1990). The effectiveness of computer-assisted instruction and computer programming in elementary and secondary mathematics: A meta-analysis (Ed.D.). University of Massachusetts Amherst, United States - Massachusetts. Retrieved from http://search.proquest.com/pqdtglobal/docview/303852079/abstract/9D8837D0D8EB4A14PQ/18.
  32. Li, Q., & Ma, X. (2010). A meta-analysis of the effects of computer technology on school students’ mathematics learning. Educational Psychology Review, 22(3), 215–243.CrossRefGoogle Scholar
  33. Moher, D., Liberati, A., Tetzlaff, J., & Altman, D. G. (2009). Preferred reporting items for sys- tematic reviews and meta-analyses: The PRISMA statement. Annals of Internal Medicine, 151, 264–269.  https://doi.org/10.7326/0003-4819-151-4-200908180-00135.CrossRefGoogle Scholar
  34. Moyer-Packenham, P. S., & Westenskow, A. (2013). Effects of virtual manipulatives on student achievement and mathematics learning. International Journal of Virtual and Personal Learning Environments, 4(3), 35–50.Google Scholar
  35. National Council of Teachers of Mathematics. (2000). Principles and standards of school mathematics. Reston: Author.Google Scholar
  36. Nepo, K. (2017). The use of technology to improve education. Child & Youth Care Forum, 46(2), 207–221.CrossRefGoogle Scholar
  37. Nikolaou, C. (2001). Hand-held calculator use and achievement in mathematics: A meta analysis (Ph.D.). Georgia State University, United States - Georgia. Retrieved from http://search.proquest.com/pqdtglobal/docview/304696658/abstract/9D8837D0D8EB4A14PQ3. Accessed 10 Sept 2015.
  38. Planty, M., Hussar, W., Snyder, T., Kena, G., KewalRamani, A., Kemp, J., Bianco, K., & Dinkes, R. (2009). The Condition of Education 2009 (NCES 2009-081). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education. Washington, DC.Google Scholar
  39. Rosen, Y., & Salomon, G. (2007). The differential learning achievements of constructivist technology-intensive learning environments as compared with traditional ones: A meta-analysis. Journal of Educational Computing Research, 36(1), 1–14.CrossRefGoogle Scholar
  40. Rosenthal, R. (1991). Meta-analytic procedures for social research (rev ed.). Beverly Hills: Sage.CrossRefGoogle Scholar
  41. Schenker, J. D. (2007). The effectiveness of technology use in statistics instruction in higher education: A meta-analysis using hierarchical linear modeling. (Doctoral dissertation). Retrieved from ProQuest Digital Dissertations. (AAT 3286857).Google Scholar
  42. Schmidt, F. L., & Hunter, J. E. (2014). Methods of meta-analysis: Correcting error and bias in research findings. Thousand Oaks: Sage publications.Google Scholar
  43. Steenbergen-Hu, S., & Cooper, H. (2013). A meta-analysis of the effectiveness of intelligent tutoring systems on K–12 students’ mathematical learning. Journal of Educational Psychology, 105(4), 970–987.Google Scholar
  44. Steel, P. D., & Kammeyer-Mueller, J. D. (2002). Comparing meta-analytic moderator estimation techniques under realistic conditions. Journal of Applied Psychology, 87(1), 96–111.CrossRefGoogle Scholar
  45. Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational research, 81(1), 4–28.CrossRefGoogle Scholar
  46. Tokpah, C. L. (2008). The effects of computer algebra systems on students’ achievement in mathematics (Ph.D.). Kent State University, United States - Ohio. Retrieved from http://search.proquest.com/pqdtglobal/docview/304549974/abstract/9D8837D0D8EB4A14PQ. Accessed 1 Dec 2015.
  47. Valk, J., Rashid, A. T., & Elder, L. (2010). Using mobile phones to improve educational outcomes: An analysis of evidence from Asia. International Review of Research in Open and Distance Learning, 11(1), 117–140.Google Scholar
  48. Wang, H. Y., Liu, G. Z., & Hwang, G. J. (2017). Integrating socio-cultural contexts and location-based systems for ubiquitous language learning in museums: A state of the art review of 2009–2014. British Journal of Educational Technology, 48(2), 653–671.CrossRefGoogle Scholar
  49. Wang, S., Jiao, H., Young, M. J., Brooks, T., Olson, J. (2007). A meta-analysis of testing mode effects in grade K-12 mathematics tests. Educational and Psychological Measurement, 67(2), 219–238CrossRefGoogle Scholar
  50. Young, J. (2017). Technology-enhanced mathematics instruction: A second-order meta-analysis of 30 years of research. Educational Research Review, 22, 19–33CrossRefGoogle Scholar
  51. Young, J. L., Young, J. R., & Capraro, R. M. (2018). Gazing past the gaps: A growth-based assessment of the mathematics achievement of black girls. The Urban Review, 50(1), 156–176.CrossRefGoogle Scholar
  52. Young, J. R., & Young, J. L. (2012). “But that’s not fair”: Teacher technology readiness and African American Students’. The Journal of the Texas Alliance of Black School Educators, 4(1), 19–32.Google Scholar
  53. Young, J. R., Young, J. L., & Hamilton, C. (2013). The use of confidence intervals as a meta-analytic lens to summarize the effects of teacher education technology courses on preservice teacher TPACK. Journal of Research on Technology in Education, 46(2), 149–172.CrossRefGoogle Scholar
  54. Young, J. R., & Young, J. L. (2016). Young, black, and anxious: Describing the black student mathematics anxiety research using confidence intervals. Journal of Urban Mathematics Education, 9(1), 79–93.Google Scholar
  55. Yung, H. I., & Paas, F. (2015). Effects of computer-based visual representation on mathematics learning and cognitive load. Educational Technology and Society, 18(4), 70–77.Google Scholar
  56. Zientek, L. R., Capraro, M. M., & Capraro, R. M. (2008). Reporting practices in quantitative teacher education research: One look at the evidence cited in the AERA Panel Report. Educational Researcher, 37(4), 208–216.CrossRefGoogle Scholar

Copyright information

© Beijing Normal University 2018

Authors and Affiliations

  1. 1.Department of Teacher Education and AdministrationUniversity of North TexasDentonUSA
  2. 2.University of North TexasDentonUSA
  3. 3.Texas A&M Central TexasKilleenUSA

Personalised recommendations