Advertisement

Mathematics Education Research Journal

, Volume 27, Issue 4, pp 423–441 | Cite as

The influence of test mode and visuospatial ability on mathematics assessment performance

  • Tracy Logan
Original Article
First Online:

Abstract

Mathematics assessment and testing are increasingly situated within digital environments with international tests moving to computer-based testing in the near future. This paper reports on a secondary data analysis which explored the influence the mode of assessment—computer-based (CBT) and pencil-and-paper based (PPT)—and visuospatial ability had on students’ mathematics test performance. Data from 804 grade 6 Singaporean students were analysed using the knowledge discovery in data design. The results revealed statistically significant differences between performance on CBT and PPT test modes across content areas concerning whole number algebraic patterns and data and chance. However, there were no performance differences for content areas related to spatial arrangements geometric measurement or other number. There were also statistically significant differences in performance between those students who possess higher levels of visuospatial ability compared to those with lower levels across all six content areas. Implications include careful consideration for the comparability of CBT and PPT testing and the need for increased attention to the role of visuospatial reasoning in student’s mathematics reasoning.

Keywords

Secondary data analysis Computer-based testing Visuospatial ability Mathematics assessment 
This is a preview of subscription content, log in to check access.

Notes

Acknowledgments

The author wishes to thank Professor Tom Lowrie for generously allowing me to re-analyse his data and for his comments and insights into the paper.

References

  1. Baker, R. S. J. D. (2010). Data mining for education. In B. McGaw, P. Peterson, & E. Baker (Eds.), International Encyclopedia of Education (3rd ed., pp. 112–118). Oxford, UK: Elsevier.CrossRefGoogle Scholar
  2. Battista, M. T. (1990). Spatial visualization and gender differences in high school geometry. Journal for Research in Mathematics Education, 21, 47–60.CrossRefGoogle Scholar
  3. Bennett, R. E., Braswell, J., Oranje, A., Sandene, B., Kaplan, B., & Yan, F. (2008). Does it matter if I take my mathematics test on a computer? A second empirical study of mode effects in NAEP. Journal of Technology, Learning and Assessment, 6(9), Retrieved September 25 from http://www.jtla.org
  4. Bugbee, A. C. (1996). The equivalence of paper-and-pencil and computer-based testing. Journal of Research on Computing in Education, 28(3), 282–290.CrossRefGoogle Scholar
  5. Casey, M. B., Nuttall, R. L., & Pezaris, E. (1997). Mediators of gender differences in mathematics college entrance test scores: A comparison of spatial skills with internalized beliefs and anxieties. Developmental Psychology, 33, 669–680.CrossRefGoogle Scholar
  6. Cheng, Y.-L., & Mix, K. S. (2014). Spatial training improves children’s mathematics ability. Journal of Cognition and Development, 15(1), 2–11. doi: 10.1080/15248372.2012.725186.CrossRefGoogle Scholar
  7. Clariana, R., & Wallace, P. (2002). Paper-based versus computer-based assessment: key factors associated with test mode effect. British Journal of Educational Technology, 33(5), 593–602.CrossRefGoogle Scholar
  8. Clements, D. H. (2004). Geometric and spatial thinking in early childhood education. In D. H. Clements, J. Sarama, A.-M. DiBiase, & A.-M. DiBiase (Eds.), Engaging young children in mathematics: standards for early childhood mathematics education (pp. pp. 267–pp. 297). Mahwah: Erlbaum.Google Scholar
  9. Clements, D. H., & Battista, M. T. (1992). Geometry and spatial reasoning. In D. A. Grouws (Ed.), Handbook of research on mathematics teaching and learning (pp. 420–464). New York: Macmillan.Google Scholar
  10. DeAngelis, S. (2000). Equivalency of computer-based and paper-and-pencil testing. Journal of Allied Health, 29(3), 161–164.Google Scholar
  11. Devine, P. (2003). Secondary data analysis. In R. L. Miller & J. D. Brewer (Eds.), The A-Z of social research (pp. pp. 286–pp. 289). London: SAGE Publications, Ltd. doi: 10.4135/9780857020024.n97.Google Scholar
  12. Ekstrom, R. B., French, J. W., Harman, H., & Derman, D. (1976). Kit of factor-referenced cognitive tests. Princeton: Educational Testing Service.Google Scholar
  13. Fayyad, U., Piatetsky-Shapiro, G., & Smyth, P. (1996). From data mining to knowledge discovery in databases. AI Magazine, 17(3), 37–54.Google Scholar
  14. Field, A. (2013). Discovering statistics using IBM SPSS statistics (4th ed.). Thousand Oaks, CA: Sage Publications Inc.Google Scholar
  15. Fuchs, L. S., Geary, D. C., Compton, D. L., Fuchs, D., Hamlett, C. L., Seethaler, P. M., Bryant, J. V., & Schatschneider, C. (2010). Do different types of school mathematics development depend on different constellations of numerical and general cognitive abilities? Developmental Psychology, 46, 1731–1746. doi: 10.1037/a0020662.CrossRefGoogle Scholar
  16. Halpern, D. F., & Collaer, M. L. (2005). Sex differences in visuospatial ability: more than meets the eye. In P. Shah & A. Miyake (Eds.), The Cambridge Handbook of Visuospatial Thinking (pp. 170–212). New York: Cambridge University Press.CrossRefGoogle Scholar
  17. Hardré, P. L., Crowson, H. M., Xie, K., & Ly, C. (2007). Testing differential effects of computer-based, web-based and paper-based administration of questionnaire research instruments. British Journal of Educational Technology, 38(1), 5–22.CrossRefGoogle Scholar
  18. Johnson, M., & Green, S. (2006). On-Line mathematics assessment: The impact of mode on performance and question answering strategies. Journal of Technology, Learning, and Assessment, 4(5). Retrieved September 25 from http://www.jtla.org.
  19. Kirby, J. R., & Boulter, D. R. (1999). Spatial ability and transformational geometry. European Journal of Psychology of Education, 14(2), 283–294.CrossRefGoogle Scholar
  20. Kozhevnikov, M., Hegarty, M., & Mayer, R. E. (1999). Students’ use of imagery in solving qualitative problems in kinematics. Washington, DC: U.S Department of Education. (ERIC Document Reproduction Service No. ED433239).Google Scholar
  21. Kyttälä, M. (2008). Visuospatial working memory in adolescents with poor performance in mathematics: Variation depending on reading skills. Educational Psychology: An International Journal of Experimental Educational Psychology, 28(3), 273–289.CrossRefGoogle Scholar
  22. Lowrie, T., & Diezmann, C. M. (2007). Solving graphics problems: Student performance in the junior grades. Journal of Educational Research, 100(6), 369–377.CrossRefGoogle Scholar
  23. MacDonald, A. S. (2002). The impact of individual differences on the equivalence of computer-based and paper-and-pencil educational assessments. Computers and Education, 39, 299–312.CrossRefGoogle Scholar
  24. Marginson, S., Tytler, R., Freeman, B., & Roberts, K. (2013). STEM: Country comparisons. Report for the Australian Council of Learned Academies. Melbourne, Vic: Australian Council of Learned Academies. Retrieved 17 Feb 2015 from www.acola.org.au.
  25. Mayer, R. E., & Massa, L. J. (2003). Three facets of visual and verbal learners: cognitive ability, cognitive style, and learning preference. Journal of Educational Psychology, 95(4), 833–846.CrossRefGoogle Scholar
  26. Mullis, I. V. S., Martin, M. O., Ruddock, G. J., O’Sullivan, C. Y., & Preuschoff, C. (2009). Chestnut Hill, MA: TIMSS & PIRLS International Study Center. International Association for the: Evaluation of Educational Achievement (IEA). TIMSS 2011 Assessment Frameworks.Google Scholar
  27. Mullis, I. V. S., Martin, M. O., Foy, P., & Arora, A. (2012). TIMSS 2011 International results in mathematics. Chestnut Hill, MA: TIMSS & PIRLS International Study Center, International Association for the Evaluation of Educational Achievement (IEA).Google Scholar
  28. Ng, S. F., & Lee, K. (2009). The model method: Singapore children’s tool for representing and solving algebraic word problems. Journal for Research in Mathematics Education, 40(3), 282–313.Google Scholar
  29. Pittalis, M., & Christou, C. (2010). Types of reasoning in 3D geometry thinking and their relation with spatial ability. Educational Studies in Mathematics, 75, 191–212. doi: 10.1007/s10649-010-9251-8.CrossRefGoogle Scholar
  30. Reuhkala, M. (2001). Mathematical skills in ninth-graders: relationship with visuo-spatial abilities and working memory. Educational Psychology: An International Journal of Experimental Educational Psychology, 21(4), 387–399. doi: 10.1080/01443410120090786.CrossRefGoogle Scholar
  31. Rohde, T. E., & Thompson, L. A. (2007). Predicting academic achievement with cognitive ability. Intelligence, 35, 83–92. doi: 10.1016/j.intell.2006.05.004.CrossRefGoogle Scholar
  32. Shrout, P. E., & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86(2), 420–428.CrossRefGoogle Scholar
  33. Skemp, R. R. (1986). The psychology of learning mathematics. London: Penguin.Google Scholar
  34. Sweller, J. (1994). Cognitive load theory, learning difficulty, and instructional design. Learning and Instruction, 4, 295–312.CrossRefGoogle Scholar
  35. Threlfall, J., Pool, P., Homer, M., & Swinnerton, B. (2007). Implicit aspects of paper and pencil mathematics assessment that come to light through the use of the computer. Educational Studies in Mathematics, 66, 335–348. doi: 10.1007/s10649-006-9078-5.CrossRefGoogle Scholar
  36. Tolar, T. D., Lederberg, A. R., & Fletcher, J. M. (2009). A structural model of algebra achievement: computational fluency and spatial visualisation as mediators of the effect of working memory on algebra achievement. Educational Psychology: An International Journal of Experimental Educational Psychology, 29(2), 239–266. doi: 10.1080/01443410802708903.CrossRefGoogle Scholar
  37. Tversky, B. (2004). Visuospatial reasoning. In K. Holyoak & R. G. Morrison (Eds.), The Cambridge handbook of thinking and reasoning (pp. 209–240). New York: Cambridge University Press.Google Scholar
  38. Uttal, D. H., & Cohen, C. A. (2012). Spatial thinking in STEM education: when, why and how. In B. H. Ross (Ed.), The psychology of learning and motivation (Vol. 57, pp. 147–182). San Diego: Academic.CrossRefGoogle Scholar
  39. Wang, S., Jiao, H., Young, M. J., Brooks, T., & Olson, J. (2008). Comparability of computer-based and paper-and-pencil testing in k–12 reading assessments: A meta-analysis of testing mode effects. Educational and Psychological Measurement, 68(1), 5–24.Google Scholar
  40. Zhang, X., Koponen, T., Räsänen, P., Aunola, K., Lerkkanen, M.-K., & Nurmi, J.-E. (2014). Linguistic and spatial skills predict early arithmetic development via counting sequence knowledge. Child Development, 85(3), 1091–1107. doi: 10.1111/cdev.12173.CrossRefGoogle Scholar

Copyright information

© Mathematics Education Research Group of Australasia, Inc. 2015

Authors and Affiliations

  1. 1.Faculty of Education, Science, Technology and MathematicsUniversity of CanberraCanberraAustralia

Personalised recommendations

Cite article

Buy options