Skip to main content

Advertisement

SpringerLink
Log in

Computer-based assessment of mathematics into the twenty-first century: pressures and tensions

  • Original Article
  • Published:
ZDM Aims and scope Submit manuscript

Abstract

In recent decades, technology has influenced various aspects of assessment in mathematics education: (1) supporting the assessment of higher-order thinking skills in mathematics, (2) representing authentic problems from the world around us to use and apply mathematical knowledge and skills, and (3) making the delivery of tests and the analysis of results through psychometric analysis more sophisticated. We argue that these developments are not pushing mathematics education in the same direction, however, which creates tensions. Mathematics education—so essential for educating young people to be creative and problem solving agents in the twenty-first century—is at risk of focusing too much on assessment of lower order goals, such as the reproduction of procedural, calculation based, knowledge and skills. While there is an availability of an increasing amount of sophisticated technology, the related advances in measurement, creation and delivery of automated assessments of mathematics are however being based on sequences of atomised test items. In this article several aspects of the use of technology in the assessment of mathematics education are exemplified and discussed, including in relation to the aforementioned tension. A way forward is suggested by the introduction of a framework for the categorisation of mathematical problem situations with an increasing sophistication of representing the problem situation using various aspects of technology. The framework could be used to reflect on and discuss mathematical assessment tasks, especially in relation to twenty-first century skills.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1
Fig. 2

Similar content being viewed by others

References

  • Ashton, H. S., Beevers, C. E., Korabinski, A. A., & Youngson, M. A. (2006). Incorporating partial credit in computer-aided assessment of Mathematics in secondary education. British Journal of Educational Technology, 37(1), 93–119. https://doi.org/10.1111/j.1467-8535.2005.00512.x.

    Article  Google Scholar 

  • Australian Association of Mathematics Teachers, & Australian Industry-Group. (2014). Identifying and Supporting Quantitative Skills of 21st Century Workers—Final Report. Retrieved from http://www.chiefscientist.gov.au/wp-content/uploads/Quantitative-Skills-of-21st-Century-Workers-Report.pdf.

  • Baird, J. -A., Andrich, D., Hopfenbeck, T. N., & Stobart, G. (2017a). Assessment and learning: fields apart? Assessment in Education: Principles, Policy & Practice, 24(3), 317–350. https://doi.org/10.1080/0969594X.2017.1319337.

    Article  Google Scholar 

  • Baird, J. -A., Andrich, D., Hopfenbeck, T. N., & Stobart, G. (2017b). Metrology of education. Assessment in Education: Principles, Policy & Practice, 24(3), 463–470. https://doi.org/10.1080/0969594X.2017.1337628.

    Article  Google Scholar 

  • Baker Dearing Educational Trust. (2017). From school work to real work: how education fails students in the real world. London: Baker Dearing Educational Trust.

    Google Scholar 

  • Bardini, C. (2015). Computer-based assessment of mathematics in PISA 2012. In K. Stacey & R. Turner (Eds.), Assessing mathematical literacy: The PISA experience (pp. 173–188). New York: Springer.

    Google Scholar 

  • Bennett, R. E. (1998). Reinventing assessment: Speculations on the future of large-scale educational testing. Princeton, NJ: ETS.

    Google Scholar 

  • Bennett, R. E. (2010a). Cognitively based assessment of, for, and as learning: a preliminary theory of action for summative and formative assessment. Measurement: Interdisciplinary Research and Perspectives, 8(91), 70–91.

    Google Scholar 

  • Bennett, R. E. (2010b). Technology for large-scale assessment. In P. Peterson, E. Baker & B. McGaw (Eds.), International encyclopedia of education (vol. 8, (3rd ed.), pp. 48–55). Oxford: Elsevier.

    Chapter  Google Scholar 

  • Bennett, R. E. (2015). The changing nature of educational assessment. Review of Research in Education, 39(1), 370–407. https://doi.org/10.3102/0091732X14554179.

    Article  Google Scholar 

  • Bennett, R. E., Persky, H., Weiss, A., & Jenkins, F. (2010). Measuring problem solving with technology: A demonstration study for NAEP. Journal of Technology, Learning, and Assessment, 8(8). Retrieved from https://ejournals.bc.edu/ojs/index.php/jtla/issue/view/166.

  • Biesta, G. J. J. (2010). What is education for? Good education in an age of measurement: Ethics, politics, democracy. London: Taylor & Francis.

    Google Scholar 

  • Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74. https://doi.org/10.1080/0969595980050102.

    Article  Google Scholar 

  • Bokhove, C., & Drijvers, P. (2010). Digital tools for algebra education: criteria and evaluation. International Journal of Computers for Mathematical Learning, 15(1), 45–62. https://doi.org/10.1007/s10758-010-9162-x.

    Article  Google Scholar 

  • Bower, M., Howe, C., McCredie, N., Robinson, A., & Grover, D. (2014). Augmented reality in education—cases, places and potentials. Educational Media International, 51(1), 1–15. https://doi.org/10.1080/09523987.2014.889400.

    Article  Google Scholar 

  • Burkhardt, H., & Schoenfeld, A. (2018). Assessment in the service of learning: Challenges and opportunities or Plus ça Change, Plus c’est la même Chose. ZDM. https://doi.org/10.1007/s11858-018-0937-1.

    Google Scholar 

  • Coben, D., Hall, C., Hutton, M., Rowe, D., Weeks, K., & Wolley, N. (2010). Benchmark assessment of numeracy for nursing: Medication dosage calculation at point of registration. Edinburgh: NHS Education for Scotland.

    Google Scholar 

  • Corcoran, T., Mosher, F. A., & Rogat, A. (2009). Learning Progressions in Science: An Evidence-based Approach to Reform (Research Report #RR-63). Retrieved from Philadelphia, PA: http://www.cpre.org/sites/default/files/researchreport/829_lpsciencerr63.pdf.

  • Csapó, B., Molnár, G., & Tóth, K. R. (2009). Comparing paper-and-pencil and online assessment of reasoning skills. A pilot study for introducing electronic testing in large-scale assessment in Hungary. In F. Scheuermann & J. Björnsson (Eds.), The transition to computer-based assessment. New approaches to skills assessment and implications for large-scale testing. Luxemburg: Office for Official Publications of the European Communities.

    Google Scholar 

  • Csapó, B., Ainley, J., Bennett, R. E., Latour, T., & Law, N. (2012). Technological Issues for computer-based assessment. In P. Griffin, B. McGaw & E. Care (Eds.), Assessment and teaching of 21st century skills (pp. 143–230). Dordrecht: Springer.

    Chapter  Google Scholar 

  • Daro, P., Mosher, F. A., & Corcoran, T. (2011). Learning trajectories in mathematics: A foundation for standards, curriculum, assessment, and instruction. CPRE Research Report #RR-68. Philadelphia: Consortium for Policy Research in Education. https://doi.org/10.12698/cpre.2011.rr68.

    Google Scholar 

  • Davis, R., Maher, C., & Noddings, N. (Eds.). (1990). Constructivist views on the teaching and learning of mathematics. Reston, VA: National Council of Teachers of Mathematics.

    Google Scholar 

  • European Commission. (2017). Commission Staff Working Document, Accompanying the document Communication from the Commission to the European Parliament, the Council, the European Economic and Social Committee and the Committee of the Regions A renewed EU agenda for higher education SWD(2017)264 Retrieved from https://ec.europa.eu/education/sites/education/files/he-swd-2017-165_en.pdf.

  • Forman, S. L., & Steen, L. A. (1999). Beyond eighth grade: Functional mathematics for life and work. Berkeley, CA: National Centre for Research in Vocational Education.

    Google Scholar 

  • Geiger, V., Goos, M., & Forgasz, H. (2015). A rich interpretation of numeracy for the 21st century: A survey of the state of the field. ZDM—Mathematics Education, 47(4), 531–548. https://doi.org/10.1007/s11858-015-0708-1.

    Google Scholar 

  • Gravemeijer, K., Stephan, M., Julie, C., Lin, F.-L., & Ohtani, M. (2017). What mathematics education may prepare students for the society of the future? International Journal of Science and Mathematics Education. https://doi.org/10.1007/s10763-017-9814-6.

    Google Scholar 

  • Greiff, S., & Kyllonen, P. (2016). Contemporary assessment challenges: The measurement of 21st century skills. Applied Measurement in Education, 29(4), 243–244. https://doi.org/10.1080/08957347.2016.1209209.

    Article  Google Scholar 

  • Griffin, P., & Care, E. (2015). Assessment and teaching of 21st Century Skills—methods and approach. New York: Springer.

    Google Scholar 

  • Heritage, M. (2008). Learning Progression: Supporting instruction and formative assessment. Retrieved from Los Angeles, CA: http://www.ccsso.org/Documents/2008/Learning_Progressions_Supporting_2008.pdf.

  • Hoogland, K. (2016). Images of numeracy: Investigating effects of visual representations of problem situations in contextual mathematical problem solving. (PhD-thesis). Eindhoven: Technical University Eindhoven.

    Google Scholar 

  • Hoogland, K., Pepin, B., Bakker, A., de Koning, J., & Gravemeijer, K. (2016). Representing contextual mathematical problems in descriptive or depictive form: Design of an instrument and validation of its uses. Studies in Educational Evaluation, 50, 22–32. https://doi.org/10.1016/j.stueduc.2016.06.005.

    Article  Google Scholar 

  • Hoogland, K., Pepin, B., de Koning, J., Bakker, A., & Gravemeijer, K. (2018). Word problems versus image-rich problems: an analysis of effects of task characteristics on students’ performance on contextual mathematics problems. Research in Mathematics Education, 20(1), 37–52. https://doi.org/10.1080/14794802.2017.1413414.

    Article  Google Scholar 

  • Hoyles, C., Noss, R., Kent, P., & Bakker, A. (2010). Improving Mathematics at Work The Need for Techno-Mathematical Literacies. London and New York: Routledge.

    Google Scholar 

  • Hoyles, C., Wolf, A., Molyneux-Hodgson, S., & Kent, P. (2002). Mathematical skills in the workplace. Retrieved from London, UK: http://eprints.ioe.ac.uk/1565/1/Hoyles2002MathematicalSkills.pdf.

  • Kent, P., Bakker, A., Hoyles, C., & Noss, R. (2011). Measurement in the workplace: the case of process improvement in manufacturing industry. ZDM—The International Journal on Mathematics Education, 43(5), 747. https://doi.org/10.1007/s11858-011-0359-9.

    Article  Google Scholar 

  • Leighton, J. P., & Gierl, M. J. (2007). Cognitive diagnostic assessment for education: Theory and applications. New York, NY: Cambridge University Press.

    Book  Google Scholar 

  • Livne, N. L., Livne, O. E., & Wight, C. A. (2007). Can automated scoring surpass hand grading of students’ constructed responses and error patterns in mathematics? MERLOT Journal of Online Learning and Teaching, 3(3), 295–306.

    Google Scholar 

  • Madison, B. L., & Steen, L. A. (2003). Quantitative Literacy: Why Numeracy Matters for Schools and Colleges. Princeton, NJ: National Council on Education and the Disciplines.

    Google Scholar 

  • Masters, G. N. (2013). Reforming Educational Assessment: Imperatives, principles and challenges—Australian education review; no. 57. Camberwell: ACER.

    Google Scholar 

  • Ministry of Education Singapore—Curriculum Planning and Development Division. (2012). Mathematics Syllabus: Primary One to Five. Retrieved from https://www.moe.gov.sg/docs/default-source/document/education/syllabuses/sciences/files/primary_mathematics_syllabus_pri1_to_pri5.pdf.

  • Mullis, I. V. S., & Martin, M. O. (Eds.). (2013). Timss 2015 Assessment Frameworks. Chestnut Hill, MA: TIMSS & PIRLS International Study Center.

    Google Scholar 

  • National Council of Teachers of Mathematics (NCTM). (2000). Principles and standards for school mathematics Retrieved from Reston, VA: http://www.nctm.org/Standards-and-Positions/Principles-and-Standards.

  • National Council of Teachers of Mathematics (NCTM). (2017). Catalyzing Change in High School Mathematics. Retrieved from Reston, VA: https://www.nctm.org/uploadedFiles/Standards_and_Positions/CatalyzingChangePublicReview.pdf.

  • OECD. (2013). PISA 2012 results: Excellence through equity. Giving every student the chance to succeed (volume II). Paris: OECD Publishing.

    Book  Google Scholar 

  • OECD. (2014). PISA 2012 results: Creative problem solving. Students’ skills in tackling real-life problems (volume V). Paris: OECD Publishing.

    Book  Google Scholar 

  • OECD. (2016). Global competency for an inclusive world. Paris: OECD Publishing.

    Google Scholar 

  • OECD. (2017). PISA 2015 assessment and analytical framework: Mathematics, reading, science, problem solving and financial literacy. Retrieved from Paris, France: http://www.oecd.org/pisa/pisaproducts/PISA%202012%20framework%20e-book_final.pdf.

  • Palm, T. (2006). Word problems as simulations of real-world situations: A proposed framework. For the Learning of Mathematics, 26(1), 42–47. https://doi.org/10.2307/40248523.

    Google Scholar 

  • Palm, T. (2009). Theory of Authentic Task Situations. In L. Verschaffel, B. Greer, W. V. Dooren & S. Mukhopadhyay (Eds.), Words and worlds—modelling verbal descriptions of situations (pp. 3–20). Rotterdam: Sense.

    Google Scholar 

  • Parshall, C. G., Spray, J. A., Kalohn, J. C., & Davey, T. (2002). Practical considerations in computer-based testing. New York, NY: Springer.

    Book  Google Scholar 

  • Partnership for 21st Century Skills. (2016). Framework for 21st Century Learning. Retrieved from Washington, DC: http://www.p21.org/storage/documents/docs/P21_framework_0816.pdf.

  • Pead, D. A. (2010). On Computer-based Assessment of Mathematics (Ph.D.-thesis). Nottingham: University of Nottingham.

  • PIAAC Numeracy Expert Group. (2009). PIAAC Numeracy: A conceptual framework. Retrieved from Paris, France: http://www.oecd.org/officialdocuments/publicdisplaydocumentpdf/?cote=EDU/WKP(2009)14&doclanguage=en.

  • Poggio, J., Glasnapp, D. R., Yang, X., & Poggio, A. J. (2005). A Comparative evaluation of score results from computerized and paper-and-pencil mathematics testing in a large scale state assessment program. The Journal of Technology, Learning, and Assessment, 3(6). http://www.jtla.org.

  • PwC. (2015). A smart move: Future-proofing Australia’s workforce by growing skills in science, technology, engineering and maths. Retrieved from Sydney Australia: https://www.pwc.com.au/pdf/a-smart-move-pwc-stem-report-april-2015.pdf.

  • Schoenfeld, A. H. (2017). On learning and assessment. Assessment in Education: Principles, Policy & Practice, 24(3), 369–378. https://doi.org/10.1080/0969594X.2017.1336986.

    Article  Google Scholar 

  • Schwab, K. (2016). The Fourth Industrial Revolution: what it means, how to respond. Retrieved from https://www.weforum.org/agenda/2016/01/the-fourth-industrial-revolution-what-it-means-and-how-to-respond/.

  • Shute, V. J., Leighton, J. P., Jang, E. E., & Chu, M.-W. (2016). Advances in the Science of Assessment. Educational Assessment, 21(1), 34–59. https://doi.org/10.1080/10627197.2015.1127752.

    Article  Google Scholar 

  • Stacey, K., & Wiliam, D. (2013). Technology and Assessment in Mathematics. In M. A. Clements, A. J. Bishop, C. Keitel, J. Kilpatrick & F. K. S. Leung (Eds.), Third international handbook of mathematics education (pp. 721–752). New York: Springer.

    Google Scholar 

  • Steen, L. A. (2003). Data, shapes, symbols: Achieving balance in school mathematics. In B. L. Madison & L. A. Steen (Eds.), Quantitative literacy: Why numeracy matters for schools and colleges. Princeton, NJ: National Council on Education and the Disciplines.

    Google Scholar 

  • Straesser, R. (2015). Numeracy at work”: A discussion of terms and results from empirical studies. ZDM—Mathematics Education, 47(4), 665–674. https://doi.org/10.1007/s11858-015-0689-0.

    Google Scholar 

  • Taubman, P. M. (2009). Teaching By Numbers: Deconstructing the Discourse of Standards and Accountability in Education. New York: Routledge.

    Google Scholar 

  • Tout, D., Coben, D., Geiger, V., Ginsburg, L., Hoogland, K., Maguire, T., et al. (2017). Review of the PIAAC numeracy assessment framework: Final report. Camberwell, Australia: Australian Council for Educational Research (ACER).

    Google Scholar 

  • Tout, D., & Spithill, J. (2015). The challenges and complexities of writing items to test mathematical literacy. In K. Stacey & R. Turner (Eds.), Assessing mathematical literacy: The PISA experience (pp. 145–171). Basel: Springer International Publishing.

    Google Scholar 

  • Verschaffel, L., Greer, B., Dooren, W. Van, & Mukhopadhyay, S. (Eds.). (2009). Words and worlds - Modelling verbal descriptions of situations. Rotterdam: Sense.

    Google Scholar 

  • Voogt, J., & Pareja Roblin, N. N. (2012). A comparative analysis of international frameworks for 21st century competences: Implications for national curriculum policies. Journal of Curriculum Studies, 44(3), 299–321. https://doi.org/10.1080/00220272.2012.668938.

    Article  Google Scholar 

  • Wake, G. (2015). Preparing for workplace numeracy: A modelling perspective. ZDM—Mathematics Education, 47(4), 675–689. https://doi.org/10.1007/s11858-015-0704-5.

    Google Scholar 

  • Wiliam, D. (2005). Assessment for learning: Why no profile in US policy? In J. Gardner (Ed.), Assessment and learning (pp. 169–183). London: Sage.

    Google Scholar 

  • Wiliam, D. (2011). What is assessment for learning? Studies in Educational Evaluation, 37(1), 3–14. https://doi.org/10.1016/j.stueduc.2011.03.001.

    Article  Google Scholar 

  • Wiliam, D., & Leahy, S. (2015). Embedding formative assessment: Practical techniques for K-12 classrooms. West Palm Beach, FL: Learning Sciences International.

    Google Scholar 

  • Zevenbergen, R. (2004). Technologizing numeracy: intergenerational differences in working mathematically in New Times. Educational Studies in Mathematics, 56(1), 97–117. https://doi.org/10.1023/b:educ.0000028399.76056.91.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. HU University of Applied Sciences Utrecht, Utrecht, The Netherlands

    Kees Hoogland

  2. Australian Council for Educational Research, Melbourne, Australia

    Dave Tout

Authors
  1. Kees Hoogland
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Dave Tout
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Kees Hoogland.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Hoogland, K., Tout, D. Computer-based assessment of mathematics into the twenty-first century: pressures and tensions. ZDM Mathematics Education 50, 675–686 (2018). https://doi.org/10.1007/s11858-018-0944-2

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11858-018-0944-2

Keywords

Search

Navigation