Mathematical Problem Solving pp 231-260 | Cite as

# Toward Designing and Developing Likert Items to Assess Mathematical Problem Solving

## Abstract

Access to science, technology, engineering, and mathematics (STEM) careers relies heavily on student success in their foundational university mathematics courses. For future STEM majors, there is little debate over the procedural skills that are needed in algebra to continue in STEM, but what are the mathematical problem solving (MPS) skills needed to persist in calculus and beyond? There are currently few efficient tools that can be used to explore research questions such as this one. In response, our project goal is to develop Likert MPS items that can be machine scored and used as an efficient tool in exploring students’ MPS practices. The items link a student’s MPS to five key domains of MPS derived from the research literature; items are designed for entry-level university students and do not require content knowledge beyond secondary school algebra. In this chapter, we report on the Likert item development, including the revision and validation processes. As part of the development, we explored the following research questions: (a) How are student scores on MPS items linked to their MPS practices during interviews, and (b) How are student scores on MPS items linked to their course performance? Participants were over 1500 students enrolled in College Algebra or Calculus at a large, urban university in the United States. Results illustrated trends in student practices within each MPS domain; these trends are informative for future item development and research. Results also suggest that students may not be developing many MPS skills in their foundational university mathematics courses.

## Keywords

Mathematical problem solving Mathematics assessment College algebra Calculus placement## Notes

### Acknowledgements

Partial support for this work was provided by the National Science Foundation (NSF) Improving Undergraduate STEM Education (IUSE) program under Award No. 1544545. Any opinions, findings, and conclusions or recommendations expressed in this material are those of the authors and do not necessarily reflect the views of the NSF. We also thank Dr. Lauren Coursey and Andrew Kercher for research assistance and reviewers for their feedback.

## References

- American Educational Research Association, American Psychological Association, National Council on Measurement in Education. (1999).
*Standards for educational and psychological testing*. Washington, DC: American Psychological Association.Google Scholar - Bryk, A. S., & Treisman, U. (2010). Make math a gateway, not a gatekeeper.
*Chronicle of Higher Education,**56*(32), B19–B20.Google Scholar - Cai, J., Wang, N., Moyer, J. C., Wang, C., & Nie, B. (2011). Longitudinal investigation of the curriculum effect: An analysis of student learning outcomes from the LieCal project.
*International Journal of Educational Research,**50,*117–136.CrossRefGoogle Scholar - Campbell, R. C. (2014).
*Reconciling secondary inservice teachers’ definitions of problem solving with those found in the research literature*(Unpublished master’s thesis). The University of Texas at Arlington, Arlington, TX.Google Scholar - Campbell, R. C. (2017, January).
*Mathematical problem solving practices: A comparison of a student in college algebra to a student in calculus.*Presentation at the 2017 Joint Meetings of the American Mathematical Society and the Mathematical Association of America, Atlanta, GA.Google Scholar - Carlson, M. P., & Bloom, I. (2005). The cyclic nature of problem solving: An emergent multidimensional problem solving framework.
*Educational Studies in Mathematics,**58,*45–75.CrossRefGoogle Scholar - Carlson, M., Oehrtman, M., & Engelke, N. (2010). The precalculus concept assessment: A tool for assessing students’ reasoning abilities and understandings.
*Cognition and Instruction,**28*(2), 113–145.CrossRefGoogle Scholar - Chapman, O. (1999). Inservice teacher development in mathematical problem solving.
*Journal for Mathematics Teacher Education,**2,*121–142.CrossRefGoogle Scholar - Charles A. Dana Center. (2007).
*Algebra II assessments: Performance tasks aligned to the Texas standards*. Austin, TX: Author.Google Scholar - Dawkins, P. C., & Epperson, J. A. M. (2014). The development and nature of problem-solving among first-semester calculus students.
*International Journal of Mathematical Education in Science and Technology,**45*(6), 839–862.CrossRefGoogle Scholar - Eisenberg, T., & Dreyfus, T. (1991). On the reluctance to visualize in mathematics. In W. Zimmermann & S. Cunningham (Eds.),
*Visualization in teaching and learning mathematics*(pp. 25–37). Washington, DC: Mathematical Association of America.Google Scholar - Engle, R. A. (2012). The productive disciplinary engagement framework: Origins, key concepts, and continuing developments. In D. Y. Dai (Ed.),
*Design research on learning and thinking in educational settings: Enhancing intellectual growth and functioning*(pp. 161–200). London: Taylor & Francis.Google Scholar - Engle, R. A., & Conant, F. R. (2002). Guiding principles for fostering productive disciplinary engagement: Explaining an emergent argument in a community of learners classroom.
*Cognition and Instruction,**20*(4), 399–483.CrossRefGoogle Scholar - Fike, D. S., & Fike, R. (2008). Mathematics: Gatekeeper for Hispanic success in health care professions.
*Hispanic Health Care International,**6*(4), 185–191.CrossRefGoogle Scholar - Fink, A. (2013).
*How to conduct surveys: A step-by-step guide*(5th ed.). Los Angeles, CA: SAGE Publications.Google Scholar - Garofalo, J., & Lester, F. K. (1985). Metacognition, cognitive monitoring, and mathematical performance.
*Journal for Research in Mathematics Education,**16*(3), 163–176.CrossRefGoogle Scholar - Gonzales, G. M. (2017).
*Investigating undergraduates’ mathematical problem solving: Alignment between interview practices and likert assessments*(Unpublished master’s thesis). The University of Texas at Arlington, Arlington, TX.Google Scholar - Green, P. (2016).
*An analysis of a University’s College Algebra curriculum: Searching for evidence of mathematical problem solving*(Unpublished master’s thesis). The University of Texas at Arlington, Arlington, TX.Google Scholar - Jarrett, E. (2000).
*Evaluating the persistence and performance of ‘successful’ precalculus students in subsequent mathematics courses*(Master’s thesis). Lubbock, TX: Texas Tech University.Google Scholar - Jonassen, D. H. (1997). Instructional design models for well-structured and ill-structured problem-solving learning outcomes.
*Educational Technology Research and Development,**45*(1), 65–94.CrossRefGoogle Scholar - Kieran, C. (2007). Learning and teaching algebra at the middle school through college levels: Building meaning for symbols and their manipulation. In F. K. Lester (Ed.),
*Second handbook of research on mathematics teaching and learning*(Vol. 1–2, pp. 707–762). Charlotte, NC: Information Age Publishing.Google Scholar - Kubiszyn, T., & Borich, G. (2003).
*Educational testing and measurement: Classroom application and practice*(7th ed.). New York, NY: Wiley.Google Scholar - Kwek, M. L. (2015). Using problem posing as a formative assessment tool. In F. M. Singer, N. F. Ellerton, & J. Cai (Eds.),
*Mathematical problem posing from research to effective practice*(pp. 273–292). New York, NY: Springer.CrossRefGoogle Scholar - Lappan, G., & Phillips, E. (1998). Teaching and learning in the connected mathematics project. In L. Leutzinger (Ed.),
*Mathematics in the middle*(pp. 83–92). Reston, VA: National Council of Teachers of Mathematics.Google Scholar - Lester, F. (2013). Thoughts about research on mathematical problem-solving instruction.
*The Mathematics Enthusiast,**10*(1&2), 245–278.Google Scholar - Lester, F., & Cai, J. (2014). Can mathematical problem solving be taught? Preliminary answers from 30 years of research. In P. Felmer, E. Pehkonen, & J. Kilpatrick (Eds.),
*Posing and solving mathematical problems advances and new perspectives*(pp. 117–135). New York, NY: Springer.Google Scholar - Maciejewski, W., & Merchant, S. (2016). Mathematical tasks, study approaches, and course grades in undergraduate mathematics: A year-by-year analysis.
*International Journal of Mathematical Education in Science and Technology,**47*(3), 373–387.CrossRefGoogle Scholar - Miles, M. B., & Huberman, A. M. (1994).
*Qualitative data analysis*(2nd ed.). Thousand Oaks, CA: SAGE Publications.Google Scholar - National Council of Teachers of Mathematics. (2000).
*Principles and standards for school mathematics*[Online version]. Retrieved from http://www.nctm.org/standards/content.aspx?id=26792. - National Governors Association Center for Best Practices & Council of Chief State School Officers. (2010).
*Common core state standards for mathematics*. Washington, DC: Author.Google Scholar - Oregon Department of Education, Office of Assessment and Evaluation. (2000).
*Mathematical problem solving scoring guide*. Salem, OR: Author.Google Scholar - Pearson Education Inc. (2010).
*Partnership for assessment of readiness for college and careers mathematics practice tests.*Retrieved from http://parcc.pearson.com/practice-tests/math/. - Peters, H. (2017).
*Mathematical problem solving in a College Algebra Course curriculum*(Unpublished master’s thesis). The University of Texas at Arlington, Arlington, TX.Google Scholar - Phan, D. V. (2017).
*A snapshot of the alignment of University Students’ mathematical problem solving practices to a Likert-scale assessment of mathematical problem solving*(Master’s thesis). The University of Texas at Arlington, Arlington, TX.Google Scholar - Pineda, E. Y. (2016).
*Mathematical problem solving: Exploring undergraduate students’ written work and self-rated use of problem-solving domains*(Unpublished master’s thesis). The University of Texas at Arlington, Arlington, TX.Google Scholar - Polya, G. (1957).
*How to solve it*. Princeton, NJ: Princeton University Press.Google Scholar - Presmeg, N. C. (2006). Research on visualization in learning and teaching in mathematics: Emergence from psychology. In A. Gutierrez & P. Boero (Eds.),
*Handbook of research on the psychology of mathematics education: Past, present and future*(pp. 205–235). Rotterdam, The Netherlands: Sense Publishers.Google Scholar - Pugalee, D. (2004). A comparison of verbal and written descriptions of students’ problem solving processes.
*Educational Studies in Mathematics,**55,*27–47.CrossRefGoogle Scholar - Rhoads, K., Epperson, J. A. M., & Campbell, R. C. (2017a). Undergraduates representing and connecting in mathematical problem solving. In T. A. Olson & L. Venenciano (Eds.),
*Proceedings of the 44th Annual Meeting of the Research Council on Mathematics Learning*(pp. 129–136). Fort Worth, TX: RCML.Google Scholar - Rhoads, K., Epperson, J. A. M., & Campbell, R. C. (2017b, January).
*The role of justifying in entry-level undergraduates’ mathematical problem solving*. Presentation at the 2017 Joint Meetings of the American Mathematical Society and the Mathematical Association of America, Atlanta, GA.Google Scholar - Santos-Trigo, M. (1998). Instructional qualities of a successful mathematical problem-solving class.
*International Journal for Math, Science, and Technology,**29*(5), 631–646.CrossRefGoogle Scholar - Santos-Trigo, M. (2014). Problem solving in mathematics education. In S. Lerman (Ed.),
*Encyclopedia of mathematics education*(pp. 496–501). The Netherlands: Springer.Google Scholar - Schoenfeld, A. H. (1985).
*Mathematical problem solving*. Orlando, FL: Academic Press.Google Scholar - Schoenfeld, A. H. (1988). Problem solving in context(s). In R. Charles & E. Silver (Eds.),
*The teaching and assessing of mathematical problem solving*(pp. 82–92). Reston, VA: National Council of Teachers of Mathematics.Google Scholar - Schoenfeld, A. H. (1992). Learning to think mathematically: Problem solving, metacognition, and sense-making in mathematics. In D. Grouws (Ed.),
*Handbook for research on mathematics teaching and learning*(pp. 334–370). New York: MacMillan.Google Scholar - Schoenfeld, A. H. (2013). Reflections on problem solving theory and practice.
*The Mathematics Enthusiast,**10*(1), 9–34.Google Scholar - Schoenfeld, A. H. (2014). What makes for powerful classrooms, and how can we support teachers in creating them? A story of research and practice.
*Productively Intertwined. Educational Researcher,**43*(8), 404–412.CrossRefGoogle Scholar - Selden, A., Selden, J., Hauk, S., & Mason, A. (2000). Why can’t calculus students access their knowledge to solve non-routine problems.
*CBMS Issues in Mathematics Education,**8,*128–153.CrossRefGoogle Scholar - Silver, E. A. (1985).
*Teaching and learning mathematical problem solving: Multiple research perspectives*. Hillsdale, NJ: Lawrence Erlbaum Associates.Google Scholar - Silver, E., & Cai, J. (1996). An analysis of arithmetic problem posing by middle school students.
*Journal for Research in Mathematics Education,**27,*521–539.CrossRefGoogle Scholar - Smarter Balanced Assessment Consortium. (2014).
*Practice and training tests*. Retrieved from http://www.smarterbalanced.org/practice-test/. - Szetela, W., & Nicol, C. (2002). Evaluating problem solving in mathematics.
*Educational Leadership,**10,*42–45.Google Scholar - Texas Education Agency. (2011).
*STAAR released test questions*. Retrieved from http://tea.texas.gov/Student_Testing_and_Accountability/Testing/State_of_Texas_Assessments_of_Academic_Readiness_(STAAR)/STAAR_Released_Test_Questions/. - Texas Education Agency. (2012).
*Texas essential knowledge and skills for mathematics*. Retrieved from http://ritter.tea.state.tx.us/rules/tac/chapter111/ch111c.html. - Turner, J. M. (2017).
*A tale of two techniques: A case study of mathematical problem-solving practices in college algebra students*(Unpublished master’s thesis). The University of Texas at Arlington: Arlington, TX.Google Scholar - Van Harpen, X., & Presmeg, N. (2015). An investigation of high school students’ mathematical problem posing in the United States and China. In F. M. Singer, N. F. Ellerton, & J. Cai (Eds.),
*Mathematical problem posing from research to effective practice*(pp. 293–308). New York, NY: Springer.CrossRefGoogle Scholar - Wilson, J. W., Fernandez, M. L., & Hadaway, N. (1993). Mathematical problem solving. In P. S. Wilson (Ed.),
*Research ideas for the classroom: High school mathematics*(pp. 57–78). New York, NY: MacMillan.Google Scholar - Yee, S. P., & Bostic, J. D. (2014). Developing a contextualization of students’ mathematical problem solving.
*The Journal of Mathematical Behavior,**36,*1–19.CrossRefGoogle Scholar - Yin, R. K. (2009).
*Case study research: Design and methods*. Thousand Oaks, CA: Sage.Google Scholar