Advertisement

Threshold Concepts for Modeling and Assessing Higher Education Students’ Understanding and Learning in Economics

  • Sebastian BrücknerEmail author
  • Olga Zlatkin-Troitschanskaia
Chapter
Part of the Methodology of Educational Measurement and Assessment book series (MEMA)

Abstract

In the last decade, the research carried out on threshold concepts as a content-based way to model students’ understanding and learning in several domains has increased. However, empirical evidence on this approach is still scarce. In this chapter, the authors investigate the adequacy of the threshold concepts approach in the domain of business and economics in higher education following an established differentiation between basic, discipline, and modeling thresholds. After conducting a cognitive interview study using verbal reports, a self-assessment questionnaire was used to assess the respondents’ familiarity with the content and their security to solve the tasks. Results indicate that there is a complex relation between students’ response processes, self-assessment, and test scores, which varies according to the different thresholds and that all three measures generally confirm our hypotheses yet have to be critically discussed. There are implications that test developers, test users, respondents, and other stakeholders should be aware of this complex relation; it affirms that the threshold concepts approach is at least a useful tool when conceptualizing and developing tests, which can be considered to be an addition to classic taxonomies of educational objectives.

References

  1. Allgood, S., & Bayer, A. (2016). Measuring college learning in economics. In R. Arum, J. Roksa, & A. Cook (Eds.), Improving quality in American higher education: Learning outcomes and assessments for the 21st century (pp. 86–134). San Francisco: Jossey-Bass.Google Scholar
  2. Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. New York: Longman.Google Scholar
  3. Asano, T., & Yamaoka, M. (2015). How to reason with economic concepts: Cognitive process of Japanese undergraduate students solving test items. Studies in Higher Education, 40(3), 412–436.CrossRefGoogle Scholar
  4. Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning. New York: Academic.Google Scholar
  5. Bloom, B. S., Engelhart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Handbook I: Cognitive domain. In Taxonomy of educational objectives (Vol. 19). New York: Longman.Google Scholar
  6. Böhner, M. M. (2010). Unterrichtsrelevante Prinzipien im Bereich Wirtschaft. Erziehungswissenschaft und Beruf, 58(3), 315–332.Google Scholar
  7. Brückner, S. (2017). Prozessbezogene Validierung anhand von mentalen Operationen bei der Bearbeitung wirtschaftswissenschaftlicher Testaufgaben. [Process-related validation using mental operations of responding to business and economics test items] Landau: Empirische Pädagogik.Google Scholar
  8. Brückner, S., & Pellegrino, J. W. (2016). Integrating the analysis of mental operations into multilevel models to validate an assessment of higher education students’ competency in business and economics. Journal of Educational Measurement, 53(3), 293–312.CrossRefGoogle Scholar
  9. Butters, R. B., & Walstad, W. B. (2011). Computer versus paper testing in precollege economics. The Journal of Economic Education, 42(4), 366–374.CrossRefGoogle Scholar
  10. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. Hillsdale: L. Erlbaum Associates.Google Scholar
  11. Cui, Y., & Leighton, J. P. (2009). The hierarchy consistency index: Evaluating person fit for cognitive diagnostic assessment. Journal of Educational Measurement, 46(4), 429–449.CrossRefGoogle Scholar
  12. Cumming, G., & Finch, S. (2005). Inference by eye: Confidence intervals and how to read pictures of data. The American Psychologist, 60(2), 170–180.CrossRefGoogle Scholar
  13. Davies, P. (2012). Threshold concepts in economic education. In G. M. Hoyt & K. McGoldrick (Eds.), International handbook on teaching and learning economics (pp. 250–258). Cheltenham/Northampton: Edward Elgar.Google Scholar
  14. Davies, P., & Mangan, J. (2007). Threshold concepts and the integration of understanding in economics. Studies in Higher Education, 32(6), 711–726.CrossRefGoogle Scholar
  15. Davies, P., & Mangan, J. (2009). Understanding graphs in economics: An interpretation through threshold concepts. Paper presented at the biennial conference of the European Association for Learning and Instruction (EARLI) in Amsterdam.Google Scholar
  16. Ericsson, K. A., & Simon, H. A. (1993). Protocol analysis: Verbal reports as data (2nd ed.). Cambridge, MA: MIT Press.Google Scholar
  17. Gagné, R. M. (1985). The conditions of learning and theory of instruction. New York: CBS College Publishing.Google Scholar
  18. Gierl, M. J. (1997). Comparing cognitive representations of test developers and students on a mathematics test with Bloom’s taxonomy. The Journal of Educational Research, 91(1), 26–32.CrossRefGoogle Scholar
  19. Jang, K., Hahn, K., & Kim, K. (2010). Comparative Korean results of TUCE with U.S. and Japan. In M. Yamaoka, W. B. Walstad, M. Watts, T. Asano, & S. Abe (Eds.), Comparative studies on economic education in Asia-Pacific region (pp. 53–78). Tokyo: Shumpusha.Google Scholar
  20. Kricks, K., Mittelstädt, E., & Liening, A. (2013). Schwellenkonzepte und Phänomenografie: Explorative Studie zur Messung von Unterschieden im ökonomischen Verstehen [Threshold concepts and phenomenography. An explorative study on the measurement of differences in economic understanding]. Zeitschrift für ökonomische Bildung, 2, 17–41.Google Scholar
  21. Kuhn, C., Brückner, S., & Zlatkin-Troitschanskaia, O. (2016). A new video-based assessment tool to enhance instructional practices of prospective teachers of economics. Paper presented at the national conference on Teaching and Research on Economic Education (CTREE) in Atlanta.Google Scholar
  22. Leiser, D., & Aroch, R. (2009). Lay understanding of macroeconomic causation: The good begets-good heuristic. Applied Psychology, 58(3), 370–384.CrossRefGoogle Scholar
  23. Liu, O. L., Bridgeman, B., & Adler, R. M. (2012). Measuring learning outcomes in higher education: Motivation matters. Educational Researcher, 41(9), 352–362.CrossRefGoogle Scholar
  24. Lucas, U., & Mladenovic, R. (2006). Developing new world views: Threshold concepts in introductory accounting. In J. Meyer & R. Land (Eds.), Overcoming barriers to student understanding: Threshold concepts and troublesome knowledge (pp. 148–160). London: Routledge.Google Scholar
  25. Lucas, U., & Mladenovic, R. (2009). The identification of variation in students’ understandings of disciplinary concepts: The application of the SOLO taxonomy within introductory accounting. Higher Education, 58(2), 257–283.CrossRefGoogle Scholar
  26. Lydersen, S., Pradhan, V., Senchaudhuri, P., & Laake, P. (2007). Choice of tests for association in small sample unordered r*c tables. Statistics in Medicine, 26, 4328–4343.CrossRefGoogle Scholar
  27. Macha, K., & Schuhen, M. (2011). Framework of measuring economic competencies. Journal of Social Science Education, 10(3), 36–45.Google Scholar
  28. Meyer, J. H. F., & Land, R. (2003). Threshold concepts and troublesome knowledge: Linkages to ways of thinking and practising within the disciplines (Occasional Report 4). Retrieved from ETL Project, Universities of Edinburgh, Coventry and Durham website: https://kennslumidstod.hi.is/wp-content/uploads/2016/04/meyerandland.pdf
  29. Meyer, J. H. F., & Land, R. (2005). Threshold concepts and troublesome knowledge (2): Epistemological considerations and a conceptual framework for teaching and learning. Higher Education, 49(3), 373–388.CrossRefGoogle Scholar
  30. Meyer, J. H. F., & Land, R. (2006). Threshold concepts and troublesome knowledge: An introduction. In J. Meyer & R. Land (Eds.), Overcoming barriers to student understanding. Threshold concepts and troublesome knowledge (pp. 3–18). London/New York: Routledge.Google Scholar
  31. Minnameier, G. (2013). The inferential construction of knowledge in the domain of business and economics. In K. Beck & O. Zlatkin-Troitschanskaia (Eds.), Professional and VET learning: Vol. 2. From diagnostics to learning success. Proceedings in vocational education and training (pp. 141–156). Rotterdam: Sense Publishers.Google Scholar
  32. Musekamp, F., & Pearce, J. (2016). Student motivation in low-stakes assessment contexts: An exploratory analysis in engineering mechanics. Assessment & Evaluation in Higher Education, 41(5), 750–769.CrossRefGoogle Scholar
  33. O’Donnell, R. M. (2009). Threshold concepts and their relevance to economics. Paper presented at the 14th annual Australasian Teaching Economics Conference (ATEC 2009) (pp. 190–200). Brisbane, Queensland: School of Economics and Finance, Queensland University of Technology.Google Scholar
  34. Oser, F., & Spychiger, M. (2005). Lernen ist schmerzhaft: Zur Theorie des negativen Wissens und zur Praxis der Fehlerkultur [Learning is painful – On the theory of negative knowledge and a practice and error culture]. Weinheim: Beltz.Google Scholar
  35. Reimann, N., & Jackson, I. (2006). Threshold concepts in economics: A case study. In J. Meyer & R. Land (Eds.), Overcoming barriers to student understanding. Threshold concepts and troublesome knowledge (pp. 115–133). London/New York: Routledge.Google Scholar
  36. Riegler, P. (2014). Schwellenkonzepte, Konzeptwandel und die Krise der Mathematikausbildung [Threshold concepts, concept transition and the crisis of mathematical education]. Zeitschrift für Hochschulentwicklung, 9(4), 241–257.CrossRefGoogle Scholar
  37. Rogers, W. T., & Yang, P. (1996). Test-Wiseness: Its nature and application. European Journal of Psychological Assessment, 12(3), 247–259.CrossRefGoogle Scholar
  38. Schumann, S., & Eberle, F. (2011). Bedeutung und Verwendung schwierigkeitsbestimmender Aufgabenmerkmale für die Erfassung ökonomischer und beruflicher Kompetenzen. In U. Faßhauer, B. Fürstenau, & E. Wuttke (Eds.), Schriftenreihe der Sektion Berufs- und Wirtschaftspädagogik der Deutschen Gesellschaft für Erziehungswissenschaft (DGfE). Grundlagenforschung zum Dualen System und Kompetenzentwicklung in der Lehrerbildung [Übersetzung] (pp. 77–90). Opladen: Budrich.Google Scholar
  39. Shanahan, M. P., Foster, G., & Meyer, J. H. (2006). Operationalising a threshold concept in economics: A pilot study using multiple choice questions on opportunity cost. International Review of Economics Education, 5(2), 29–57.CrossRefGoogle Scholar
  40. Uribe, R. V. (2013). Measurement of learning outcomes in higher education: The case of Ceneval in Mexico. In S. Blömeke, O. Zlatkin-Troitschanskaia, C. Kuhn, & J. Fege (Eds.), Modeling and measuring competencies in higher education (pp. 137–146). Rotterdam: Sense Publishers.CrossRefGoogle Scholar
  41. Walstad, W. B. (2001). Improving assessment in university economics. The Journal of Economic Education, 32(3), 281–294.CrossRefGoogle Scholar
  42. Walstad, W. B., Watts, M., & Rebeck, K. (2007). Test of understanding in college economics: Examiner’s manual. New York: National Council on Economic Education.Google Scholar
  43. Zlatkin-Troitschanskaia, O., Förster, M., Brückner, S., & Happ, R. (2014). Insights from the German assessment of business and economics competence. In H. Coates (Ed.), Assessing learning outcomes: Perspectives for quality improvement (pp. 175–197). Frankfurt am Main: Lang.Google Scholar
  44. Zumbo, B. D. (2016). Standard-setting methodology: Establishing performance standards and setting cut-scores to assist score interpretation. Applied Physiology, Nutrition, and Metabolism, 41(6), 74–82.CrossRefGoogle Scholar

Copyright information

© Springer International Publishing AG, part of Springer Nature 2018

Authors and Affiliations

  • Sebastian Brückner
    • 1
    Email author
  • Olga Zlatkin-Troitschanskaia
    • 1
  1. 1.Johannes Gutenberg UniversityMainzGermany

Personalised recommendations