Advertisement

Performance Assessment of Generic and Domain-Specific Skills in Higher Education Economics

  • M.-T. NagelEmail author
  • Olga Zlatkin-Troitschanskaia
  • S. Schmidt
  • K. Beck
Chapter
  • 47 Downloads

Abstract

Following criticisms by employers about academic graduates’ lack of 21st century skills, students need to develop skills such as professional knowledge, critical thinking and problem solving. Accordingly, there is a demand for suitable assessments of these skills. One approach is to develop a performance assessment using tasks adapted from real-world decision-making and judgment situations that students and graduates have to face in academic and professional domains. Such tasks employ real-life scenarios and require generic and domain-specific skills in different facets to handle a given problem adequately. In this paper, we present a newly developed performance assessment that aims to measure such skills among higher education economics students and graduates of economics and we report results from two validation studies.

Keywords

Generic skills domain-specific knowledge critical thinking graduates of economics performance assessment validation 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Alexander, P. A., Singer, L. M., Jablansky, S., & Hattan, C. (2016). Relational reasoning in word and in figure. Journal of Educational Psychology, 108 (8), 1140–1152.Google Scholar
  2. American Educational Research Association, American Psychological Association and National Council on Measurement in Education (AERA, APA and NCME). (2014). Standards for educational and psychological testing. Washington, DC: AERA.Google Scholar
  3. Allgood, S., & Bayer, A. (2016). Measuring College Learning in Economics. MPRA Paper No. 85104. Online: https://mpra.ub.uni-muenchen.de/85104/.
  4. Allgood, S., Walstad, W. B., & Siegfried, J. J. (2015). Research on Teaching Economics to Undergraduates. Journal of Economic Literature, 53 (2), 285–325.Google Scholar
  5. Arum, R., & Roksa, J. (2010). Academically adrift: Limited learning on college campuses. Chicago, IL: University of Chicago Press.Google Scholar
  6. Browne, M. N., Hoag, J. H., & Boudreau, N. (1995). Critical Thinking in Graduate Economic Programs: A Study of Faculty Perceptions. The Journal of Economic Education, 26 (2), 177–181.Google Scholar
  7. Facione, P. A. (2000). The disposition toward critical thinking: Its character, measurement, and relation to critical thinking skill. Informal Logic, 20 (1), 61–84.Google Scholar
  8. Fives, H., & Dinsmore, D. L. (Eds.). (2017). The Model of Domain Learning: Understanding the Development of Expertise. New York, Abingdon: Routledge.Google Scholar
  9. Halpern, D. F. (2014). Thought and knowledge: An introduction to Critical Thinking. New York: Psychology Press.Google Scholar
  10. Happ, R., Nagel, M.-T., Zlatkin-Troitschanskaia, O., & Schmidt, S. (2019). How migration background affects master degree students’ knowledge of business and economics. Studies in Higher Education.  https://doi.org/10.1080/03075079.2019.1640670.
  11. Hattie, J., & Cooksey, R. W. (1984). Procedures for assessing the validities of tests using the” known-groups” method. Applied Psychological Measurement, 8 (3), 295–305.Google Scholar
  12. Hoyt G. M., & McGoldrick, K. (Eds.). (2012). International Handbook on Teaching and Learning Economics. Cheltenham, Northampton: Edward Elgar.Google Scholar
  13. Kahneman, D. (2011). Thinking, fast and slow. New York, NY: Farrar, Straus and Giroux.Google Scholar
  14. Kobrin, J. L., Patterson, B. F., Shaw, E. J., Mattern, K. D., & Barbuti, S. M. (2008). Validity of the SAT® for Predicting First-Year College Grade Point Average. College Board Research Report (5). New York: The College Board.Google Scholar
  15. Lai, E. R., & Viering, M. (2012). Assessing 21st Century Skills: Integrating research findings. Paper presented at the annual meeting of the National Council on Measurement in Education. Vancouver, B.C., Canada.Google Scholar
  16. Liepmann, D., Beauducel, A., Brocke, B., & Amthauer, R. (2007). Intelligenz-Struktur-Test 2000 R (I-S-T 2000 R). Available on https://psychowissen.jimdo.com/psychologische-tests/intelligenztests/i-s-t-2000-r/
  17. Liu, O. L., Frankel, L., & Roohr, K. C. (2014). Assessing Critical Thinking in Higher Education: Current State and Directions for Next-Generation Assessments. ETS Research Report, 14 (10).Google Scholar
  18. McGoldrick, K., & Garnett, R. (2013). Big Think: A Model for Critical Inquiry in Economics Courses. The Journal of Economic Education, 44 (4), 389–398.Google Scholar
  19. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational Measurement (3rd ed., pp. 13–103). New York: Macmillan Publishing.Google Scholar
  20. Paul, R. W., & Elder, L. (2006). Critical thinking: The nature of critical and creative thought. Journal of Developmental Education, 30 (2), 34–35.Google Scholar
  21. Pellegrino, J. W., & Hilton, M. L. (2012). Education for Life and Work: Developing Transferable Knowledge and Skills in the 21st Century. Washington, DC: The National Academies Press.Google Scholar
  22. Pellegrino, J. W., Chudowsky, N., & Glaser, R. (2001). Knowing What Students Know: The Science and Design of Educational Assessment. Washington, DC: National Academy Press.Google Scholar
  23. Shavelson, R. J., Zlatkin-Troitschanskaia, O., Beck, K., Schmidt, S., & Marino, J. (2019). Assessment of University Students’ Critical Thinking: Next Generation Performance Assessment. International Journal of Testing. doi: doi.org/ https://doi.org/10.1080/15305058.2018.1543309
  24. Shavelson, R. J., Zlatkin-Troitschanskaia, O., & Mariño, J. P. (2018). International Performance Assessment of Learning in higher education (iPAL) – Research and Development. Wiesbaden: Springer.Google Scholar
  25. Walstad, W. B. (2001). Improving Assessment in University Economics. The Journal of Economic Education, 32 (3), 281–294.CrossRefGoogle Scholar
  26. Walstad, W. B., Rebeck, K., & Butters, R. B. (2013). Test of economic literacy: Examiner’s manual, 4th ed. New York: Council for Economic Education.Google Scholar
  27. Walstad, W. B., Watts, M., & Rebeck, K. (2007). Test of Understanding in College Economics: Examiner`s manual, 4th ed. New York: National Council on Economic Education.Google Scholar
  28. West, R. F., Toplak, M. E., & Stanovich, K. E. (2008). Heuristics and biases as measures of critical thinking: Associations with cognitive ability and thinking dispositions. Journal of Educational Psychology, 100 (4), 930–941.Google Scholar
  29. Willingham, D. T. (2007). Critical thinking: Why is it so hard to teach? American Educator, 31 (2), 8–19.Google Scholar
  30. Zlatkin-Troitschanskaia, O., Jitomirski, J., Happ, R., Molerov, D., Schlax, J., Kühling-Thees, C., Förster, M. & Brückner, S. (2019). Validating a Test for Measuring Knowledge and Understanding of Economics Among University Students. Zeitschrift für Pädagogische Psychologie, 33(2), 119–133.Google Scholar
  31. Zlatkin-Troitschanskaia, O., Shavelson, R. J., Schmidt, S., & Beck, K. (2019b). On the complementarity of holistic and analytic approaches to performance assessment scoring. The British Journal of Educational Psychology. Advance online publication.  https://doi.org/10.1111/bjep.12286
  32. Zlatkin-Troitschanskaia, O., Shavelson, R. J., & Pant, H. A. (2018a). Assessment of Learning Outcomes in Higher Education – International Comparisons and Perspectives. In C. Secolsky and B. Denison (Eds.). Handbook on Measurement, Assessment and Evaluation in Higher Education (2nd ed.). New York: Routledge.Google Scholar
  33. Zlatkin-Troitschanskaia, O., Toepper, M., Molerov, D., Buske, R., Brückner, S., Pant, H. A., Hofmann, S., & Hansen-Schirra, S. (2018b). Adapting and Validating the Collegiate Learning Assessment to Measure Generic Academic Skills of Students in Germany: Implications for International Assessment Studies in Higher Education. In O. Zlatkin-Troitschanskaia, M. Toepper, H. A. Pant, C. Lautenbach, C. Kuhn (Eds.) Assessment of Learning Outcomes in higher education – Cross-National Comparisons and Perspectives (pp. 245–266). Cham: Springer.Google Scholar

Copyright information

© Springer Fachmedien Wiesbaden GmbH, part of Springer Nature 2020

Authors and Affiliations

  • M.-T. Nagel
    • 1
    Email author
  • Olga Zlatkin-Troitschanskaia
    • 2
  • S. Schmidt
    • 1
  • K. Beck
    • 1
  1. 1.Johannes Gutenberg-UniversitätMainzGermany
  2. 2.Johannes Gutenberg University MainzMainzGermany

Personalised recommendations