Advertisement

CDMs in Vocational Education: Assessment and Usage of Diagnostic Problem-Solving Strategies in Car Mechatronics

  • Stephan AbeleEmail author
  • Matthias von Davier
Chapter
Part of the Methodology of Educational Measurement and Assessment book series (MEMA)

Abstract

The aim of this chapter is to use psychometric models including DCMs to assess diagnostic problem-solving strategies and to investigate the usage of these strategies in car mechatronics. The present study not only advances research on the strategies’ assessment, but also informs professional and vocational education. From the educational perspective, it is not only important to know how to assess diagnostic problem-solving strategies but also to gather information about the strategies’ usage. Such knowledge helps teaching when and under which conditions the strategies are applicable.

References

  1. Abele, S. (2014). Modellierung und Entwicklung berufsfachlicher Kompetenz [Modeling and development of vocational competence]. Stuttgart, Germany: Franz Steiner.Google Scholar
  2. Abele, S. (2017). Diagnostic problem-solving process in professional contexts: Theory and empirical investigation in the context of car mechatronics using computer-generated log-files. Vocations and Learning, 11, 133–159.CrossRefGoogle Scholar
  3. Abele, S., & von Davier, M. (2018). Applying cognitive diagnosis models and latent class analysis to computer-generated process data to identify diagnostic problem-solving strategies in car mechatronics. Manuscript in preparation.Google Scholar
  4. Abele, S., Walker, F., & Nickolaus, R. (2014). Zeitökonomische und reliable Diagnostik beruflicher Problemlösekompetenzen bei Auszubildenden zum Kfz-Mechatroniker. Zeitschrift für Pädagogische Psychologie, 28, 167–179.CrossRefGoogle Scholar
  5. Akaike, H. (1973). Information theory and an extension of the maximum likelihood principle. In B. N. Petrov & F. Caski (Eds.), Proceedings of the second international symposium on information theory (pp. 267–281). Budapest, Hungary: Akademiai Kiado.Google Scholar
  6. Baethge, M., & Arends, L. (2009). Feasibility study VET-LSA: A comparative analysis of occupational profiles and VET programmes in 8 European countries—International report. Vocational training research (Vol. 8). Bielefeld, Germany: Bertelsmann.Google Scholar
  7. Billett, S. (2011). Vocational education: Purposes, traditions and prospects. Dordrecht, The Netherlands: Springer.CrossRefGoogle Scholar
  8. Boshuizen, H. P., & Schmidt, H. G. (2008). The development of clinical reasoning expertise. In J. Higgs, M. A. Jones, S. Loftus, & N. Christensen (Eds.), Clinical reasoning in the health professions (3rd ed., pp. 113–121). Oxford, UK: Elsevier Ltd.Google Scholar
  9. Bozdogan, H. (1987). Model selection and Akaike’s information cri- terion (AIC): The general theory and its analytical extensions. Psychometrika, 52, 345–370.CrossRefGoogle Scholar
  10. Coderre, S., Mandin, H., Harasym, P. H., & Fick, G. H. (2003). Diagnostic reasoning strategies and diagnostic success. Medical Education, 37, 695–703.CrossRefGoogle Scholar
  11. Croskerry, P. (2009). A universal model of diagnostic reasoning. Academic Medicine, 84(8), 1022–1028.CrossRefGoogle Scholar
  12. Elstein, A. S., Shulman, L. S., & Sprafka, S. A. (1990). Medical problem solving: A ten-year retrospective. Evaluation & the Health Professions, 13(1), 5–36.CrossRefGoogle Scholar
  13. Frey, A., Hartig, J., & Rupp, A. A. (2009). An NCME instructional module on booklet designs in large-scale assessments of student achievement: Theory and practice. Educational Measurement: Issues and Practice, 28(3), 39–53.CrossRefGoogle Scholar
  14. Goldhammer, F., Kröhne, U., Keßel, Y., Senkbeil, M., & Ihme, J. M. (2014). Diagnostik von ICT-literacy: Multiple-choice- vs. simulationsbasierte Aufgaben. Diagnostica, 60(1), 10–21.CrossRefGoogle Scholar
  15. Greiff, S., Wüstenberg, S., & Avvisati, F. (2015). Computer-generated log-file analyses as a window into students’ minds? A showcase study based on the PISA 2012 assessment of problem solving. Computers & Education, 91, 92–105.CrossRefGoogle Scholar
  16. Gschwendtner, T., Abele, S., & Nickolaus, R. (2009). Computersimulierte Arbeitsproben: Eine Validierungsstudie am Beispiel der Fehlerdiagnoseleistungen von Kfz-Mechatronikern Mechatronikern [Computer-simulated work samples: A statistical validation study using the example of trouble-shooting competency of car mechatronics]. Zeitschrift für Berufs- und Wirtschaftspädagogik, 105, 557–578.Google Scholar
  17. Hanushek, E. A., Schwerdt, G., Woessmann, L., & Zhang, L. (2017). General education, vocational education, and labor-market outcomes over the lifecycle. Journal of Human Resources, 52(1), 48–87.CrossRefGoogle Scholar
  18. He, Q., & von Davier, M. (2015). Identifying feature sequences from process data in problem-solving items with N-grams. In A. van der Ark, D. Bolt, S. Chow, J. Douglas, & W. Wang (Eds.), Quantitative psychology research: Proceedings of the 79th annual meeting of the psychometric society (pp. 173–190). New York, NY: Springer.  https://doi.org/10.1007/978-3-319-19977-1_13 CrossRefGoogle Scholar
  19. He, Q., & von Davier, M. (2016). Analyzing process data from problem-solving items with N-grams: Insights from a computer-based large-scale assessment. In Y. Rosen, S. Ferrara, & M. Mosharraf (Eds.), Handbook of research on technology tools for real-world skill development (pp. 749–776). Hershey, PA: Information Science Reference.  https://doi.org/10.4018/978-1-4666-9441-5.ch029 CrossRefGoogle Scholar
  20. Hoc, J.-M., & Amalberti, R. (1995). Diagnosis: Some theoretical questions raised by applied research. Current Psychology of Cognition, 14(1), 73–101.Google Scholar
  21. Jonassen, D. H. (2011). Learning to solve problems. A handbook for designing problem-solving learning environments. New York, NY: Routledge.Google Scholar
  22. Jonassen, D. H., & Hernandez-Serrano, J. (2002). Case-based reasoning and instructional design using stories to support problem solving. Educational Technology Research and Development, 50, 65–77.CrossRefGoogle Scholar
  23. Junker, B. W., & Sijtsma, K. (2001). Cognitive assessment models with few assumptions, and connections with nonparametric item response theory. Applied Psychological Measurement, 25, 258–272.CrossRefGoogle Scholar
  24. Kassirer, J., Wong, J., & Kopelman, R. (2010). Learning clinical reasoning (3rd ed.). Baltimore, MD: Lippincott Williams & Wilkins.Google Scholar
  25. Kluwe, R. H., & Haider, H. (1990). Modelle zur internen Repräsentation komplexer technischer Systeme. Sprache & Kognition, 9(4), 173–192.Google Scholar
  26. Konradt, U. (1995). Strategies of failure diagnosis in computer-controlled manufacturing systems: Empirical analysis and implications for the design of adaptive decision support systems. International Journal of Human-Computer Studies, 43(4), 503–521.CrossRefGoogle Scholar
  27. Lazarsfeld, P. F., & Henry, N. W. (1968). Latent structure analysis. Boston, MA: Houghton Mifflin Company.Google Scholar
  28. Lerman, R. I. (2016). Restoring opportunity by expanding apprenticeship. In I. Kirsch & H. Braun (Eds.), The dynamics of opportunity in America. Cham, Switzerland: Springer.  https://doi.org/10.1007/978-3-319-25991-8_10 CrossRefGoogle Scholar
  29. Lord, F. M., & Novick, M. R. (1968). Statistical theories of mental test scores. Menlo Park: Addison-Wesley.Google Scholar
  30. McDonald, R. P. (1999). Test theory: A unified treatment. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  31. Moustaki, I., & Knott, M. (2000). Generalised latent trait models. Psychometrika, 65(3), 391–411.CrossRefGoogle Scholar
  32. Nickolaus, R., Abele, S., Gschwendtner, T., Nitzschke, A., & Greiff, S. (2012). Fachspezifische Problemlösefähigkeit in gewerblich-technischen Ausbildungsberufen—Modellierung, erreichte Niveaus und relevante Einflussfaktoren [Occupation-specific problem solving competency as an essential competency dimension of professional competency: Models, achieved levels and relevant predictors in technical education]. Zeitschrift für Berufs- und Wirtschaftspädagogik, 108, 243–272.Google Scholar
  33. Norman, G., Young, M., & Brooks, L. (2007). Non-analytical models of clinical reasoning: The role of experience. Medical Education, 41, 1140–1145.Google Scholar
  34. Norman, G. R. (2005). Research in clinical reasoning: Past history and current trends. Medical Education, 39, 418–427.CrossRefGoogle Scholar
  35. Perez, R. S. (2012). A view from troubleshooting. In M. U. Smith (Ed.), Toward a unified theory of problem solving (pp. 127–166). New York, NY: Routledge.Google Scholar
  36. Rasmussen, J. (1981). Models of mental strategies in process plant diagnosis. In J. Rasmussen & W. Rouse (Eds.), Human detection and diagnosis of system failures (NATO conference series, Bd. 3, Bd. 15, S., pp. 241–258). New York, NY: Springer US.Google Scholar
  37. Rasmussen, J. (1993). Diagnostic reasoning in action. IEEE Transactions on Systems, Man and Cybernetics, 23(4), 981–992.  https://doi.org/10.1109/21.247883 CrossRefGoogle Scholar
  38. Rausch, A., Seifried, J., Wuttke, E., Kögler, K., & Brandt, S. (2016). Reliability and validity of a computer-based assessment of cognitive and non-cognitive facets of problem-solving competence in the business domain. Empirical Research in Vocational Education and Training, 8(1), 1–23.CrossRefGoogle Scholar
  39. Rouse, W. B. (1983). Models of human problem solving: Detection, diagnosis, and compensation for system failures. Automatica, 19(6), 613–625.  https://doi.org/10.1016/0005-1098(83)90025-0 CrossRefGoogle Scholar
  40. Schaper, N., Hochholdinger, S., & Sonntag, K. (2004). Förderung des Transfers von Diagnosestrategien durch computergestütztes Training mit kognitiver Modellierung [Improving transfer of troubleshooting skills by computer-based training with modeling]. Zeitschrift für Personalpsychologie, 3(2), 51–62.CrossRefGoogle Scholar
  41. Schwartz, A., & Elstein, A. S. (2008). Clinical reasoning in medicine. In J. Higgs, M. A. Jones, S. Loftus & N. Christensen (Eds.), Clinical reasoning in the health professions (3. Aufl., pp. 223–234). Oxford, UK: Elsevier Ltd.Google Scholar
  42. Schwarz, G. (1978). Estimating the dimension of a model. Annals of Statistics, 6, 461–464.CrossRefGoogle Scholar
  43. Sembill, D., Rausch, A., & Kögler, K. (2013). Non-cognitive facets of competence. Theoretical foundations and implications for measurement. In O. Zlatkin-Troitschanskaia & K. Beck (Eds.), From diagnostics to learning success—Proceedings in vocational education and training (pp. S. 199–S. 212). Rotterdam, The Netherlands: Sense.CrossRefGoogle Scholar
  44. Skrondal, A., & Rabe-Hesketh, S. (2004). Generalized latent variable modeling: Multilevel, longitudinal and structural equation models. Boca Raton, FL: Chapman & Hall/CRC.CrossRefGoogle Scholar
  45. Tatsuoka, K. K. (1983). Rule-space: An approach for dealing with misconceptions based on item response theory. Journal of Educational Measurement, 20, 345–354.CrossRefGoogle Scholar
  46. van Merriënboer, J. G. (2013). Perspectives on problem solving and instruction. Computers & Education, 64, 153–160.CrossRefGoogle Scholar
  47. von Davier, M. (2005). A general diagnostic model applied to language testing data. ETS Research Report Series, 2005, i–35.  https://doi.org/10.1002/j.2333-8504.2005.tb01993.x CrossRefGoogle Scholar
  48. von Davier, M. (2008). A general diagnostic model applied to language testing data. British Journal of Mathematical and Statistical Psychology, 61, 287–307.  https://doi.org/10.1348/000711007X193957 CrossRefGoogle Scholar
  49. von Davier, M. (2009, March). Some notes on the reinvention of latent structure models as diagnostic classification models. Measurement—Interdisciplinary Research and Perspectives, 7(1), 67–74.CrossRefGoogle Scholar
  50. von Davier, M. (2013). The DINA model as a constrained general diagnostic model—Two variants of a model equivalency. BJMSP, 67, 49–71.. http://onlinelibrary.wiley.com/doi/10.1111/bmsp.12003/abstract Google Scholar
  51. von Davier, M. (2014), The log-linear cognitive diagnostic model (LCDM) as a special case of the general diagnostic model (GDM). ETS research report series. http://onlinelibrary.wiley.com/doi/10.1002/ets2.12043/abstract
  52. von Davier, M., DiBello, L., & Yamamoto, K. (2008). Reporting test outcomes using models for cognitive diagnosis. In J. Hartig, E. Klieme, & D. Leutner (Eds.), Assessment of competencies in educational contexts (pp. 151–176). Göttingen, Germany: Hogrefe & Huber Publishers.Google Scholar
  53. von Davier, M., & Rost, J. (2016). Logistic mixture-distribution response models. In W. van der Linden (Ed.), Handbook of item response theory (Vol. 1, 2nd ed., pp. 393–406). Boca Raton, FL: CRC Press. http://www.crcnetbase.com/doi/abs/10.1201/9781315374512-24 Google Scholar
  54. von Davier, M., Xu, X., & Carstensen, C. H. (2011). Measuring growth in a longitudinal large-scale assessment with a general latent variable model. Psychometrika, 76, 318–336.  https://doi.org/10.1007/s11336-011-9202-z CrossRefGoogle Scholar
  55. von Davier, M., & Yamamoto, K. (2004, October). A class of models for cognitive diagnosis. Paper presented at the 4th Spearman conference “Diagnostics for education: Theory, measurement, applications.” ETS: The Inn at Penn, Philadelphia, PA.Google Scholar
  56. von Davier, M., Yamamoto, K., Shin, H.-J., Chen, H., Khorramdel, L., Weeks, J., Davis, S., Kong, N., & Kandathil, M. (2019). Evaluating item response theory linking and model fit for data from PISA 2000–2012. Assessment in Education: Principles, Policy & Practice.Google Scholar
  57. von Davier, M. (2007). Hierarchical general diagnostic models. Research Report, RR-07-19. Princeton, NJ: ETS. https://onlinelibrary.wiley.com/doi/abs/10.1002/j.2333-8504.2007.tb02061.x CrossRefGoogle Scholar
  58. van der Linden, W. (2018). Handbook of item response theory, three volume set. Chapman & Hall/CRC statistics in the social and Behavioral sciences. ISBN: 148228247X, 9781482282474CrossRefGoogle Scholar
  59. Yamamoto, K., Khorramdel, L., & von Davier, M. (2013). Scaling PIAAC cognitive data. Technical report of the survey of adult skills (PIAAC). Paris: Organisation for Economic Co-operation and Development.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2019

Authors and Affiliations

  1. 1.Institute of Vocational Education and Vocational DidacticsTechnische Universität DresdenDresdenGermany
  2. 2.National Board of Medical Examiners (NBME)PhiladelphiaUSA

Personalised recommendations