Zusammenfassung
Das Kapitel gibt einen Überblick, wie mit Hilfe von Computern im weiteren Sinne Tests und Fragebogen realisiert und dabei die Möglichkeiten von klassischen Papier-und-Bleistift-Verfahren erweitert bzw. deutlich überschritten werden können. Dies betrifft beispielsweise die Entwicklung computerbasierter Items mit innovativen Antwortformaten und multimedialen Stimuli sowie die automatische Bewertung des gezeigten Antwortverhaltens. Des Weiteren ermöglicht der Computer eine flexiblere Testzusammenstellung, d. h., Items können automatisch unter Berücksichtigung inhaltlicher und statistischer Kriterien sequenziert werden. Das Kapitel behandelt außerdem die Frage, wie durch Logfiledaten das Analysepotential gesteigert und durch die automatische und zeitnahe Rückmeldung von Testdaten beispielsweise das Lernen unterstützt werden kann. Das Kapitel schließt mit Hinweisen auf einschlägige und frei zugängliche Softwarelösungen für Assessmentzwecke.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Similar content being viewed by others
Literatur
Abele, S., Walker, F. & Nickolaus, R. (2014). Zeitökonomische und reliable Diagnostik beruflicher Problemlösekompetenzen bei Auszubildenden zum Kfz-Mechatroniker. Zeitschrift für Pädagogische Psychologie, 4, 167–179.
Aleven, V., Roll, I., McLaren, B. & Koedinger, K. (2010). Automated, unobtrusive, action-by-action assessment of self-regulation during learning with an intelligent tutoring system. Educational Psychologist, 45, 224–233.
Bassili, J. N. & Fletcher, J. F. (1991). Response-time measurement in survey research a method for CATI and a new look at nonattitudes, Public Opinion Quarterly, 55, 331–346.
Brace, I. (2008). Questionnaire design: How to plan, structure and write survey material for effective market research (2nd ed). London; Philadelphia: Kogan Page.
Braun, H., Bejar, I. I. & Williamson, D. M. (2006). Rule-based methods for automated scoring: Application in a licensing context. In D. M. Williamson, I. I. Bejar & R. J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 83–122). Mahwah, NJ: Lawrence Erlbaum Associates.
Chen, M., Mao, S. & Liu, Y. (2014). Big Data: A Survey. Mobile Networks and Applications, 19, 171–209.
Clark, D., Nelson, B., Sengupta, P. & D’Angelo, C. (2009). Rethinking science learning through digital games and simulations: Genres, examples, and evidence. An NAS commissioned paper. Retrieved from http://sites.nationalacademies.org/cs/groups/dbassesite/documents/webpage/dbasse_080068.pdf [20.12.2019]
Cor, K., Alves, C. & Gierl, M. (2009). Three applications of automated test assembly within a user-friendly modeling environment. Practical Assessment, Research & Evaluation, 14. Retrieved from http://pareonline.net/getvn.asp?v=14%26n=14 [20.12.2019]
Deroos, D., Deutsch, T., Eaton, C., Lapis, G. & Zikopoulos, P. (2012). Understanding big data: analytics for enterprise class Hadoop and streaming data. New York, NY: McGraw-Hill.
Diao, Q. & van der Linden, W. J. (2011). Automated Test Assembly Using lp_Solve Version 5.5 in R. Applied Psychological Measurement, 35, 398–409.
Dirk, J. & Schmiedek, F. (2016). Fluctuations in elementary school children’s working memory performance in the school context. Journal of Educational Psychology, 108, 722–739.
Ebner-Priemer, U. W., Kubiak, T. & Pawlik, K. (2009). Ambulatory Assessment. European Psychologist, 14, 95–97.
Frey, A. & Hartig, J. (2013). Wann sollten computerbasierte Verfahren zur Messung von Kompetenzen anstelle von Papier- und Bleistift-basierten Verfahren eingesetzt werden? Zeitschrift für Erziehungswissenschaft, 16, 53–57.
Funke, U. (1995). Using complex problem solving tasks in personnel selection and training. In P. A. Frensch & J. Funke (Eds.), Complex problem solving: The European perspective (pp. 219–240). Hillsdale, NJ: Lawrence Erlbaum Associates.
Gawrilow, C., Stadler, G., Langguth, N., Naumann, A. & Boeck, A. (2016). Physical activity, affect, and cognition in children with symptoms of ADHD. Journal of Attention Disorders, 20, 151–162.
Gierl, M. J. & Lai, H. (2013). Using automated processes to generate test items. Educational Measurement: Issues and Practice, 32, 36–50.
Goldhammer, F., Kröhne, U., Keßel, Y., Senkbeil, M. & Ihme, J. M. (2014a). Diagnostik von ICT-Literacy: Multiple-Choice- vs. simulationsbasierte Aufgaben. Diagnostica, 60, 10–21.
Goldhammer, F., Naumann, J., Rölke, H., Stelter, A. & Tóth, K. (2017). Relating product data to process data from computer-based competency assessment. In D. Leutner, J. Fleischer, J. Grünkorn & E. Klieme (Eds.), Competence Assessment in Education: Research, Models and Instruments (pp. 407–425). Berlin, Heidelberg: Springer.
Goldhammer, F., Naumann, J., Stelter, A., Tóth, K., Rölke, H. & Klieme, E. (2014b). The time on task effect in reading and problem solving is moderated by task difficulty and skill: Insights from a computer-based large-scale assessment. Journal of Educational Psychology, 106, 608–626.
Greiff, S., Niepel, C., Scherer, R. & Martin, R. (2016). Understanding students’ performance in a computer-based assessment of complex problem solving: An analysis of behavioral data from computer-generated log files. Computers in Human Behavior, 61, 36–46.
Greiff, S., Wüstenberg, S. & Funke, J. (2012). Dynamic Problem Solving: A new measurement perspective. Applied Psychological Measurement, 36, 189–213.
Hahnel, C., Goldhammer, F., Naumann, J. & Kröhne, U. (2016). Effects of linear reading, basic computer skills, evaluating online information, and navigation on reading digital text. Computers in Human Behavior, 55, 486–500.
Han, K. T. & Guo, F. (2014). Multistage testing by shaping modules on the fly. In D. Yan, A. A. von Davier & Lewis (Eds.), Computerized multistage testing: Theory and applications (pp. 119–133). New York, NY: CRC Press.
Holling, H., Bertling, J. P. & Zeuch, N. (2009). Automatic item generation of probability word problems. Studies in Educational Evaluation, 35, 71–76.
Hornke, L. F., Wosnitza, M. & Bürger, K. (2013). SelfAssessment: Ideen, Hintergründe, Praxis und Evaluation, Wirtschaftspsychologie, 15, 5–16.
Klein Entink, R. H., Fox, J.-P. & van der Linden, W. J. (2009). A multivariate multilevel approach to the modeling of accuracy and speed of test takers. Psychometrika, 74, 21–48.
Koedinger, K. R. & Aleven V. (2007). Exploring the assistance dilemma in experiments with cognitive tutors. Educational Psychology Review, 19, 239–264.
Kong, X., Wise, S. L. & Bhola, D. S. (2007). Setting the response time threshold parameter to differentiate solution behavior from rapid-guessing behavior. Educational and Psychological Measurement, 67, 606–619.
Kosinski, M., Stillwell, D. & Graepel, T. (2013). Private traits and attributes are predictable from digital records of human behavior. Proceedings of the National Academy of Sciences, 110, 5802–5805.
Kreuter, F. (2013). Improving surveys with paradata: Analytic uses of process information (Vol. 581). New York, NY: John Wiley & Sons.
Kroehne, U. & Martens, T. (2011). Computer-based competence tests in the national educational panel study: The challenge of mode effects. Zeitschrift für Erziehungswissenschaft, 14, 169–186.
Kuhn, J.-T. & Kiefer, T. (2013). Optimal test assembly in practice: The design of the Austrian Educational Standards Assessment in Mathematics. Zeitschrift für Psychologie, 221, 190–200.
Kühnhausen, J., Leonhardt, A., Dirk, J. & Schmiedek, F. (2013). Physical activity and affect in elementary school children’s daily lives. Frontiers in Movement Science and Sport Psychology, 4, 456.
Landauer, T. K., Foltz, P. W. & Laham, D. (1998). Introduction to Latent Semantic Analysis. Discourse Processes, 25, 259–284.
Larson, R. & Csikszentmihalyi, M. (1983). The experience sampling method. In H. T. Reis (Ed.), Naturalistic approaches to studying social interaction. New directions for methodology of social and behavioral sciences (pp. 41–56). San Francisco, CA: Jossey-Bass.
Luecht, R. L. & Sireci (2011). A review of models for computer-based testing. Research report 2011–2012. New York, NY: The College Board.
Margolis, M. J. & Clauser, B. E. (2006). A regression-based procedure for automated scoring of a complex medical performance assessment. In D. M. Williamson, I. I. Bejar & R. J. Mislevy (Eds.), Automated scoring of complex tasks in computer-based testing (pp. 123–168). Mahwah, NJ: Lawrence Erlbaum Associates.
Masters, J. (2010). Automated Scoring of an Interactive Geometry Item: A Proof-of-Concept. Journal of Technology, Learning, and Assessment, 8. Retrieved from https://ejournals.bc.edu/index.php/jtla/article/view/1626 [20.12.2019]
Mead, A. D. & Drasgow, F. (1993). Equivalence of Computerized and Paper-and-Pencil Cognitive Ability Tests: A Meta-Analysis. Psychological Bulletin, 114, 449–458.
Mislevy, R. J. (2013). Evidence-centered design for simulation-based assessment. Military Medicine, 178, 107–114.
Mislevy, R. J., Almond, R. G. & Lukas, J. F. (2003). A brief introduction to evidence-centered design (Research Report 03-16). Princeton, NJ: Educational Testing Service.
Mislevy, R. J., Behrens, J. T., Dicerbo, K. E. & Levy, R. (2012). Design and discovery in educational assessment: Evidence-centered design, psychometrics, and educational data mining. Journal of Educational Data Mining, 4, 11–48.
Mislevy, R. J., Steinberg, L. S., Breyer, F. J., Almond, R. G. & Johnson, L. (2002). Making sense of data from complex assessments. Applied Measurement in Education, 15, 363–389.
Moosbrugger, H. (2011). Lineare Modelle. Regressions- und Varianzanalysen (4. Aufl.). Bern: Huber.
Moosbrugger, H. & Goldhammer, F. (2007). FAKT-II. Frankfurter Adaptiver Konzentrationsleistungs-Test II. Computerprogramm. Grundlegend neu bearbeitete und neu normierte 2. Auflage des FAKT von Moosbrugger und Heyden (1997). Bern: Huber.
Organisation for Economic Co-operation and Development (OECD). (2011). PISA 2009 results Vol VI: Students on line – Digital technologies and performance. Paris: PISA, OECD Publishing. Retrieved from https://doi.org/10.1787/9789264112995-en [20.12.2019]
Organisation for Economic Co-operation and Development (OECD). (2013). OECD Skills Outlook 2013: First Results from the Survey of Adult Skills. Paris: OECD Publishing.
Organisation for Economic Co-operation and Development (OECD). (2014). PISA 2012 Results: What Students Know and Can Do – Student Performance in Mathematics, Reading and Science (Volume I, Revised edition, February 2014). Paris: PISA, OECD Publishing. Retrieved from https://doi.org/10.1787/9789264201118-en [20.12.2019]
Parshall, C. G., Harmes, J. C., Davey, T. & Pashley, P. J. (2010). Innovative item types for computerized testing. In W. J. van der Linden & C. A. W. Glas (Eds.), Elements of adaptive testing (pp. 215–230). New York, NY: Springer.
Parshall, C. G., Spray, J. A., Kalohn, J. C. & Davey, T. (2002). Practical considerations in computer-based testing. New York, NY: Springer.
Richman-Hirsch, W. L., Olson-Buchanan, J. B. & Drasgow, F. (2000). Examining the impact of administration medium on examinee perceptions and attitudes. Journal of Applied Psychology, 85, 880–887.
Reiß, S. & Moosbrugger, H. (2008) Online Self Assessment Psychologie. Institut für Psychologie der Goethe-Universität Frankfurt am Main. https://www.psychologie.uni-frankfurt.de/49829947/20_self-Assessment [20.12.2019]
Reips, U.-D. (2010). Design and formatting in Internet-based research. In S. Gosling & J. Johnson (Eds.), Advanced methods for conducting online behavioral research (pp. 29–43). Washington, DC: American Psychological Association.
Robitzsch, A., Lüdtke, O., Köller, O., Kröhne, U., Goldhammer, F. & Heine, J. H. (2017). Herausforderungen bei der Schätzung von Trends in Schulleistungsstudien. Diagnostica, 63, 148–165.
Rölke, H. (2012a). Automata and Petri Net Models for visualizing and analyzing complex questionnaires – A case study. In S. Donatelli, J. Kleijn, R. J. Machado & J. M. Fernandes (Eds.) CEUR Workshop Proceedings (Vol. 827, p. 317–329). Braga: CEUR-WS.org. Retrieved from https://www.researchgate.net/publication/220852463_Automata_and_Petri_Net_Models_for_Visualizing_and_Analyzing_Complex_Questionnaires_A_Case_Study [20.12.2019]
Rölke, H. (2012b). The Item Builder: A graphical authoring system for complex item development. In T. Bastiaens & G. Marks (Eds), Proceedings of World Conference on E-Learning in Corporate, Government, Healthcare, and Higher Education (pp. 344–353). Chesapeake, VA: AACE. Retrieved from http://www.editlib.org/p/41614 [20.12.2019]
Russel, M. (2010). Technology-aided formative assessment and learning: New developments and applications. In H. L. Andrade & G. J. Cizek (Eds.), Handbook of formative assessment (pp. 125–138). New York, NY: Routledge.
Russell, M. (2011). Computerized tests sensitive to individual needs. In S. N. Elliott, R. J. Kettler, P. A. Beddow & A. Kurz (Eds.), Handbook of accessible achievement tests for all students. (pp. 255–273). New York, NY: Springer.
Saris, W. E. (1991). Computer-assisted interviewing. Newbury Park, Calif: Sage Publications.
Scalise, K. (2012). Creating innovative assessment items and test forms. In R. W. Lissitz & H. Jiao (Eds.), Computers and their impact on state assessment: Recent history and predications for the future (pp. 134–156). Charlotte, NC: Information Age Publisher.
Scalise, K. & Gifford, B. R. (2006). Computer-based assessment in E-learning: A framework for constructing “intermediate constraint” questions and tasks for technology platforms. Journal of Teaching, Learning and Assessment, 4(6), 1–45.
Schwartz, H. A., Eichstaedt, J. C., Kern, M. L., Dziurzynski, L., Ramones, S. M., Agrawal, M., et al (2013). Personality, gender, and age in the language of social media: The open-vocabulary approach. PLoS ONE 8: e73791. https://doi.org/10.1371/journal.pone.0073791
Seidel, T., Blomberg, G., Stürmer, K. (2010). „Observer“ – Validierung eines videobasierten Instruments zur Erfassung der professionellen Wahrnehmung von Unterricht. Zeitschrift für Pädagogik, 56, 296–306.
Shah, D. V., Cappella, J. N. & Neuman, W. R. (2015). Big data, digital media, and computational social science: Possibilities and perils. Annals of the American Academy of Political and Social Science, 659, 6–13.
Shermis, M. D. & Burstein, J. (Eds.), (2003). Automated essay scoring: A cross-disciplinary perspective. Hillsdale, NJ: Lawrence Erlbaum Associates.
Sinharay, S. & Johnson, M. S. (2013). Statistical modeling of automatically generated items. In M. J . Gierl & T. Haladyna (Eds.), Automatic item generation: Theory and practice (pp. 183–195). New York, NY: Routledge.
Sireci, S. G., Li, S. & Scarpati, S. (2005). Test accommodations for students with disabilities: An analysis of the interaction hypothesis. Review of Educational Research, 75, 457–490.
Sireci, S. G. & Zenisky, A. L. (2006). Innovative item formats in computer-based testing: In pursuit of improved construct representation. In S. M. Downing & Th. M. Haladyna (Eds.), Handbook of test development (pp. 329–347). Mahwah, NJ: Lawrence Erlbaum Associates.
Steinwascher, M. A. & Meiser, T. (2016). How a high working memory capacity can increase proactive interference. Consciousness and Cognition, 44, 130–145.
Stout, W. (2002). Test Models for Traditional and Complex CBTs. In C. Mills, M. Potenza, J. Fremer & W. Ward (Eds.), Computer-Based Testing (pp. 103–118). Mahwah, NJ: Lawrence Erlbaum Associates.
Souvignier, E., Förster, N. & Salaschek, M. (2014). quop: ein Ansatz internet-basierter Lernverlaufsdiagnostik und Testkonzepte für Mathematik und Lesen. In M. Hasselhorn, W. Schneider & U. Trautwein (Hrsg.), Lernverlaufsdiagnostik: Test und Trends N. F. (Bd. 12, S. 239–256). Göttingen: Hogrefe.
Tien, J. M. (2013). Big data: Unleashing information. Journal of Systems Science and Systems Engineering, 22, 127–151.
Upsing, B., Goldhammer, F., Schnitzler, M., Baumann, R., Johannes, R., Barkow, I., Rölke, H., Jars, I., Latour, T., Plichart P., Jadoul, R., Henry, C. &, Wagner, M. (2013). Chapter 5: Development of the Cognitive Items. In Organisation for Economic Co-Operation and Development (OECD) (Ed.), Technical Report of the Survey of Adult Skills (PIAAC) (p. 148–156). Paris: OECD.
Vale, C. D. (2006). Computerized item banking. In S. M. Downing & T. M. Haladyna (Eds.), Handbook of test development (pp. 261–285). Mahwah, NJ: Erlbaum.
van der Linden, W. J. (1998). Optimal assembly of psychological and educational tests. Applied Psychological Measurement, 22, 195–211.
van der Linden, W. J. (2002). On complexity in CBT. In C. Mills, M. Potenza, J. Fremer & W. Ward (Eds.), Computer-Based Testing (pp. 89–102). Mahwah, NJ: Lawrence Erlbaum Associates.
van der Linden, W. J. (2005). Linear models of optimal test design. New York, NY: Springer.
VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46, 197–221.
Visible Learning Lab (2010). Educator manual. E-asTTle fitness for national standards. Retrieved from http://e-asttle.tki.org.nz/User-manuals [20.12.2019]
Waalkens, M., Aleven, V. & Taatgen, N. (2013). Does supporting multiple student strategies lead to greater learning and motivation? Investigating a source of complexity in the architecture of intelligent tutoring systems. Computers and Education, 60, 159–171.
Wenzel, S. F. C., Engelhardt, L., Hartig, K., Kuchta, K., Frey, A., Goldhammer, F., Naumann, J. & Horz, H. (2016). Computergestützte, adaptive und verhaltensnahe Erfassung Informations- und Kommunikationstechnologie-bezogener Fertigkeiten (ICT-Skills) (CavE-ICT). In BMBF (Hrsg.). Forschung in Ankopplung an Large-Scale Assessments (S. 161–180). Bonn, Berlin: BMBF.
Williamson, D. M., Mislevy, R. J. & Bejar, I. I. (Eds.). (2006). Automated scoring of complex tasks in computer-based testing. Mahwah, NJ: Lawrence Erlbaum Associates.
Wise, S. L., Finney, S. J., Enders, C. K., Freeman, S. A. & Severance, D. D. (1999). Examinee Judgments of Changes in Item Difficulty: Implications for Item Review in Computerized Adaptive Testing. Applied Measurement in Education, 12, 185–198.
Zehner, F., Sälzer, C. & Goldhammer, F. (2016). Automatic Coding of Short Text Responses via Clustering in Educational Assessment. Educational and Psychological Measurement, 76, 280–303.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2020 Springer-Verlag GmbH Deutschland, ein Teil von Springer Nature
About this chapter
Cite this chapter
Goldhammer, F., Kröhne, U. (2020). Computerbasiertes Assessment. In: Moosbrugger, H., Kelava, A. (eds) Testtheorie und Fragebogenkonstruktion. Springer, Berlin, Heidelberg. https://doi.org/10.1007/978-3-662-61532-4_6
Download citation
DOI: https://doi.org/10.1007/978-3-662-61532-4_6
Published:
Publisher Name: Springer, Berlin, Heidelberg
Print ISBN: 978-3-662-61531-7
Online ISBN: 978-3-662-61532-4
eBook Packages: Psychology (German Language)