Advertisement

DEVELOPMENT, EVALUATION, AND VALIDATION OF A PAPER-AND-PENCIL TEST FOR MEASURING TWO COMPONENTS OF BIOLOGY TEACHERS’ PEDAGOGICAL CONTENT KNOWLEDGE CONCERNING THE “CARDIOVASCULAR SYSTEM”

  • Stephan Schmelzing
  • Jan H. van Driel
  • Melanie Jüttner
  • Stefanie Brandenbusch
  • Angela Sandmann
  • Birgit J. NeuhausEmail author
Article

ABSTRACT

One main focus of teacher education research concentrates on teachers’ pedagogical content knowledge (PCK). It has been shown that teachers’ PCK correlates with teaching effectiveness as well as with students’ achievement gains. Teachers’ PCK should be analyzed as one of the main important components to evaluate professional development programs. On this account, it is necessary to develop standardized measures of biology teachers’ topic-specific PCK that are labor-efficient. This paper presents a study on the development, evaluation, and validation of a paper-and-pencil test to measure biology teachers’ declarative PCK on the topic of blood and the human cardiovascular system. The development of the test was based, among other considerations, on a review of research literature on PCK and an analysis of 50 videotaped biology lessons. The final test instrument was comprised of 15 items distributed across 2 scales. The findings of the main study—with 93 preservice and in-service biology teachers and 12 biologists—confirmed that this measure of biology teachers’ declarative PCK was reliable, objective, and valid. In-service biology teachers scored higher on the test than preservice teachers (effect size Cohen’s d, 0.65) on one hand and, also, than biologists (Cohen’s d, 1.00) on the other hand. Future versions of this test should explore enlarging the scales and measuring procedural aspects of PCK.

KEY WORDS

pedagogical content knowledge teacher assessment teacher knowledge 

Preview

Unable to display preview. Download preview PDF.

Unable to display preview. Download preview PDF.

References

  1. Abell, S. K. (2008). Twenty years later: Does pedagogical content knowledge remain a useful idea? International Journal of Science Education, 30, 1405–1416.CrossRefGoogle Scholar
  2. Allen, M. J. & Yen, W. M. (2002). Introduction to measurement theory. Long Grove: Waveland.Google Scholar
  3. Anderson, J. R. (1981). Cognitive skills and their acquisition. Hillsdale: Lawrence Erlbaum.Google Scholar
  4. Schmelzing, S. (2010). Das fachdidaktische Wissen von Biologielehrkräften: Konzeptionalisierung, Diagnostik, Struktur und Entwicklung im Rahmen der Biologielehrerbildung. [Pedagogical content knowledge of biology teachers: conceptualization, diagnostics, structure and development within biology teacher education]. Berlin: Logos.Google Scholar
  5. van Driel, J. H., Verloop, N. & de Vos, W. (1998). Developing science teachers' pedagogical content knowledge. Journal of Research in Science Teaching, 35, (6), 673–695.Google Scholar
  6. Schmelzing, S., Wüsten, S., Sandmann, A. & Neuhaus, B. (2008). Evaluation von zentralen Inhalten der Lehrerbildung: Ansätze zur Diagnostik des fachdidaktischen Wissens von Biologielehrkräften. [Evaluation of central aspects of teacher education: Initial stages of diagnosing the pedagogical content knowledge of biology teachers.] Lehrerbildung auf dem Prüfstand, 1, 641–663.Google Scholar
  7. Schmelzing, S., Fuchs, Ch., Wüsten, S., Sandmann, A. & Neuhaus, B. (2009a). Entwicklung und Evaluation eines Instruments zur Erfassung des fachdidaktischen Reflexionswissens von Biologielehrkräften. [Development and Evaluation of an instrument to measure the reflection abilities of biology teachers with regard to pedagogical content knowledge.] Lehrerbildung auf dem Prüfstand, 2, 56–80.Google Scholar
  8. Schmelzing, S., Wüsten, S, Sandmann, A. & Neuhaus, B. (2010a). Measuring declarative and reflective components of biology teachers‘ pedagogical content knowledge. In: M.F. Taşar & G. Çakmakcı (Eds.). Contemporary science education research: preservice and inservice teacher education. Ankara, Turkey: Pegem Akademi, pp. 71–77.Google Scholar
  9. Schmelzing, S., Wüsten, S., Sandmann, A. & Neuhaus, B. (2010b). Fachdidaktisches Wissen und Reflektieren im Querschnitt der Biologielehrerbildung. [Pedagogical content knowledge and reflection in frame of biology teacher education.] Zeitschrift für Didaktik der Naturwissenschaften, 16, 189–207.Google Scholar
  10. Jüttner, M. & Neuhaus, B. J. (2012). Development of items for a pedagogical content knowledge-test based on empirical analysis of students’ errors. International Journal of Science Education, 34, (7), 1125–1143.Google Scholar
  11. Baumert, J., Blum, W. & Neubrand, M. (2004). Drawing the lessons from PISA 2000. Zeitschrift für Erziehungswissenschaft, Beiheft, 3, 143–157.Google Scholar
  12. Baumert, J., Kunter, M., Blum, W., Brunner, M., Voss, T., Jordan, A., Klusmann, U., Tsai, Y. M. et al. (2010). Teachers’ mathematical knowledge, cognitive activation in the classroom, and student progress. American Educational Research Journal, 47, 133–180.Google Scholar
  13. Baxter, J. A. & Lederman, N. G. (1999). Assessment and measurement of pedagogical content knowledge. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge (pp. 147–161). Dordrecht: Kluwer Academic.Google Scholar
  14. Beaton, A. E., Martin, M. O., Mullis, I. V. S., Gonzalez, E. J., Smith, T. A. & Kelly, D. L. (1996). Science achievement in the middle school years: IEA’s Third International Mathematics and Science Study (TIMSS). Chestnut Hill: TIMMS International Study Center, Boston College.Google Scholar
  15. Carlson, R. E. (1990). Assessing teachers’ pedagogical content knowledge: Item development issues. Journal of Personal Evaluation in Education, 4, 157–163.CrossRefGoogle Scholar
  16. Cochran, K. F., King, R. A. & De Ruiter, J. A. (1993). Pedagogical content knowing: An integrative model for teacher preparation. Journal of Teacher Education, 44, 263–272.CrossRefGoogle Scholar
  17. Cohen, J. (1988). Statistical power analysis for the behavioral sciences. New York: Lawrence Erlbaum.Google Scholar
  18. Fenstermacher, G. D. (1994). The knower and the known: The nature of knowledge in research on teaching. In L. Darling-Hammond (Ed.), Review of research in education (20th ed., pp. 3–56). Washington, DC: American Educational Research Association.Google Scholar
  19. Fernandez-Balboa, J.-M. & Stiehl, J. (1995). The generic nature of pedagogical content knowledge among college professors. Teaching and Teacher Education, 11, 293–306.CrossRefGoogle Scholar
  20. Gardner, A. L., & Gess-Newsome, J. (2011, April). A rubric to measure teachers’ knowledge of inquiry-based instruction using three data sources. Paper presented at the NARST Annual Meeting, Orlando.Google Scholar
  21. Geddis, A. N. (1993). Transforming subject-matter knowledge: The role of pedagogical content knowledge in learning to reflect on teaching. International Journal of Science Education, 15, 673–683.CrossRefGoogle Scholar
  22. Gess-Newsome, J. (1999). Pedagogical content knowledge: An introduction and orientation. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge (pp. 3–17). Dordrecht: Kluwer Academic.Google Scholar
  23. Gess-Newsome, J., Cardenas, S., Austin, B. A., Carlson, J., Gardner, A. L., Stuhlsatz, M. A. M., Wilson, C. D. et al. (2011, April). Impact of educative materials and transformative professional development on teachers’ PCK, practice, and student achievement. Paper presented at the NARST Annual Meeting, Orlando.Google Scholar
  24. Grossman, P. L. (1990). The making of a teacher: Teacher knowledge and teacher education. New York: Teachers College Press.Google Scholar
  25. Hashweh, M. (2005). Teacher pedagogical constructions: A reconfiguration of PCK. Teachers and Teaching: Theory and Practice, 11, 237–292.CrossRefGoogle Scholar
  26. Hattie, J. & Cooksey, R. W. (1984). Procedures for assessing the validities of tests using the “known-groups” method. Applied Psychological Measurement, 8, 295–305.CrossRefGoogle Scholar
  27. Heller, J. L., Daehler, K. R., Shinohara, M., & Kaskowitz, S. R. (2004, April). Fostering pedagogical content knowledge about electric circuits through case-based professional development. Paper presented at the NARST Annual Meeting, Vancouver. Retrieved from http://www.wested.org/understandingscience/downloads/ppr_fostped.pdf.
  28. Hill, H. C., Ball, D. L., Blunk, M., Goffney, I. M. & Rowan, B. (2007). Validating the ecological assumption: The relationship of measure scores to classroom teaching and student learning. Measurement: Interdisciplinary Research and Perspectives, 5, 371–406.Google Scholar
  29. Hill, H. C., Loewenberg Ball, D. & Schilling, S. G. (2008). Unpacking pedagogical content knowledge: Conceptualizing and measuring teachers’ topic-specific knowledge of students. Journal for Research in Mathematics Education, 38, 372–400.Google Scholar
  30. Hill, H. C., Rowan, B. & Ball, D. (2005). Effects of teachers’ mathematical knowledge for teaching on student achievement. American Educational Research Journal, 42, 371–406.CrossRefGoogle Scholar
  31. Jatzwauk, P., Rumann, S. & Sandmann, A. (2008). Der Einfluss des Aufgabeneinsatzes im Biologieunterricht auf die Lernleistung der Schüler—Ergebnisse einer Videostudie. [The influence of tasks on the students’ learning performance in biology lessons—Results of a video study]. Zeitschrift für Didaktik der Naturwissenschaften, 14, 263–281.Google Scholar
  32. Kind, V. (2009). Pedagogical content knowledge in science education: Perspectives and potential for progress. Studies in Science Education, 45, 169–204.CrossRefGoogle Scholar
  33. Knight, P. (2002). A systemic approach to professional development: Learning as practice. Teaching and Teacher Education, 18, 229–241.CrossRefGoogle Scholar
  34. Krauss, S., Baumert, J. & Blum, W. (2008). Secondary mathematics teachers’ pedagogical content knowledge and content knowledge: Validation of the COACTIV constructs. The International Journal of Mathematics Education, 40, 873–892.Google Scholar
  35. Kromrey, J. D., & Renfrow, D. D. (1991). Using multiple choice examination items to measure teachers’ content specific pedagogical knowledge. Annual Meeting of the Eastern Educational Research Association, Boston. Retrieved from ERIC database (ED329594).Google Scholar
  36. Lee, E. & Luft, J. A. (2008). Experienced secondary science teachers’ representation of pedagogical content knowledge. International Journal of Science Education, 30, 1343–1363.CrossRefGoogle Scholar
  37. Loughran, J., Berry, A. & Mulhall, P. (2006). Understanding and developing science teachers’ pedagogical content knowledge. Rotterdam: Sense.Google Scholar
  38. Loughran, J., Milroy, P., Berry, A., Gunstone, R. & Mulhall, P. (2001). Documenting science teachers’ pedagogical content knowledge through PaP-eRs. Research in Science Education, 31, 289–307.CrossRefGoogle Scholar
  39. Magnusson, S., Krajcik, J. & Borko, H. (1999). Nature, sources, and development of pedagogical content knowledge. In J. Gess-Newsome & N. G. Lederman (Eds.), Examining pedagogical content knowledge (pp. 95–132). Dordrecht: Kluwer Academic.Google Scholar
  40. Marks, R. (1990). Pedagogical content knowledge: From a mathematical case to a modified conception. Journal of Teacher Education, 41, 3–11.CrossRefGoogle Scholar
  41. Mulhall, P., Berry, A., & Loughran, J. (2003). Frameworks for representing science teachers’ pedagogical content knowledge. Asia-Pacific Forum on Science Learning and Teaching, 4(2), Article 2. Retrieved from http://www.ied.edu.hk/apfslt/v4_issue2/mulhall/index.htm.
  42. OECD (2000). Measuring student knowledge and skills—The PISA 2000 Assessment of Reading Mathematical and Scientific Literacy. Paris: OECD—Organisation for Economic Co-Operation and Development.Google Scholar
  43. Park, S., Jang, J., Chen, Y. & Jung, J. (2011). Is pedagogical content knowledge (PCK) necessary for reformed science teaching? Evidence from an empirical study. Research in Science Education, 41, 254–260.CrossRefGoogle Scholar
  44. Park, S. & Oliver, S. J. (2008). Revisiting the conceptualisation of pedagogical content knowledge (PCK): PCK as a conceptual tool to understand teachers as professionals. Research in Science Education, 38, 261–284.CrossRefGoogle Scholar
  45. Piburn, M., & Sawada, D. (2003). Reformed Teaching Observation Protocol (RTOP): Reference manual (rep. no. IN00-3). Retrieved from http://physicsed.buffalostate.edu/AZTEC/RTOP/RTOP_full/.
  46. Polanyi, M. (1958). Personal knowledge. Chicago: University of Chicago Press.Google Scholar
  47. Rohaan, E. J., Taconis, R. & Jochems, W. M. G. (2009). Measuring teachers’ pedagogical content knowledge in primary biology education. Research in Science and Technological Education, 27, 327–338.CrossRefGoogle Scholar
  48. Ryle, G. (1971). ‘Knowing how and knowing that’ (Proceedings of the Aristotelian Society, 1946). In G. Ryle (Ed.), Collected papers. Volume II: Collected essays, 1929–1968 (pp. 212–225). London: Hutchinson & Co.Google Scholar
  49. Schilling, S. G. & Hill, H. C. (2007). Assessing measures of mathematical knowledge for teaching: A validity argument approach. Measurement: Interdisciplinary Research and Perspectives, 5, 93–106.CrossRefGoogle Scholar
  50. Schmidt, W. H., Tatto, M. T., Bankov, K., Blömeke, S., Cedillo, T., Cogan, L., Schwille, J. et al. (2007). The preparation gap: Teacher education for middle school mathematics in six countries. MT21 report. East Lansing: Michigan State University.Google Scholar
  51. Shrout, P. E. & Fleiss, J. L. (1979). Intraclass correlations: Uses in assessing rater reliability. Psychological Bulletin, 86, 420–428.CrossRefGoogle Scholar
  52. Shulman, L. (1986). Those who understand: Knowledge growth in teaching. Educational Researcher, 15(2), 4–14.CrossRefGoogle Scholar
  53. Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57, 1–22.Google Scholar
  54. Smith, D. C. & Neale, D. C. (1989). The construction of subject matter knowledge in primary science teaching. Teaching and Teacher Education, 5, 1–20.CrossRefGoogle Scholar
  55. Staub, F. C. & Stern, E. (2002). The nature of teachers’ pedagogical content beliefs matters for students’ achievement gains. Quasi-experimental evidence from elementary mathematics. Journal of Educational Psychology, 94, 344–355.CrossRefGoogle Scholar
  56. Stillings, N., Weisler, S. E., Chase, C. H., Feinstein, M. H., Garfield, J. L. & Rissland, E. L. (1995). Cognitive science: An introduction. Cambridge: MIT.Google Scholar
  57. Sungur, S., Tekkaya, C. & Geban, O. (2001). The contribution of conceptual change texts accompanied by concept mapping students understanding of the human circulatory system. School Science and Mathematics: Official Journal of the School Science and Mathematics Association, 101, 1–14.Google Scholar
  58. Tamir, P. (1988). Subject matter and related pedagogical knowledge in teacher education. Teaching and Teacher Education: an Internal Journal of Research and Studies, 4, 99–110.CrossRefGoogle Scholar
  59. Tepner, O., Borowski, A., Dollny, S., Fischer, H. E., Jüttner, M., Kirschner, S., Wirth, J. et al. (2012). Modell zur Entwicklung von Testitems zur Erfassung des Professionswissens von Lehrkräften in den Naturwissenschaften [Item development model for assessing professional knowledge of science teachers]. Zeitschrift für Didaktik der Naturwissenschaften, 18, 7–28.Google Scholar
  60. Tiemann, R., Rumann, S., Jatzwauk, P. & Sandmann, A. (2006). Aufgaben aus Lehrersicht. [Tasks from teachers’ point of view]. Der Mathematische und Naturwissenschaftliche Unterricht, 59, 304–307.Google Scholar
  61. Wadouh, J., Sandmann, A. & Neuhaus, B. J. (2009). Vernetzung im Biologieunterricht—deskriptive Befunde einer Videostudie. [Knowledge linking levels in biology lessons—Descriptive results of a video study]. Zeitschrift für Didaktik der Naturwissenschaften, 15, 69–87.Google Scholar

Copyright information

© National Science Council, Taiwan 2013

Authors and Affiliations

  • Stephan Schmelzing
    • 1
  • Jan H. van Driel
    • 2
  • Melanie Jüttner
    • 3
  • Stefanie Brandenbusch
    • 1
  • Angela Sandmann
    • 4
  • Birgit J. Neuhaus
    • 3
    Email author
  1. 1.Formerly: Research Group and Graduate School “Teaching and Learning Science”University of Duisburg–EssenEssenGermany
  2. 2.ICLON—Leiden University Graduate School of TeachingLeidenNetherlands
  3. 3.Biology Education, Faculty of BiologyLudwig-Maximilians-University MunichMunichGermany
  4. 4.Biology EducationUniversity of Duisburg–EssenEssenGermany

Personalised recommendations