Abstract
This paper provides an example of the application of the cognitive interview, a qualitative tool for pre-testing a survey instrument to check its cognitive validity, that is, whether the items mean to respondents what they mean to the item designers. The instrument is the survey used in the final phase of the International Study of Principal Preparation (ISPP http://www.ucalgary.ca/~cwebber/ISPP/index.htm). This study involving researchers in 13 countries investigates those aspects of principals’ work perceived by them to be most challenging in their early years in the position and the extent to which principals believe they were prepared for these challenges. Both probing and think aloud approaches to the cognitive interview were used and revealed a small number of items requiring amendment.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Ackermann, A.C., & Blair, J. (2006). Efficient respondent selection for cognitive interviewing. In Proceedings of the Survey Research Methods Section, American Statistical Association. http://www.amstat.org/sections/srms/Proceedings/y2006f.html. Accessed 21 August 2008.
Andrich, D. (1988). Rasch models for measurement. Sage University Series on Quantitative Approaches to the Social Sciences, pp. 07–068. Beverley Hills & London: Sage.
Andrich, D., Sheridan, B. S., & Luo, G. (2005). RUMM2020: Rasch Unidimensional Models for Measurement. Perth, Western Australia: RUMM Laboratory.
Blair, J., Conrad, F., Ackermann, A.C., & Claxton, G. (2006). The effect of sample size on cognitive interview findings. In Proceedings of the Survey Research Methods Section, American Statistical Association. http://www.amstat.org/sections/srms/Proceedings/y2006f.html. Accessed 21 August 2008.
Clarke, S., Wildy, H., & Pepper, C. (2007). Connecting preparation with reality: Primary principals’ experiences of their first year out in Western Australia. Leading & Managing, 13(1), 81–90.
Collins, D. (2003). Pretesting survey instruments: An overview of cognitive methods. Quality of Life Research, 12(3), 229–238. doi:10.1023/A:1023254226592.
Cowie, M., & Crawford, M. (2007). Principal preparation—still an act of faith? School Leadership & Management, 27(2), 129–146. doi:10.1080/13632430701237198.
Desimone, L. M., & Le Floch, K. C. (2004). Are we asking the right questions: Using cognitive interviews to improve surveys in education research. Educational Evaluation and Policy Analysis, 26(1), 1–22. doi:10.3102/01623737026001001.
Drennan, J. (2003). Cognitive interviewing: Verbal data in the design and pretesting of questionnaires. Journal of Advanced Nursing, 42(1), 52–63. doi:10.1046/j.1365-2648.2003.02579.x.
Ericsson, K. A., & Simon, H. (1984). Protocol analysis, verbal reports as data. Cambridge, MA: MIT.
Karabenick, S., Woolley, M., Friedel, J., Ammon, B., Blazevski, J., Bonney, C., et al. (2007). Cognitive processing of self-report items in educational research: Do they think what we mean? Educational Psychologist, 42(3), 139–151.
Knafl, K., Deatrick, J., Gallo, A., Holcombe, G., Bakitas, M., Dixon, J., et al. (2007). The analysis and interpretation of cognitive interviews for instrument development. Research in Nursing & Health, 30(2), 224–234. doi:10.1002/nur.20195.
Rasch, G. (1960). Probablistic models for some intelligence and attainment tests. (Copenhagen, Danish Institute for Educational Research)). Expanded edition (1980) with foreword and afterword by B.D. Wright. Chicago: The University of Chicago Press.
Rasch, G. (1980). Probablistic models for some intelligence and attainment tests. (Copenhagen, Danish Institute for Educational Research)). Expanded edition (1980) with foreword and afterword by B.D. Wright. Chicago: The University of Chicago Press.
Slater, C. L., Boone, M., Nelson, S., De La Colina, M., Garcia, E., Grimaldo, L., et al. (2007). El Escalafón y el Doble Turno: An international perspective on school director preparation. Journal for Educational Research and Policy Studies, 6(2), 60–90.
Wildy, H., & Clarke, S. (2008a). Charting an arid landscape: The preparation of novice primary principals in Western Australia. School Leadership & Management, 28(5), 469–487. doi:10.1080/13632430802500106.
Wildy, H., & Clarke, S. (2008b). Principals on L-plates: Rear view mirror reflections. Journal of Educational Administration, 46(6), 727–738. doi:10.1108/09578230810908316.
Wildy, H., Clarke, S., & Slater, C. (2007). International perspectives of principal preparation: How does Australia fare? Leading & Managing Special Edition, 13(2), 1–14.
Willis, G. B. (2005). Cognitive interviewing: A tool for improving questionnaire design. Thousand Oaks, CA: Sage.
Wright, B. D. (1999). Fundamental measurement for psychology. In S. E. Embretson & S. C. Herschberger (Eds.), The new rules of measurement, pp. 65–104. New York: Lawrence Erlbaum.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Wildy, H., Clarke, S. Using cognitive interviews to pilot an international survey of principal preparation: A Western Australian perspective. Educ Asse Eval Acc 21, 105–117 (2009). https://doi.org/10.1007/s11092-009-9073-3
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11092-009-9073-3