Abstract
Practitioners in health sciences education and assessment regularly use a range of psychometric techniques to analyse data, evaluate models, and make crucial progression decisions regarding student learning. However, a recent editorial entitled “Is Psychometrics Science?” highlighted some core epistemological and practical problems in psychometrics, and brought its legitimacy into question. This paper attempts to address these issues by applying some key ideas from history and philosophy of science (HPS) discourse. I present some of the conceptual developments in HPS that have bearing on the psychometrics debate. Next, by shifting the focus onto what constitutes the practice of science, I discuss psychometrics in action. Some incorrectly conceptualize science as an assemblage of truths, rather than an assemblage of tools and goals. Psychometrics, however, seems to be an assemblage of methods and techniques. Psychometrics in action represents a range of practices using specific tools in specific contexts. This does not render the practice of psychometrics meaningless or futile. Engaging in debates about whether or not we should regard psychometrics as ‘scientific’ is, however, a fruitless enterprise. The key question and focus should be whether, on what grounds, and in what contexts, the existing methods and techniques used by psychometricians can be justified or criticized.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Arabatzis, T., & Schickore, J. (2012). Introduction: Ways of integrating history and philosophy of science. Perspectives on Science, 20(4), 395–408.
Bird, A. (2002). Kuhn’s wrong turning. Studies in History and Philosophy of Science, 33, 443–463.
Bird, A. (2008). The historical turn in the philosophy of science. In S. Psillos & M. Curd (Eds.), Routledge companion to the philosophy of science (pp. 67–77). Abingdon: Routledge.
Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71(3), 425–440.
Burian, R. M. (1977). More than a marriage of convenience: On the inextricability of the history and philosophy of science. Philosophy of Science, 44, 1–42.
Butterfield, H. (1931). The Whig interpretation of history. London: G. Bell.
Callebaut, W. (1993). Taking the naturalist turn, or how real philosophy of science is done. Chicago: University of Chicago Press.
Cano, S. J., & Hobart, J. C. (2011). The problem with health measurement. Patient Preference and Adherence, 5, 279–290.
Feyerabend, P. (1975). Against method. London: Verso.
Forster, M. (2000). Hard problems in the philosophy of science: Idealization and commensurability. In R. Nola & H. Sankey (Eds.), After Popper, Kuhn and Feyerabend: Issues in theories of scientific method (pp. 231–250). Dordrecht: Springer.
Fuller, S. (2003). Kuhn vs Popper: The struggle for the soul of science. Cambridge: Icon Books.
Gieryn, T. (1983). Boundary-work and the demarcation of science from non-science: Strains and interests in professional ideologies of scientists. American Sociological Review, 48(6), 781–795.
Gieryn, T. (1999). Cultural boundariews of sceince: Credibility on the line. Chicago: University of Chicago Press.
Hacking, I. (1975). The emergence of probability. Cambridge: Cambridge University Press.
Hacking, I. (1999). The social construction of what?. Cambridge, MA: Harvard University Press.
Hacking, I. (2002). ‘Style’ for historians and philosophers. Historical ontology (pp. 178–199). Cambridge MA: Harvard University Press.
Hanson, N. R. (1962). The irrelevance of history of science to philosophy of science. The Journal of Philosophy, 59, 574–586.
Hansson, S. O. (2006). Falsificationism falsified. Foundations of Science, 11(3), 275–286.
Hobart, J. C., Cano, S. J., Zajicek, J. P., & Thompson, A. J. (2007). Rating scales as outcome measures for clinical trials in neurology: Problems, solutions, and recommendations. Lancet Neurology, 6, 1094–1105.
Hoyningen-Huene, P. (2006). Context of discovery versus context of justification and Thomas Kuhn. In J. Schickore & F. Steinle (Eds.), Revisiting discovery and justification. Dordrecht: Springer.
Jardine, N. (2000). The scenes of inquiry: On the reality of questions in the sciences (2nd ed.). Oxford: Clarendon Press.
Jones, L. V., & Thissen, D. (2007). A history and overview of psychometrics. Handbook of Statistics. doi:10.1016/S0169-7161(06)26001-2.
Kant, I. (1998 [1781]). Critique of pure reason (T. b. P. G. a. A. Wood, Trans.). Cambridge: Cambridge University Press.
Klein, U. (2003). Experiments, models, paper tools: Cultures of organic chemistry in the nineteenth century. Stanford: Stanford University Press.
Lakatos, I. (1970a). Falsification and the methodology of scientific research programmes. In I. Lakatos & A. Musgrave (Eds.), Criticism and the growth of knowledge (pp. 91–195). Cambridge: Cambridge University Press.
Lakatos, I. (1970b). History of science and its rational reconstructions. In PSA: Proceedings of the biennial meeting of the philosophy of science association (pp. 91–136).
Lakatos, I. (1976). Proofs and refutations. Cambridge: Cambridge University Press.
Latour, B. (1987). Science in action. Cambridge, MA: Harvard University Press.
Mari, L., & Wilson, M. (2014). An introduction to the Rasch measurement approach for metrologists. Measurement, 51, 315–327.
Massof, R. W. (2008). Moving toward scientific measurements of quality of life. Ophthalmic Epidemiology, 15, 209–211.
Masterman, M. (1970). The Nature of a paradigm. In I. Lakatos & A. Musgrave (Eds.), Criticism and the growth of knowledge (pp. 59–89). Cambridge: Cambridge University Press.
Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13–103). New York: Macmillan.
Messick, S. (1995). Standards of validity and the validity of standards in performance assessment. Euducational Measurement: Issues and Practice, 14(4), 5–8.
Michell, J. (2008). Is psychometrics pathalogical science? Measurement: Interdisciplinary Research and Perspectives, 6, 7–24.
Norman, G. (2016). Is psychometrics science? Advances in Health Sciences Education (Theory and Practice), 21, 731–734.
Pearce, J. (2015). Historicizing scale development: Shifting epistemic practices in Rasch measurement. Paper presented at the IMEKO XXI World Congress, Prague. https://www.imeko.org/publications/wc-2015/IMEKO-WC-2015-TC7-194.pdf
Pendrill, L. (2014). Man as a measurement instrument. NCSLI Measure Journal of Measurement Science, 9(4), 24–35.
Pendrill, L., & Fisher, W. P. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, 71, 46–55.
Pesudovs, K. (2010). Item banking: A generational change in patient-reported outcome measurement. Optometry and Vision Science, 87(4), 285–293.
Pickering, A. (1992). From science as knowledge to science as practice. In A. Pickering (Ed.), Science as practice and culture. Chicago: Chicago University Press.
Prelli, L. J. (1989). The rhetorical construction of scientific ethos. Evolution, 34(5), 87–104.
Rasmussen, N. (1997). Picture control: The electron microscope and the transformation of biology in America, 1940–1960. Stanford: Stanford University Press.
Rheinberger, H.-J. (2010). On Historicizing Epistemology: An Essay (D. Fernbach, Trans.): Stanford University Press.
Ritson, S., & Camilleri, K. (2015). Contested boundaries: The string theory debates and ideologies of science. Perspectives on Science, 23(2), 192–227.
Ruse, M. (2009). The Darwinian revolution: Rethinking its meaning and significance. PNAS, 106(1), 10040–10047.
Salmon, W. C. (1970). Bayes’s theorem and the history of science. In R. H. Stuewer (Ed.), Historical and philosophical perspectives on science. Minnesota studies in the philosophy of science 5. Minneapolis: University of Minnesota Press.
Schoenherr, J. R., & Hamstra, S. J. (2016). Psychometrics and its discontents: An historical perspeective on the discourse of the measurement tradition. Advances in Health Sciences Education (Theory and Practice), 21, 719–729.
Sturm, T., & Feest, U. (2011). What (good) is historical epistemology? Erkenntnis, 75, 285–302.
Weinberg, S. (1998). The revolution that didn’t happen. New York Review of Books, 45(15), 48–52.
Wilkins, J. S. (2010). Retrieved from https://thonyc.wordpress.com/2010/11/12/galileo%E2%80%99s-great-bluff-and-part-of-the-reason-why-kuhn-is-wrong/
Wilson, M. (2013). Seeking a balance between the statistical and scientific elements in psychometrics. Psychometrika, 78(2), 211–236.
Acknowledgements
I would like to thank Kristian Camilleri, Neville Chiavaroli and William P Fisher Jr for constructive feedback on an earlier version of this paper, along with the helpful comments made by editors and reviewers during the peer review process. I would also like to acknowledge Kristian Camilleri for his guidance over the years regarding HPS discourse.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Pearce, J. Psychometrics in action, science as practice. Adv in Health Sci Educ 23, 653–663 (2018). https://doi.org/10.1007/s10459-017-9789-7
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10459-017-9789-7