Advertisement

Advances in Health Sciences Education

, Volume 23, Issue 3, pp 653–663 | Cite as

Psychometrics in action, science as practice

  • Jacob PearceEmail author
Reflections

Abstract

Practitioners in health sciences education and assessment regularly use a range of psychometric techniques to analyse data, evaluate models, and make crucial progression decisions regarding student learning. However, a recent editorial entitled “Is Psychometrics Science?” highlighted some core epistemological and practical problems in psychometrics, and brought its legitimacy into question. This paper attempts to address these issues by applying some key ideas from history and philosophy of science (HPS) discourse. I present some of the conceptual developments in HPS that have bearing on the psychometrics debate. Next, by shifting the focus onto what constitutes the practice of science, I discuss psychometrics in action. Some incorrectly conceptualize science as an assemblage of truths, rather than an assemblage of tools and goals. Psychometrics, however, seems to be an assemblage of methods and techniques. Psychometrics in action represents a range of practices using specific tools in specific contexts. This does not render the practice of psychometrics meaningless or futile. Engaging in debates about whether or not we should regard psychometrics as ‘scientific’ is, however, a fruitless enterprise. The key question and focus should be whether, on what grounds, and in what contexts, the existing methods and techniques used by psychometricians can be justified or criticized.

Keywords

History of science HPS Philosophy of science Psychometrics Rasch Science Scientific Science as practice Science in action 

Notes

Acknowledgements

I would like to thank Kristian Camilleri, Neville Chiavaroli and William P Fisher Jr for constructive feedback on an earlier version of this paper, along with the helpful comments made by editors and reviewers during the peer review process. I would also like to acknowledge Kristian Camilleri for his guidance over the years regarding HPS discourse.

References

  1. Arabatzis, T., & Schickore, J. (2012). Introduction: Ways of integrating history and philosophy of science. Perspectives on Science, 20(4), 395–408.CrossRefGoogle Scholar
  2. Bird, A. (2002). Kuhn’s wrong turning. Studies in History and Philosophy of Science, 33, 443–463.CrossRefGoogle Scholar
  3. Bird, A. (2008). The historical turn in the philosophy of science. In S. Psillos & M. Curd (Eds.), Routledge companion to the philosophy of science (pp. 67–77). Abingdon: Routledge.Google Scholar
  4. Borsboom, D. (2006). The attack of the psychometricians. Psychometrika, 71(3), 425–440.CrossRefGoogle Scholar
  5. Burian, R. M. (1977). More than a marriage of convenience: On the inextricability of the history and philosophy of science. Philosophy of Science, 44, 1–42.CrossRefGoogle Scholar
  6. Butterfield, H. (1931). The Whig interpretation of history. London: G. Bell.Google Scholar
  7. Callebaut, W. (1993). Taking the naturalist turn, or how real philosophy of science is done. Chicago: University of Chicago Press.Google Scholar
  8. Cano, S. J., & Hobart, J. C. (2011). The problem with health measurement. Patient Preference and Adherence, 5, 279–290.CrossRefGoogle Scholar
  9. Feyerabend, P. (1975). Against method. London: Verso.Google Scholar
  10. Forster, M. (2000). Hard problems in the philosophy of science: Idealization and commensurability. In R. Nola & H. Sankey (Eds.), After Popper, Kuhn and Feyerabend: Issues in theories of scientific method (pp. 231–250). Dordrecht: Springer.CrossRefGoogle Scholar
  11. Fuller, S. (2003). Kuhn vs Popper: The struggle for the soul of science. Cambridge: Icon Books.Google Scholar
  12. Gieryn, T. (1983). Boundary-work and the demarcation of science from non-science: Strains and interests in professional ideologies of scientists. American Sociological Review, 48(6), 781–795.CrossRefGoogle Scholar
  13. Gieryn, T. (1999). Cultural boundariews of sceince: Credibility on the line. Chicago: University of Chicago Press.Google Scholar
  14. Hacking, I. (1975). The emergence of probability. Cambridge: Cambridge University Press.Google Scholar
  15. Hacking, I. (1999). The social construction of what?. Cambridge, MA: Harvard University Press.Google Scholar
  16. Hacking, I. (2002). ‘Style’ for historians and philosophers. Historical ontology (pp. 178–199). Cambridge MA: Harvard University Press.Google Scholar
  17. Hanson, N. R. (1962). The irrelevance of history of science to philosophy of science. The Journal of Philosophy, 59, 574–586.CrossRefGoogle Scholar
  18. Hansson, S. O. (2006). Falsificationism falsified. Foundations of Science, 11(3), 275–286.CrossRefGoogle Scholar
  19. Hobart, J. C., Cano, S. J., Zajicek, J. P., & Thompson, A. J. (2007). Rating scales as outcome measures for clinical trials in neurology: Problems, solutions, and recommendations. Lancet Neurology, 6, 1094–1105.CrossRefGoogle Scholar
  20. Hoyningen-Huene, P. (2006). Context of discovery versus context of justification and Thomas Kuhn. In J. Schickore & F. Steinle (Eds.), Revisiting discovery and justification. Dordrecht: Springer.Google Scholar
  21. Jardine, N. (2000). The scenes of inquiry: On the reality of questions in the sciences (2nd ed.). Oxford: Clarendon Press.CrossRefGoogle Scholar
  22. Jones, L. V., & Thissen, D. (2007). A history and overview of psychometrics. Handbook of Statistics. doi: 10.1016/S0169-7161(06)26001-2.Google Scholar
  23. Kant, I. (1998 [1781]). Critique of pure reason (T. b. P. G. a. A. Wood, Trans.). Cambridge: Cambridge University Press.Google Scholar
  24. Klein, U. (2003). Experiments, models, paper tools: Cultures of organic chemistry in the nineteenth century. Stanford: Stanford University Press.Google Scholar
  25. Lakatos, I. (1970a). Falsification and the methodology of scientific research programmes. In I. Lakatos & A. Musgrave (Eds.), Criticism and the growth of knowledge (pp. 91–195). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  26. Lakatos, I. (1970b). History of science and its rational reconstructions. In PSA: Proceedings of the biennial meeting of the philosophy of science association (pp. 91–136).Google Scholar
  27. Lakatos, I. (1976). Proofs and refutations. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  28. Latour, B. (1987). Science in action. Cambridge, MA: Harvard University Press.Google Scholar
  29. Mari, L., & Wilson, M. (2014). An introduction to the Rasch measurement approach for metrologists. Measurement, 51, 315–327.CrossRefGoogle Scholar
  30. Massof, R. W. (2008). Moving toward scientific measurements of quality of life. Ophthalmic Epidemiology, 15, 209–211.CrossRefGoogle Scholar
  31. Masterman, M. (1970). The Nature of a paradigm. In I. Lakatos & A. Musgrave (Eds.), Criticism and the growth of knowledge (pp. 59–89). Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  32. Messick, S. (1989). Validity. In R. L. Linn (Ed.), Educational measurement (pp. 13–103). New York: Macmillan.Google Scholar
  33. Messick, S. (1995). Standards of validity and the validity of standards in performance assessment. Euducational Measurement: Issues and Practice, 14(4), 5–8.CrossRefGoogle Scholar
  34. Michell, J. (2008). Is psychometrics pathalogical science? Measurement: Interdisciplinary Research and Perspectives, 6, 7–24.Google Scholar
  35. Norman, G. (2016). Is psychometrics science? Advances in Health Sciences Education (Theory and Practice), 21, 731–734.CrossRefGoogle Scholar
  36. Pearce, J. (2015). Historicizing scale development: Shifting epistemic practices in Rasch measurement. Paper presented at the IMEKO XXI World Congress, Prague. https://www.imeko.org/publications/wc-2015/IMEKO-WC-2015-TC7-194.pdf
  37. Pendrill, L. (2014). Man as a measurement instrument. NCSLI Measure Journal of Measurement Science, 9(4), 24–35.CrossRefGoogle Scholar
  38. Pendrill, L., & Fisher, W. P. (2015). Counting and quantification: Comparing psychometric and metrological perspectives on visual perceptions of number. Measurement, 71, 46–55.CrossRefGoogle Scholar
  39. Pesudovs, K. (2010). Item banking: A generational change in patient-reported outcome measurement. Optometry and Vision Science, 87(4), 285–293.Google Scholar
  40. Pickering, A. (1992). From science as knowledge to science as practice. In A. Pickering (Ed.), Science as practice and culture. Chicago: Chicago University Press.CrossRefGoogle Scholar
  41. Prelli, L. J. (1989). The rhetorical construction of scientific ethos. Evolution, 34(5), 87–104.Google Scholar
  42. Rasmussen, N. (1997). Picture control: The electron microscope and the transformation of biology in America, 1940–1960. Stanford: Stanford University Press.Google Scholar
  43. Rheinberger, H.-J. (2010). On Historicizing Epistemology: An Essay (D. Fernbach, Trans.): Stanford University Press.Google Scholar
  44. Ritson, S., & Camilleri, K. (2015). Contested boundaries: The string theory debates and ideologies of science. Perspectives on Science, 23(2), 192–227.CrossRefGoogle Scholar
  45. Ruse, M. (2009). The Darwinian revolution: Rethinking its meaning and significance. PNAS, 106(1), 10040–10047.CrossRefGoogle Scholar
  46. Salmon, W. C. (1970). Bayes’s theorem and the history of science. In R. H. Stuewer (Ed.), Historical and philosophical perspectives on science. Minnesota studies in the philosophy of science 5. Minneapolis: University of Minnesota Press.Google Scholar
  47. Schoenherr, J. R., & Hamstra, S. J. (2016). Psychometrics and its discontents: An historical perspeective on the discourse of the measurement tradition. Advances in Health Sciences Education (Theory and Practice), 21, 719–729.CrossRefGoogle Scholar
  48. Sturm, T., & Feest, U. (2011). What (good) is historical epistemology? Erkenntnis, 75, 285–302.CrossRefGoogle Scholar
  49. Weinberg, S. (1998). The revolution that didn’t happen. New York Review of Books, 45(15), 48–52.Google Scholar
  50. Wilson, M. (2013). Seeking a balance between the statistical and scientific elements in psychometrics. Psychometrika, 78(2), 211–236.CrossRefGoogle Scholar

Copyright information

© Springer Science+Business Media B.V. 2017

Authors and Affiliations

  1. 1.Assessment and Psychometric ResearchAustralian Council for Educational ResearchCamberwellAustralia
  2. 2.History and Philosophy of Science, School of Historical and Philosophical StudiesUniversity of MelbourneParkvilleAustralia

Personalised recommendations