Skip to main content

A Guide for the Perplexed: Scientific Educational Research, Methodolatry, and the Gold Versus Platinum Standards

  • 3309 Accesses

Abstract

The discussion opens by characterizing recent discourse about empirical educational research as the “new Babel” – critics, using different theoretical vocabularies and making different deep assumptions about the nature of social life, are failing to communicate with each other. After locating some of the critical positions on a left-right continuum, the main discussion focuses upon the end of this continuum where there are located the recent attempts to restore rigor in educational research by using the so-called “gold standard” of randomized field trials. It is argued that positions at this end of the continuum misrepresent the nature of science, and some examples are mentioned briefly to convey the point that it is fruitful to view scientists as making convincing cases, cases that appeal to a wide variety of evidence. The assessment of scientific cases is called the “platinum standard”.

Keywords

  • Gold standard
  • Randomized field trials
  • Nature of science
  • NRC report
  • Rigorous educational research

This is a preview of subscription content, access via your institution.

Buying options

Chapter
USD   29.95
Price excludes VAT (USA)
  • DOI: 10.1007/978-94-007-6809-3_16
  • Chapter length: 11 pages
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
eBook
USD   169.00
Price excludes VAT (USA)
  • ISBN: 978-94-007-6809-3
  • Instant PDF download
  • Readable on all devices
  • Own it forever
  • Exclusive offer for individuals only
  • Tax calculation will be finalised during checkout
Softcover Book
USD   219.99
Price excludes VAT (USA)
Hardcover Book
USD   219.99
Price excludes VAT (USA)

Notes

  1. 1.

    For other discussions of the context-dependent nature of human action, see Cronbach (1975) who develops the notion of webs of interactive effects that change over time – hence “generalizations decay”, and Labaree (1998) who makes the point that humans can often act so as to contradict any generalization that is made about them – hence researchers must live with a “lesser form of knowledge”. Neither of these authors seem to have held serious doubts that educational research is both possible and useful; instead they hold that it must have modest aspirations.

  2. 2.

    In the spirit of full disclosure the present author must admit to being a member of the Committee that authored this report.

  3. 3.

    Various attempts to legislate about scientific rigor are detailed in Eisenhart and Towne (2003).

  4. 4.

    Boruch was also a member of the NRC committee, where not surprisingly he was a strong advocate for the use of RFTs.

  5. 5.

    Lee Shulman recently discussed the different conclusions reached by three different groups of researchers, all highly competent, with respect to the impact of high stakes testing on students in the USA; see Shulman (2005).

  6. 6.

    They also argue, not unreasonably, that the RFT allows the making of good estimates of effect size, a matter that will not be pursued here.

  7. 7.

    Fuller discussion, and examples of each, can be found in Phillips (2005b).

  8. 8.

    Achinstein (2001) gives a detailed discussion of factors of this sort, and shows that the notion of evidence can be relativized to the epistemic situation of the scientist who accepts it, without destroying the objectivity that is so important in the concept of evidence.

References

  • Achinstein P (2001) The book of evidence. Oxford University Press, Oxford

    CrossRef  Google Scholar 

  • Booth W, Colomb G, Williams J (2003) The craft of research, 2nd edn. University of Chicago Press, Chicago

    CrossRef  Google Scholar 

  • Boruch R, de Moya D, Snyder B (2002) The importance of randomized field trials in education and related areas. In: Mosteller F, Boruch R (eds) Evidence matters. Brookings Institution Press, Washington, DC, pp 50–79

    Google Scholar 

  • Boruch R, Mosteller F (2002) Overview and new directions. In: Boruch R, Mosteller F (eds) Evidence matters: randomized trials in educational research. Brookings Institution, Washington, DC, pp 1–14

    Google Scholar 

  • Bryk A, Lee V, Holland P (1993) Catholic schools and the common good. Harvard University Press, Cambridge, MA

    Google Scholar 

  • Carr D (2003) Making sense of education. Routledge-Falmer, London/New York

    Google Scholar 

  • Cronbach L (1975) Beyond the two disciplines of scientific psychology. Am Psychol 30(2):116–127

    CrossRef  Google Scholar 

  • Dewey J (1938/1966) Logic: the theory of inquiry. Holt, Rinehart and Winston, New York

    Google Scholar 

  • Educational Researcher (2002) Theme issue on. Sci Res Educ 31(8):3–29

    Google Scholar 

  • Educational Theory (2005) The education science question: a symposium. Educ Theory 55(3):235–322

    Google Scholar 

  • Eisenhart M, DeHaan R (2005) Doctoral preparation of scientifically based education researchers. Educational Res 34(4):3–13

    Google Scholar 

  • Eisenhart M, Towne L (2003) Contestation and change in national policy on ‘scientifically based’ education research. Educational Res 32(7):31–38

    Google Scholar 

  • Elster J (1989) Nuts and bolts for the social sciences. Cambridge University Press, Cambridge, UK

    CrossRef  Google Scholar 

  • Fay B (1996) Contemporary philosophy of social science: a multicultural approach. Blackwell, Oxford

    Google Scholar 

  • Feuer M, Towne L, Shavelson R (2002) Scientific culture and educational research. Educational Res 31(8):4–14

    Google Scholar 

  • Flyvbjerg B (2001) Making social science matter. Cambridge University Press, Cambridge

    Google Scholar 

  • Goldman A (1994) Argumentation and social epistemology. J Philos 91(1):27–49

    CrossRef  Google Scholar 

  • Hargreaves D (1997) In defence of research for evidence-based teaching. Brit Educ Res J 23(4):405–419

    CrossRef  Google Scholar 

  • Heath SB (1983) Ways with words: language, life and work in communities and classrooms. Cambridge University Press, New York

    Google Scholar 

  • Kaplan A (1964) The conduct of inquiry. Chandler, Scranton

    Google Scholar 

  • Kuhn T (1970) The structure of scientific revolutions. University of Chicago Press, Chicago

    Google Scholar 

  • Labaree D (1998) Educational researchers: living with a lesser form of knowledge. Educational Res 27(8):4–12

    Google Scholar 

  • Lather P (2004) This IS your father’s paradigm: government intrusion and the case of qualitative research in education. Qual Inq 10(1):15–34

    CrossRef  Google Scholar 

  • Levitt S, Dubner S (2005) Freakonomics. William Morrow, New York

    Google Scholar 

  • Lyotard J-F (1984) The postmodern condition: a report on knowledge. Manchester University Press, Manchester

    Google Scholar 

  • Michaels D (2005) Doubt is their product. Sci Am 292(6):96–101

    CrossRef  Google Scholar 

  • Mosteller F, Boruch R (eds) (2002) Evidence matters. Brookings Institution Press, Washington, DC

    Google Scholar 

  • National Research Council (2002) Scientific research in education. National Academies Press, Washington, DC

    Google Scholar 

  • National Research Council (2005) Advancing scientific research in education. National Academies Press, Washington, DC

    Google Scholar 

  • Nelson J, Megill A, McCloskey D (eds) (1987) The rhetoric of the human sciences. University of Wisconsin Press, Madison

    Google Scholar 

  • Olssen M (2004) The school as the microscope of conduction: doing Foucauldian research in education. In: Marshall J (ed) Poststructuralism, philosophy, pedagogy. Kluwer, Dordrecht, pp 57–84

    Google Scholar 

  • Phillips D (2000) The expanded social scientist’s bestiary. Rowman and Littlefield, Lanham

    Google Scholar 

  • Phillips D (2005a) The contested nature of empirical educational research (and why philosophy of education offers little help). J Philos Educ 39(4):1–21

    CrossRef  Google Scholar 

  • Phillips D (2005b) Muddying the waters: the many purposes of educational inquiry. In: Conrad C, Serlin R (eds) SAGE handbook for research in education: engaging ideas and enriching inquiry. Sage, Thousand Oaks, pp 7–21

    Google Scholar 

  • Phillips D, Burbules N (2000) Postpositivism and educational research. Rowman and Littlefield, Lanham

    Google Scholar 

  • Qualitative Inquiry (2004) Symposium on scientific educational research. Qual Inq 10(1):5–129

    CrossRef  Google Scholar 

  • Shulman L (2005) Seek simplicity … and distrust it. Educ Week 24(39):36–48

    Google Scholar 

  • St. Pierre E (2002) “Science” rejects postmodernism. Educational Res 31(8):25–27

    Google Scholar 

  • Teachers College Record (2005) A symposium on the implications of the scientific research in education report for qualitative inquiry. Teach Coll Rec 107(1):1–58

    Google Scholar 

  • Toulmin S (2003) The uses of argument. Cambridge University Press, Cambridge

    CrossRef  Google Scholar 

Download references

Acknowledgements

This chapter is a modified version of Educational Research Review, 1 (1) 2006, 15–26.

Author information

Authors and Affiliations

Authors

Corresponding author

Correspondence to D. C. Phillips .

Editor information

Editors and Affiliations

Rights and permissions

Reprints and Permissions

Copyright information

© 2014 Springer Science+Business Media Dordrecht

About this chapter

Cite this chapter

Phillips, D.C. (2014). A Guide for the Perplexed: Scientific Educational Research, Methodolatry, and the Gold Versus Platinum Standards. In: Reid, A., Hart, E., Peters, M. (eds) A Companion to Research in Education. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-6809-3_16

Download citation