This chapter considers the notion of educational research quality and evaluation from an international perspective. We consider how and why these approaches differ from the US-based Gold Standard design (i.e., research based on randomized controlled trials [RCTs] mimicking third-stage drug trials; see Shelley, Yore, & Hand, Chap. 1). The Gold Standard is based on the assumption that RCT design alone, regardless of other factors, provides the desired quality.
We suggest here that the notion of quality in research and the mechanisms used to evaluate research quality are highly dependent on the overarching aim of education. To illustrate, the governments of many countries see education, especially science education, as a key component in economic progress and as a means of delivering on social services (Coll & Taylor, 2008). Hence, there are a number of reasons why we need to evaluate research quality in science education. We need to provide evidence that our science education regimes (and vocational education and training) do in fact produce outputs in terms of qualified people needed to drive economic success. There is then the notion of accountability; the expenditure of taxpayer monies by government—especially in the area of education—is subject to much public scrutiny and often to criticism. There also is accountability to specific legalization in which governments require the education sector to deliver on education aims, such as scientific literacy. In each of these examples, we need to be as sure as we can that the research findings are trustworthy—to use Guba and Lincoln's (1989) term—or believable. New curricula and teaching and learning approaches often prove highly controversial (e.g., Bell, Jones, & Carr, 1995; Coll & Taylor; Matthews, 1994), and education stakeholders—including government—naturally want to see convincing evidence that costly educational interventions actually work.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Preview
Unable to display preview. Download preview PDF.
References
Antman, E. M., Lau, J., Kupelnick, B., Mosteller, F.,&Chalmers, T. C. (1992). A comparison of results of meta-analyses of randomized control trials and recommendations of clinical experts: Treatments for myocardial infarction. Journal of the American Medical Association, 268(2), 240–248.
Bell, B., Jones, A.,&Carr, M. (1995). The development of the recent national New Zealand science curriculum. Studies in Science Education, 26(1), 73–105.
Brazil Conselho Nacional de Desenvolvimento Científico e Tecnológico [National Council of Scientific and Technological Development]. (n.d.). Homepage. Retrieved July 4, 2008, from http://www.cnpq.br/english/cnpq/index.htm
Brazil Coordenação de Aperfeiçoamento de Pessoal de Nível Superior [Coordination for Improvement of University Level Staff]. (2006). Homepage. Retrieved April 17, 2008, from http://www.capes.gov.br/ [in Portuguese]
Cheng, Y.-J. (2006, Fall). Enhancing the quality of group proposals in science education. Paper presented at the annual workshops for Writing NSC Project Proposals held in several universities in Taiwan.
Coll, R. K.,&Taylor, N. (Eds.) (2008). Science education in context: An international examination of the influence of context on science curricula development and implementation. Rotterdam, The Netherlands: Sense.
Excluding the many for the sake of the few. (2007, February 2). The Times Higher Education Supplement, p. 12. Available from http://www.timeshighereducation.co.uk/story.asp?sectionco de=26&storycode=207676)
Guba, E. G.,&Lincoln, Y. S. (1989). Fourth generation evaluation. Newbury Park, CA: Sage.
Ioannidis, J. P. A. (2005). Why most published research findings are false. Public Library of Science Medicine, 2(8), e124. Retrieved from http://dx.doi.org/10.1371%2Fjournal. pmed.0020124
Johnson, B.,&Cross, M. (2004). Academic leadership under siege: Possibilities and limits of executive deanship. South African Journal of Higher Education, 18(2), 34–58.
Koh, T. S., Tan, K. C. D.,&Cheah, H. M. (2008). Science education in Singapore: Meeting the challenges ahead. In R. K. Coll&N. Taylor (Eds.), Science education in context: An international examination of the influence of context on science curricula development and implementation (pp. 283–290). Rotterdam, The Netherlands: Sense.
Lane, B. (2007, August 1). Our papers have influence. The Australian Higher Education Section, p. 22. Lipsett, A. (2007, February 2). Casualties rise even before the RAE starts. The Times Higher Education Supplement, p. 6. Available from http://www.timeshighereducation.co.uk/story.asp?sectioncode=26&storycode=207665)
Mather, J. (2001). RQF blamed for spike in discovery applications. Campus Review, 17(40), 1–14.
Matthews, M. R. (Ed.). (1994). Science teaching: The role of history and philosophy of science. New York: Routledge.
New Zealand Tertiary Education Commission. (2004). Performance-based research fund (PBRF) — Evaluating research excellence—The 2003 assessment. Retrieved April 17, 2008, from http://www.tec.govt.nz/templates/standard.aspx?id=589
New Zealand Tertiary Education Commission. (2007). Development of the performance-based research fund. Retrieved April 29, 2008, from http://www.minedu.govt.nz/index.cfm?layout= document&documentid=7503
New Zealand Tertiary Education Commission. (n.d.-a). PBRF Quality evaluation 2006 — Questions and answers? Retrieved April 29, 2008, from http://www.tec.govt.nz/templates/ StandardSummary.aspx?id=1206
New Zealand Tertiary Education Commission. (n.d.-b). Performance-Based Research Fund — Evaluating research excellence — The 2006 assessment. Retrieved April 29, 2008, from http://www.tec.govt.nz/upload/downloads/pbrf-full-report-2006.pdf
No Child Left Behind Act of 2001. Pub. L. No. 107–110, 115 Stat. 1425. (2002).
Roberts, G. (2003). Review of research assessment. Report by Sir Gareth Roberts to the UK funding bodies. Bristol, Avon, UK: Higher Education Funding Council for England.
Shulman, L. S. (1997). Disciplines of inquiry in education: A new overview. In R. M. Jaeger (Ed.), Complementary methods for research in education (2nd edn., pp. 3–31). Washington, DC: American Educational Research Association.
Singapore Centre for Research in Pedagogy and Practice. (n.d.). Core research program. Retrieved April 17, 2008, from http://www.crpp.nie.edu.sg/course/view.php?id=306
Singapore Learning Sciences Lab. (2006). Research overview. Retrieved April 17, 2008, from http://lsl.nie.edu.sg/overview.htm
Singapore Ministry of Education. (2006). Education in Singapore. Retrieved April 18, 2008, from http://www.moe.gov.sg/about/files/moe_corporate_brochure_2006.pdf
Singapore Ministry of Finance. (2007). Budget highlights financial year 2007: Ready for the future, ready for the world. Retrieved April 17, 2008, from http://www.mof.gov.sg/budget_2007/budget_ speech/downloads/FY2007_Budget_Highlights.pdf
Singapore National Research Foundation. (2007). Call for proposals. Retrieved April 17, 2008, from https://rita.nrf.gov.sg/IDM/default.aspx
South Africa Human Sciences Research Council. (n.d.). Homepage. Retrieved July 4, 2008, from http://www.hsrc.ac.za/index.phtml
South Africa National Research Foundation. (n.d.-a). Evaluation and rating facts and figures.Retrieved April 20, 2008, from http://evaluation.nrf.ac.za/Content/Facts/factsfigures.htm
South Africa National Research Foundation. (n.d.-b). Homepage. Retrieved April 17, 2008, from http://www.nrf.ac.za/
South Africa National Research Foundation. (n.d.-c). The NRF focus area programme. Retrieved April 20, 2008, from http://www.nrf.ac.za/focusareas/
South Africa National Research Foundation. (n.d.-d). NRF key performance indicator report 2003/04. Retrieved April 17, 2008, from http:www.nrf.ac.za/publications/annrep/04/intro.htm
South African Government Gazette. (2003). Policy and procedures for measurement of research output of public higher education institutions. Retrieved April 17, 2008, from http://www.uct. ac.za/downloads/uct.ac.za/research/office/pubcount/307.pdf
Stewart, P. (2007). Re-envisioning the academic profession in the shadow of corporate manageri-alism. Journal of Higher Education in Africa, 5(1), 131–147.
Taiwan Department of Science Education. (2006). Mission. Retrieved April 17, 2008, from http://www.nsc.gov.tw/sci/mp.asp?mp=1
Taiwan National Science Council. (2006). Request for proposals. Retrieved April 17, 2008, from http://www.nsc.gov.tw/sci/lp.asp?ctNode=1622&CtUnit=1224&BaseDSD=7 [in Chinese]
Taiwan National Science Council. (n.d.-a). Homepage. Retrieved April 17, 2008, from http://web.nsc.gov.tw/default.asp?mp=7
Taiwan National Science Council. (n.d.-b). Organization. Retrieved June 20, 2008, from http://web.nsc.gov.tw/ct.asp?xItem=14932&CtNode=3417
United Kingdom Research Assessment Exercise. (2001). What is RAE 2001? Retrieved April 17, 2008, from http://www.hero.ac.uk/rae/AboutUs/
United Kingdom Research Assessment Exercise. (2006). Panel criteria and working methods: Panel K. Retrieved April 17, 2008, from http://www.rae.ac.uk/pubs/2006/01/docs/k45.pdf
United Kingdom Research Assessment Exercise. (n.d.-a). History of the RAE. Retrieved April 17, 2008, from http://www.rae.ac.uk/aboutus/history.asp
United Kingdom Research Assessment Exercise. (n.d.-b). Making a submission to the RAE 2008. Retrieved April 17, 2008, from http://www.rae.ac.uk/aboutus/subs.asp
United Kingdom Research Assessment Exercise. (n.d.-c). Panel members for the 2008 RAE. Retrieved April 17, 2008, from http://www.rae.ac.uk/panels/members/
United Kingdom Research Assessment Exercise. (n.d.-d). Panels. Retrieved April 17, 2008, from http://www.rae.ac.uk/panels/
United Kingdom Research Assessment Exercise. (n.d.-e). Quality profiles. Retrieved April 14, 2008, from http://www.rae.ac.uk/aboutus/quality.asp
United Kingdom Research Assessment Exercise. (n.d.-f). RAE 2008: Research assessment exercise. Retrieved April 17, 2008, from http://www.rae.ac.uk/
Author information
Authors and Affiliations
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2009 Springer Science + Business Media B.V
About this chapter
Cite this chapter
Coll, R.K. et al. (2009). An International Perspective of Monitoring Educational Research Quality: Commonalities and Differences. In: Shelley, M.C., Yore, L.D., Hand, B. (eds) Quality Research in Literacy and Science Education. Springer, Dordrecht. https://doi.org/10.1007/978-1-4020-8427-0_6
Download citation
DOI: https://doi.org/10.1007/978-1-4020-8427-0_6
Publisher Name: Springer, Dordrecht
Print ISBN: 978-1-4020-8426-3
Online ISBN: 978-1-4020-8427-0
eBook Packages: Humanities, Social Sciences and LawEducation (R0)