Skip to main content

Advertisement

SpringerLink
Log in

Communicating the Significance of Research Questions: Insights from Peer Review at a Flagship Journal

  • Published:
International Journal of Science and Mathematics Education Aims and scope Submit manuscript

Abstract

Significant research in science and mathematics education should advance the field’s knowledge and understanding of the teaching and learning of science and mathematics. How, then, should the significance of a research question in science and mathematics education be assessed? And, when disseminating the findings of research, how should the significance of the research questions be communicated? In this study, we analyzed peer reviews to answer these questions. Our analysis revealed the main issues peer reviewers identify about research questions and the ways they are communicated during the dissemination of research. The findings provide insights for new and experienced researchers about communicating the significance of research questions, and they also illustrate how reviewer comments in peer-reviewed journals can provide a window into the field’s frontiers.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. 1This includes manuscripts that were resubmitted for review after the authors had completed major revisions in response to a prior decision of Revise and Resubmit.

  2. 2Manuscripts resubmitted for review after the authors completed major revisions in response to a prior decision of Revise and Resubmit undergo full peer review by three to five reviewers in the same way as new submissions.

  3. 3As noted above in the results, in the initial editorial screening of manuscripts for this journal, a certain percentage of manuscripts are desk rejected because the editor judges that they have serious flaws or because the work does not move the field of research in mathematics education forward in significant ways. These manuscripts are therefore not sent out for external peer review (and thus were not included in our analysis for this paper). Were they to be sent for peer review, our results would likely have included a much larger number of reviewer comments about the significance of the research questions.

References

  • Bornmann, L., Mutz, R., & Daniel, H.-D. (2010). A reliability-generalization study of journal peer reviews: A multilevel meta-analysis of inter-rater reliability and its determinants. PLoS One, 5(12), e14331. https://doi.org/10.1371/journal.pone.0014331.

    Article  Google Scholar 

  • Brownell, W. A., & Moser, H. E. (1949). Meaningful vs. mechanical learning: A study in Grade III subtraction. Durham, England: Duke University Press.

  • Cai, J., Hwang, S., & Robison, V. (2019). Journal for research in mathematics education: Practical guides for promoting and disseminating significant research in mathematics education. In G. Kaiser & N. Presmeg (Eds.), Compendium for early career researchers in mathematics education (pp. 425–442). Cham, Switzerland: Springer.

  • Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., Cirillo, M., . . . Hiebert, J. (2019a). Choosing and justifying robust methods for educational research. Journal for Research in Mathematics Education, 50(4), 342–348. https://doi.org/10.5951/jresematheduc.50.4.0342.

    Article  Google Scholar 

  • Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., Cirillo, M., . . . Hiebert, J. (2019b). Posing significant research questions. Journal for Research in Mathematics Education, 50(2), 114–120. https://doi.org/10.5951/jresematheduc.50.2.0114.

    Article  Google Scholar 

  • Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., Cirillo, M., . . . Hiebert, J. (2019c). So what? Justifying conclusions and interpretations of data. Journal for Research in Mathematics Education, 50(5), 470–477.

    Article  Google Scholar 

  • Cai, J., Morris, A., Hohensee, C., Hwang, S., Robison, V., Cirillo, M., . . . Hiebert, J. (2019d). Theoretical framing as justifying. Journal for Research in Mathematics Education, 50(3), 218–224. https://doi.org/10.5951/jresematheduc.50.3.0218.

    Article  Google Scholar 

  • Cicchetti, D. V. (1991). The reliability of peer review for manuscript and grant submissions: A cross-disciplinary investigation. Behavioral and Brain Science, 14, 119–135.

    Article  Google Scholar 

  • Confrey, J. (2017). Research: To inform, deform, or reform? In J. Cai (Ed.), Compendium for research in mathematics education (pp. 3–27). Reston, VA: National Council of Teachers of Mathematics.

  • Cooley, W. W., & Leinhardt, G. (1975). The application of a model for investigating classroom processes. Pittsburg, KS: University of Pittsburg, Learning Research and Development Center.

  • Cronbach, L. J. (1986). Social inquiry by and for earthlings. In D. W. Fiske & R. A. Shweder (Eds.), Metatheory in social science: Pluralisms and subjectivities (pp. 83–107). Chicago, IL: University of Chicago Press.

  • Einstein, A., & Infeld, L. (1938). The evolution of physics: The growth of ideas from early concepts to relativity and quanta. Cambridge, England: Cambridge University Press.

  • Gutiérrez, K. D., & Penuel, W. R. (2014). Relevance to practice as a criterion for rigor. Educational Researcher, 43(1), 19–23.

    Article  Google Scholar 

  • Hadamard, J. (1945). An essay on the psychology of invention in the mathematical field. Princeton, NJ: Princeton University Press.

  • Heid, M. K., & Blume, G. W. (2011). Strengthening manuscript submissions. Journal for Research in Mathematics Education, 42(2), 106–108. https://doi.org/10.5951/jresematheduc.42.2.0106.

    Article  Google Scholar 

  • Hemlin, S., & Rasmussen, S. B. (2006). The shift in academic quality control. Science Technology & Human Values, 31, 173–198.

    Article  Google Scholar 

  • Holbrook, A., Bourke, S., Fairbairn, H., Preston, G., Cantwell, R., & Scevak, J. (2009). Publishing in academic journals in education [CD]. Melbourne, Australia: Professional Resources Services.

  • Kelly, J., Sadeghieh, T., & Adeli, K. (2014). Peer review in scientific publications: Benefits, critiques, & a survival guide. EJIFCC, 25(3), 227–243.

    Google Scholar 

  • Kempe, U. R. (2019). Teachers and researchers in collaboration: A possibility to overcome the research-practice gap? European Journal of Education, 54(2), 250–260.

    Article  Google Scholar 

  • Klamkin, M. S. (1968). On the teaching of mathematics so as to be useful. Educational Studies in Mathematics, 1(1–2), 126–160. https://doi.org/10.1007/BF00426240.

    Article  Google Scholar 

  • Lester, F. K., & Lambdin, D. V. (1998). The ship of Theseus and other metaphors for thinking about what we value in mathematics education research. In A. Sierpinska & J. Kilpatrick (Eds.), Mathematics education as a research domain: A search for identity (pp. 415–425). Dordrecht, The Netherlands: Kluwer.

  • Lindsey, D. (1988). Assessing precision in the manuscript review process - a little better than a dice roll. Scientometrics, 14, 75–82.

    Article  Google Scholar 

  • Maxwell, J. A. (2004). Causal explanation, qualitative research, and scientific inquiry in education. Educational Researcher, 33(2), 3–11. https://doi.org/10.3102/0013189X033002003.

    Article  Google Scholar 

  • National Research Council. (2002). Scientific research in education. Washington, DC: National Academies Press. https://doi.org/10.17226/10236.

  • Nivens, R. A., & Otten, S. (2017). Assessing journal quality in mathematics education. Journal for Research in Mathematics Education, 48(4), 328–368.

    Google Scholar 

  • Norton, A., & Bell, M. A. (2017). Mathematics educational neuroscience: Promises and challenges (pp. 879–892). In J. Cai (Ed.), Compendium for research in mathematics education. Reston, VA: National Council of Teachers of Mathematics.

  • Nuthall, G. (2004). Relating classroom teaching to student learning: A critical analysis of why research has failed to bridge the theory-practice gap. Harvard Educational Review, 74(3), 273–306. https://doi.org/10.17763/haer.74.3.e08k1276713824u5.

    Article  Google Scholar 

  • Simon, M. (2004). Raising issues of quality in mathematics education research. Journal for Research in Mathematics Education, 35, 147–163.

    Google Scholar 

  • Smith, R. (2006). Peer review: A flawed process at the heart of science and journals. Journal of the Royal Society of Medicine, 99, 178–182.

    Article  Google Scholar 

  • Stokes, D. E. (1997). Pasteur’s quadrant: Basic science and technological innovation. Washington, DC: Brookings Institution Press.

  • Toerner, G., & Arzarello, F. (2012). Grading mathematics education research journals. Newsletter of the European Mathematical Society, 86, 52–54.

    Google Scholar 

  • Ware, M. (2008). Peer review in scholarly journals: Perspective of the scholarly community – Results from an international study. Information Services & Use, 28, 109–112.

    Article  Google Scholar 

  • Williams, S. R., & Leatham, K. R. (2017). Journal quality in mathematics education. Journal for Research in Mathematics Education, 48(4), 369–396.

    Article  Google Scholar 

Download references

Author information

Authors and Affiliations

  1. University of Delaware, Newark, DE, USA

    Jinfa Cai, Stephen Hwang, James Hiebert, Charles Hohensee, Anne Morris & Victoria Robison

Authors
  1. Jinfa Cai
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Stephen Hwang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. James Hiebert
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Charles Hohensee
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Anne Morris
    View author publications

    You can also search for this author in PubMed Google Scholar

  6. Victoria Robison
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Jinfa Cai.

Additional information

This work was supported in part by the National Science Foundation under Grant No. 1941494. Any opinions, findings, and conclusions are those of the authors and do not necessarily reflect the views of the National Science Foundation.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Cai, J., Hwang, S., Hiebert, J. et al. Communicating the Significance of Research Questions: Insights from Peer Review at a Flagship Journal. Int J of Sci and Math Educ 18 (Suppl 1), 11–24 (2020). https://doi.org/10.1007/s10763-020-10073-x

Download citation

  • Received:

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s10763-020-10073-x

Keywords

Search

Navigation