Answers to 18 Questions About Open Science Practices
- 1.5k Downloads
Open science refers to an array of practices that promote openness, integrity, and reproducibility in research; the merits of which are being vigorously debated and developed across academic journals, listservs, conference sessions, and professional associations. The current paper identifies and clarifies major issues related to the use of open science practices (e.g., data sharing, study pre-registration, open access journals). We begin with a useful general description of what open science in organizational research represents and adopt a question-and-answer format. Through this format, we then focus on the application of specific open science practices and explore future directions of open science. All of this builds up to a series of specific actionable recommendations provided in conclusion, to help individual researchers, reviewers, journal editors, and other stakeholders develop a more open research environment and culture.
KeywordsOpen science Philosophy of science Questionable research practices Research ethics
- American Psychological Association. (2010). Publication manual of the American Psychological Association (6th ed.). Washington, DC: American Psychological Association.Google Scholar
- Anderson, K. (2016). 96 things pubishers do. The scholarly kitchen. Retrieved on March 9, 2017 from https://scholarlykitchen.sspnet.org/2016/02/01/guest-post-kent-anderson-updated-96-things-publishers-do-2016-edition/
- Bakker, M., van Dijk, A., & Wicherts, J. M. (2012). The rules of the game called psychological science. Perspectives on Psychological Science, 7, 543–554.Google Scholar
- Bakker, M., & Wicherts, J. M. (2014). Outlier removal and the relation with reporting errors and quality of psychological research. PLos One.Google Scholar
- Banks, G. C., Kepes, S., & McDaniel, M. A. (2015). Publication bias: Understanding the myths concerning threats to the advancement of science. In C. E. Lance & R. J. Vandenberg (Eds.), Statistical and methodological myths and urban legends: Doctrine, verity, and fable in organizational and social sciences (pp. 36–64). New York: Routledge.Google Scholar
- Banks, G. C., O’Boyle Jr., E., Pollack, J. M., White, C. D., Batchelor, J. H., Whelpley, C. E., Abston, K. A., Bennett, A. A., & Adkins, C. L. (2016). Questions about questionable research practices in the field of management: A guest commentary. Journal of Management, 42, 5–20.CrossRefGoogle Scholar
- Bedeian, A. G., Taylor, S. G., & Miller, A. N. (2010). Management science on the credibility bubble: Cardinal sins and various misdemeanors. Academy of Management Learning & Education, 9, 715–725.Google Scholar
- Benjamin, D. J., Berger, J. O., Johannesson, M., Nosek, B. A., Wagenmakers, E.-J., Berk, R., … Camerer, C. (2017). Redefine statistical significance. Nature Human Behaviour.Google Scholar
- Bhattacharjee, Y. (2013). The mind of a con man. The New York Times. Retrieved March 9, 2017 from http://www.nytimes.com/2013/04/28/magazine/diederik-stapels-audacious-academic-fraud.html?pagewanted=all&_r=0.Google Scholar
- Carey, B. (2015). Journal science releases guidelines for publishing scientific studies. The New York Times. Retrieved from http://www.nytimes.com/2015/06/26/science/journal-science-releases-guidelines-for-publishing-scientific-studies.html?_r=0.
- Cortina, J. M. (2015). A revolution with a solution. Philadelphia, PA: Opening plenary presented at the meeting of the Society for Industrial/Organizational Psychology.Google Scholar
- Fang, F. C., & Casadevall, A. (2015). Competitive science: Is competition ruining science? Infection and Immunity: IAI. 02939–02914.Google Scholar
- Findley, M. G., Jensen, N. M., Malesky, E. J., & Pepinsky, T. B. (2016). Can results-free review reduce publication bias? The results and implications of a pilot study. Comparative Political Studies, 1–37.Google Scholar
- Grand, J. A., Rogelberg, S. G., Allen, T. D., Landis, R. S., Reynolds, D. H., Scott, J. C., Tonidandel, S., Truxillo, D. M. (2017). A systems-based approach to fostering robust science in industrial-organizational psychology. Industrial and Organizational Psychology: Perspectives on Science and Practice. 1–39.Google Scholar
- Grand, J. A., Rogelberg, S. G., Banks, G. C., Landis, R. S., & Tonidandel, S. (in press). From outcome to process focus: Fostering a more robust psychological science through registered reports and results-blind reviewing. Perspectives on Psychological Science.Google Scholar
- Healy, L. W. (2015). Tipping point: Information industry outlook. Retrieved from http://info.outsellinc.com/rs/422-MBV-091/images/Outsell_CEO_Topics_01oct2015_Info_Industry_Outlook_2016_Tipping_Point.pdf.
- Internal Revenue Service. (2014). Form 990: Return of organization exempt from income tax. Retrieved from http://www.irs.gov/pub/irs-pdf/f990.pdf.
- Jebb, A. T., Parrigon, S., & Woo, S. (in press). Exploratory data analysis as a foundation of inductive research. Human Resource Management Review.Google Scholar
- John, L. K., Loewenstein, G., & Prelec, D. (2012). Measuring the prevalence of questionable research practices with incentives for truth telling. Psychological Science, 23(5), 524–532. https://doi.org/10.1177/0956797611430953.
- Kepes, S., Banks, G., C., McDaniel, M. A., & Whetzel, D. L. (2012). Publication bias in the organizational sciences. Organizational Research Methods, 15, 624–662. https://doi.org/10.1177/1094428112452760.
- Korn, M. (2014). Management research is fish, says new management research. The Wall Street Journal. Retrieved from http://blogs.wsj.com/atwork/2014/02/18/management-research-is-fishy-says-new-management-research/.
- Lakens, D., Adolfi, F. G., Albers, C. J., Anvari, F., Apps, M. A. J., Argamon, S. E., … Zwaan, R. A. (2017, September 18). Justify your alpha: A response to “redefine statistical significance”. Retrieved from psyarxiv.com/9s3y6
- Nosek, B. A., Alter, G., Banks, G. C., Borsboom, D., Bowman, S. D., Breckler, S., Buck, S., Chambers, C., Chin, G., Christensen, G., Contestabile, M., Dafoe, A., Eich, E., Freese, J., Glennerster, R., Goroff, D., Green, D. P., Heese, B., Humphreys, M., Ishiyama, J., Karlan, D., Kraut, A., Lupia, A., Marbry, P., Madon, T., Malhotra, N., Wilson, E. M., McNutt, M., Miguel, E., Paluck, E. L., Simonsohn, U., Soderberg, C., Spellman, B. A., Tornow, J., Turitto, J., VandenBos, G. R., Vazire, S., Wagenmakers, E. J., Wilson, R., & Yarkoni, T. (2015). Promoting an open research culture: Author guidelines for journals to promote transparency, openness, and reproducibility. Science, 348, 1422–1425.CrossRefGoogle Scholar
- O'Boyle, E. H., Banks, G. C., Carter, K., Walter, S., & Yuan, Z. (2018). A 20-year review of outcome reporting bias in moderated multiple regression. Journal of Business and Psychology. https://doi.org/10.1007/s10869-018-9539-8.
- O'Boyle, E. H., Banks, G. C., & Gonzalez-Mule, E. (2017). The Chrysalis effect: How ugly initial results metamorphosize into beautiful articles. Journal of Management, 43, 400–425.Google Scholar
- Office of Science and Technology Policy. (2000). Federal research misconduct policy. Federal Register., 65(235), 76260–76264.Google Scholar
- Pordes, R., Petravick, D., Kramer, B., Olson, D., Livny, M., Roy, A., Avery, P., Blackburn, K., Wenaus, T., & Würthwein, F. (2007). The open science grid. Journal of Physics: Conference Series IOP Publishing, 12057, 140–146.Google Scholar
- Schmidt, F. L., & Oh, I.-S. (2016). The crisis of confidence in research findings in psychology: Is lack of replication the real problem? Or is it something else? Archives of Scientific Psychology, 4(1), 32.Google Scholar
- Spector, P. E., Rogelberg, S. G., Ryan, A. M., Schmitt, N., & Zedeck, S. (2014). Moving the pendulum back to the middle: Reflections on and introduction to the inductive research special issue of Journal of Business and Psychology. Journal of Business and Psychology, 29, 499–502.CrossRefGoogle Scholar
- Sterling, T. D., Rosenbaum, W. L., & Weinkam, J. J. (1995). Publication decisions revisited: The effect of the outcome of statistical tests on the decision to publish and vice versa. The American Statistician, 49, 108–112.Google Scholar
- Tanner, A. (2013). How a zip code can tell a marketer exactly who you are. Forbes Retrieved from http://www.forbes.com/sites/adamtanner/2013/07/22/how-just-a-zip-code-can-tell-a-marketer-exactly-who-you-are/#1a25491b12a7
- Wagenmakers, E.-J. & Dutilh, G. (2016). Seven selfish-reasons for pre-registration. Association for Psychological Science, 1–6.Google Scholar
- Wicherts, J. M. (2016). Data re-analysis and open data. In J. Plucker & M. Makel (Eds.), Doing good social science: Trust, accuracy, transparency. American Psychological Association: Washington.Google Scholar
- Woo, S. E., O'Boyle, E. H., & Spector, P. E. (in press). Best practices in developing, conducting, and evaluating inductive research. Human Resource Management Review.Google Scholar