Development and Validation of the Survey of Organizational Research Climate (SORC)

Abstract

Development and targeting efforts by academic organizations to effectively promote research integrity can be enhanced if they are able to collect reliable data to benchmark baseline conditions, to assess areas needing improvement, and to subsequently assess the impact of specific initiatives. To date, no standardized and validated tool has existed to serve this need. A web- and mail-based survey was administered in the second half of 2009 to 2,837 randomly selected biomedical and social science faculty and postdoctoral fellows at 40 academic health centers in top-tier research universities in the United States. Measures included the Survey of Organizational Research Climate (SORC) as well as measures of perceptions of organizational justice. Exploratory and confirmatory factor analyses yielded seven subscales of organizational research climate, all of which demonstrated acceptable internal consistency (Cronbach’s α ranging from 0.81 to 0.87) and adequate test–retest reliability (Pearson r ranging from 0.72 to 0.83). A broad range of correlations between the seven subscales and five measures of organizational justice (unadjusted regression coefficients ranging from 0.13 to 0.95) document both construct and discriminant validity of the instrument. The SORC demonstrates good internal (alpha) and external reliability (test–retest) as well as both construct and discriminant validity.

This is a preview of subscription content, log in to check access.

Fig. 1

References

  1. Adams, J. S. (1965). Inequity in social exchange. In L. Berkowitz (Ed.). Advances in experimental social psychology (pp. 267–299). New York: Academic Press.

  2. Anderson, M. S., Horn, A. S., Risbey, K. R., Ronning, E. A., De Vries, R., & Martinson, B. C. (2007). What do mentoring and training in the responsible conduct of research have to do with scientists’ misbehavior? Findings from a national survey of NIH-funded scientists. Academic Medicine, 82, 853–860.

    Article  Google Scholar 

  3. Antes, A. L., Brown, R. P., Murphy, S. T., Waples, E. P., Mumford, M. D., Connelly, S., et al. (2007). Personality and ethical decision-making in research: The role of perceptions of self and others. Journal of Empirical Research on Human Research Ethics, 2, 15–34.

    Article  Google Scholar 

  4. Arnaud, A. (2010). Conceptualizing and measuring ethical work climate: Development and validation of the ethical climate index. Business & Society, 49, 345–358.

    Article  Google Scholar 

  5. Clayton, S., & Opotow, S. (2003). Justice and identity: Changing perspectives on what is fair. Personality and Social Psychology Review, 7, 298–310.

    Article  Google Scholar 

  6. Colquitt, J. A. (2001). On the dimensionality of organizational justice: A construct validation of a measure. Journal of Applied Psychology, 86, 386–400.

    Article  Google Scholar 

  7. Committee on Assessing Integrity in Research Environments (U.S.), National Research Council (U.S.), United States. Office of the Assistant Secretary for Health. Office of Research Integrity. (2002). Integrity in scientific research: Creating an environment that promotes responsible conduct. Washington, DC: National Academies Press.

  8. Council of Canadian Academies, The Expert Panel on Research Integrity. (2010). Honesty, accountability and trust: Fostering research integrity in Canada. Ottawa, ON: Council of Canadian Academies.

  9. Council of Graduate Schools. (2011). Project for scholarly integrity (WWW document). http://www.scholarlyintegrity.org/ShowContent.aspx?id=402.

  10. Crain, A. L., Martinson, B. C., Ronning, E. A., McGree, D., Anderson, M. S., & DeVries, R. (2008). Supplemental sampling frame data as a means of assessing response bias in a hierarchical sample of university faculty. In Presented at annual meetings of the American Association for public opinion research, New Orleans, LA.

  11. Crain, A. L., Martinson, B. C., Thrush, C. R., (2012). Relationships between the survey of organizational research climate (SORC) and self-reported research practices. Journal of Science and Engineering Ethics. doi:10.1007/s11948-012-9409-0

  12. Department of Health and Human Services. (2000). NIH guide for grants and contracts: Announcement of final PHS policy on instruction in the responsible conduct of research. http://grants.nih.gov/grants/guide/notice-files/NOT-OD-01-007.html.

  13. Department of Health and Human Services. (2001). NIH guide for grants and contracts: Notice of suspension of PHS policy. (http://grants.nih.gov/grants/guide/notice-files/NOT-OD-01-020.html).

  14. DuBois, J. M., Anderson, E. E., Carroll, K., Gibb, T., Kraus, E., Rubbelke, T., et al. (2012). Environmental factors contributing to wrongdoing in medicine: A criterion-based review of studies and cases. Ethics and Behavior, 22, 163–188.

    Article  Google Scholar 

  15. Folger, R., & Cropanzano, R. (1998). Organizational justice and human resource management. Thousand Oaks: Sage.

    Google Scholar 

  16. Geller, G., Boyce, A., Ford, D. E., & Sugarman, J. (2010). Beyond “Compliance”: The role of institutional culture in promoting research integrity. Academic Medicine, 85, 1296–1302.

    Article  Google Scholar 

  17. Groves, R. M. (2006). Nonresponse rates and nonresponse bias in household surveys. Public Opinion Quarterly, 70, 646–675.

    Article  Google Scholar 

  18. Hackett, E. J. (1994). A social control perspective on scientific misconduct. The Journal of Higher Education, 65, 242–260.

    Article  Google Scholar 

  19. Heitman, E., Anestidou, L., Olsen, C., & Bulger, R. E. (2005). Do researchers learn to overlook misbehavior? Hastings Center Report, 35, 49.

    Article  Google Scholar 

  20. Heitman, E., & Bulger, R. E. (2005). Assessing the educational literature in the responsible conduct of research for core content. Accountability in Research, 12, 207–224.

    Article  Google Scholar 

  21. Integrity and Misconduct in Research: Report of the Commission on Research Integrity. (1995). Commission by U.S. Congress No. U.S. Government Printing Office: 1996-746-425. Washington, DC: U.S. Department of Health and Human Services, Public Health Service.

  22. Irish Council for Bioethics, Rapporteur Group. (2010). Recommendations for promoting research integrity. Dublin: The Irish Council for Bioethics.

    Google Scholar 

  23. Kohn, L. T., Corrigan, J. M., & Donaldson, M. S., (2000). To Err is human: Building a safer health system. Washington, DC: The National Academies Press.

  24. Landy, F. J., & Conte, J. M. (2010). Work in the 21st century: An introduction to industrial and organizational psychology. New York: Wiley.

    Google Scholar 

  25. Leape, L. L. (2010). Transparency and public reporting are essential for a safe health care system. New York: The Commonwealth Fund.

    Google Scholar 

  26. Martinson, B. C., Anderson, M. S., Crain, A. L., & De Vries, R. (2006). Scientists’ perceptions of organizational justice and self-reported misbehaviors. Journal of Empirical Research on Human Research Ethics, 1, 51–66. http://www.pubmedcentral.nih.gov/articlerender.fcgi?tool=pubmed&pubmedid=16810337.

  27. Martinson, B. C., Anderson, M. S., & De Vries, R. (2005). Scientists behaving badly. Nature, 435, 737–738.

    Article  Google Scholar 

  28. Martinson, B. C., Crain, A. L., Anderson, M. S., & De Vries, R. (2009). Institutions’ expectations for researchers’ self-funding, federal grant holding and private industry involvement: Manifold drivers of self-interest and researcher behavior. Academic Medicine, 84, 1491–1499.

    Article  Google Scholar 

  29. Martinson, B. C., Crain, A. L., De Vries, R., & Anderson, M. S. (2010). The importance of organizational justice in ensuring research integrity. Journal of Empirical Research on Human Research Ethics, 5, 67–83.

    Article  Google Scholar 

  30. Mumford, M., & Helton, W. B. (2001). Organizational influences on scientific integrity. In: Proceedings of the 2000 ORI conference on research on research integrity (pp. 73–90). Bethesda, MD: Investigating Research Integrity.

  31. Mumford, M. D., Murphy, S. T., Connelly, S., Hill, J. H., Antes, A. L., Brown, R. P., et al. (2007). Environmental influences on ethical decision making: Climate and environmental predictors of research integrity. Ethics and Behavior, 17, 337–366.

    Article  Google Scholar 

  32. Panel on Scientific Responsibility and the Conduct of Research. (1992). Responsible science, Volume I: Ensuring the integrity of the research process. Washington, DC: National Academy of Sciences.

  33. Peugh, J. L., & Enders, C. K. (2010). Specification searches in multilevel structural equation modeling: A Monte Carlo investigation. Structural Equation Modeling: A Multidisciplinary Journal, 17, 42–65.

    Article  Google Scholar 

  34. Pfeffer, J., & Langton, N. (1993). The effects of wage dispersion on satisfaction, productivity, and working collaboratively: Evidence from college and university faculty. Administrative Science Quarterly, 38, 382–407.

    Article  Google Scholar 

  35. Pimple, K. D. (2002). Six domains of research ethics: A heuristic framework for the responsible conduct of research. Science and Engineering Ethics, 8, 191–205.

    Article  Google Scholar 

  36. Rennie, D., & Gunsalus, C. K. (2008). What is research misconduct? In F. O. Wells & M. J. G. Farthing (Eds.), Fraud and misconduct in biomedical research. London: Royal Society of Medicine Press.

    Google Scholar 

  37. Schein, E. H. (1991). What is culture? In P. J. Frost, L. F. Moore, M. R. Louis, C. C. Lundberg, & J. Martin (Eds.), Reframing organizational culture (pp. 243–253). Newbury Park, CA: Sage.

    Google Scholar 

  38. Sovacool, B. (2008). Exploring scientific misconduct: Isolated individuals, impure institutions, or an inevitable idiom of modern science? Journal of Bioethical Inquiry, 5, 271–282.

    Article  Google Scholar 

  39. Steneck, N. H. (2006). Fostering integrity in research: Definitions, current knowledge, and future directions. Science and Engineering Ethics, 12, 53–74.

    Google Scholar 

  40. Steneck, N. H., & Bulger, R. E. (2007). The history, purpose, and future of instruction in the responsible conduct of research. Academic Medicine, 82, 829–834.

    Article  Google Scholar 

  41. Steneck, N., & Mayer, T. (2010). Singapore statement on research integrity (WWW document). http://www.singaporestatement.org/.

  42. Teitelbaum, M. S. (2008). Research funding: Structural disequilibria in biomedical research. Science, 321, 644–645.

    Article  Google Scholar 

  43. Thrush, C. R., Vander Putten, J., Rapp, C. G., Pearson, L. C., Berry, K. S., & O’Sullivan, P. S. (2007). Content validation of the Organizational Climate for Research Integrity (OCRI) survey. Journal of Empirical Research on Human Research Ethics, 2, 35–52.

    Article  Google Scholar 

  44. Titus, S. L., Wells, J. A., & Rhoades, L. J. (2008). Repairing research integrity. Nature, 453, 980–982.

    Article  Google Scholar 

  45. Tyler, T. R., & Blader, S. L. (2003). The group engagement model: procedural justice, social identity, and cooperative behavior. Personality and Social Psychology Review, 7, 349–361.

    Article  Google Scholar 

  46. Vandenberg, R. J., & Lance, C. E. (2000). A review and synthesis of the measurement invariance literature: Suggestions, practices, and recommendations for organizational research. Organizational Research Methods, 3, 4–70.

    Article  Google Scholar 

  47. Wenger, N. S., Korenman, S. G., Berk, R., & Berry, S. (1997). The ethics of scientific research: An analysis of focus groups of scientists and institutional representatives. Journal of Investigative Medicine, 45, 371–380.

    Google Scholar 

Download references

Acknowledgments

The authors wish to acknowledge the excellent work of Shannon Donald in several key aspects contributing to this manuscript including project coordination and sample frame development. This research was supported by Award Number R21-RR025279 from the NIH National Center for Research Resources and the DHHS Office of Research Integrity through the collaborative Research on Research Integrity Program.

Conflict of interest

None.

Ethical approval

The study protocol was approved by the Regions Hospital Institutional Review Board, the oversight body with responsibility for all research conducted at HealthPartners Institute for Education and Research, and by the University of Arkansas for Medical Sciences Institutional Review Board.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Brian C. Martinson.

Additional information

The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Center for Research Resources, the National Institutes of Health, or the Office of Research Integrity.

Rights and permissions

Reprints and Permissions

About this article

Cite this article

Martinson, B.C., Thrush, C.R. & Lauren Crain, A. Development and Validation of the Survey of Organizational Research Climate (SORC). Sci Eng Ethics 19, 813–834 (2013). https://doi.org/10.1007/s11948-012-9410-7

Download citation

Keywords

  • Research integrity
  • Organizational climate
  • Reliability
  • Validity
  • Organizational survey