Journal of General Internal Medicine

, Volume 32, Issue 7, pp 783–789 | Cite as

Assessing the Culture of Residency Using the C - Change Resident Survey: Validity Evidence in 34 U.S. Residency Programs

  • Linda H. Pololi
  • Arthur T. Evans
  • Janet T. Civian
  • Sandy Shea
  • Robert T. Brennan
Original Research



A practical instrument is needed to reliably measure the clinical learning environment and professionalism for residents.


To develop and present evidence of validity of an instrument to assess the culture of residency programs and the clinical learning environment.


During 2014–2015, we surveyed residents using the C - Change Resident Survey to assess residents’ perceptions of the culture in their programs.


Residents in all years of training in 34 programs in internal medicine, pediatrics, and general surgery in 14 geographically diverse public and private academic health systems.

Main Measures

The C - Change Resident Survey assessed residents’ perceptions of 13 dimensions of the culture: Vitality, Self-Efficacy, Institutional Support, Relationships/Inclusion, Values Alignment, Ethical/Moral Distress, Respect, Mentoring, Work–Life Integration, Gender Equity, Racial/Ethnic Minority Equity, and self-assessed Competencies. We measured the internal reliability of each of the 13 dimensions and evaluated response process, content validity, and construct-related evidence validity by assessing relationships predicted by our conceptual model and prior research. We also assessed whether the measurements were sensitive to differences in specialty and across institutions.

Key Results

A total of 1708 residents completed the survey [internal medicine: n = 956, pediatrics: n = 411, general surgery: n = 311 (51% women; 16% underrepresented in medicine minority)], with a response rate of 70% (range across programs, 51–87%). Internal consistency of each dimension was high (Cronbach α: 0.73–0.90). The instrument was able to detect significant differences in the learning environment across programs and sites. Evidence of validity was supported by a good response process and the demonstration of several relationships predicted by our conceptual model.


The C - Change Resident Survey assesses the clinical learning environment for residents, and we encourage further study of validity in different contexts. Results could be used to facilitate and monitor improvements in the clinical learning environment and resident well-being.


residents culture of medicine clinical learning environment quantitative survey resident well-being 



The authors gratefully acknowledge the funding support of the Arnold P. Gold Foundation and supplemental funds from the Committee of Interns and Residents, the University of New Mexico Health Sciences Center, Boston Medical Center, and Oakland University William Beaumont School of Medicine. We thank Vasilia Vasiliou and Alexander Feldman for their excellent assistance with manuscript preparation. We are grateful to the many hardworking residents who took time to complete the survey.

An early phase of the work was presented orally and as a poster at the 2016 ACGME Annual Educational Conference, February 2016. There has been no prior publication of this work.

Compliance with Ethical Standards

Conflict of Interest

The fact that the survey under discussion is funded at any time by financial support for administering the survey and analyzing data, and that any investigator could be paid from these funds, may seem to qualify in the strictest sense as a potential conflict of interest in a paper reporting on the survey’s validity. In this regard, the National Initiative on Gender, Culture and Leadership in Medicine, C - Change, is entirely externally funded. All grants and contracts from various institutions requesting our survey administration and “blind” data analysis of collected data are awarded to Brandeis University, which pays some salary support to some of the investigators (LP, JC, RB, SS). In this study, all survey work was paid for by a grant from the Arnold P. Gold Foundation and supplemental funders. C - Change maintains rigorous scientific standards for conducting its studies, and there is complete separation of income flowing into the University for the overall Center support and the survey administration and analysis.

IRB approval was obtained from Brandeis University.


  1. 1.
    Leach DC. Changing education to improve patient care. Qualitative Health Care. 2001;10(II suppl):54–58.Google Scholar
  2. 2.
    Hoff TJ, Pohl H, Bartfield J. Creating a learning environment to produce competent residents: the roles of culture and context. Acad Med. 2004;79:532–40.CrossRefPubMedGoogle Scholar
  3. 3.
  4. 4.
    Markakis KM, Beckman HB, Suchman AL, Frankel RM. The path to professionalism: cultivating humanistic values and attitudes in residency training. Acad Med. 2000;75:141–150.CrossRefPubMedGoogle Scholar
  5. 5.
    ACGME Outcomes Project: General Competencies. Accreditation Council for Graduate Medical Education. Chicago, IL; 1999.Google Scholar
  6. 6.
    Butterfield PS. The stress of residency, a review of the literature. Arch Intern Med. 1998;148:1428–1435.CrossRefGoogle Scholar
  7. 7.
    IsHak WW, Lederer S, Mandili C, Nikravesh R, Seligman L, Vasa M, Ogunyemi D, Bernstein CA. Burnout during residency training: a literature review. J Grad Med Educ. 2009;1:236–242.CrossRefPubMedPubMedCentralGoogle Scholar
  8. 8.
    Fnais N, Soobiah C, Chen MH, Lillie E, Perrier L, Tashkhandi M, Straus SE, Mamdani M, Al-Omran M, Tricco AC. Harassment and discrimination in medical training: a systematic review and meta-analysis. Acad Med. 2014;89(5):817–827.CrossRefPubMedGoogle Scholar
  9. 9.
    Jennings ML, Slavin SJ. Resident wellness matters: optimizing resident education and wellness through the learning environment. Acad Med. 2015;90(9):1246–1250.CrossRefPubMedGoogle Scholar
  10. 10.
    Seelig CB, DuPre CT. Development and validation of scaled questionnaire for evaluation of residency programs. South Med J. 1995;88:745–51.CrossRefPubMedGoogle Scholar
  11. 11.
    Maslach C, Jackson SE, Leiter MP. Maslach burnout inventory manual. 3rd ed. Palo Alto, CA: Consulting Psychologists Press; 1996.Google Scholar
  12. 12.
    Gruppen GD, Stansfield RB, Zhao Z, Sen S. Institution and specialty contribute to resident satisfaction with their learning environment and workload. Acad Med. 2015;90:S77–S82.CrossRefPubMedPubMedCentralGoogle Scholar
  13. 13.
    Senge P. The fifth discipline: the art and practice of the learning organization. New York: Currency and Doubleday; 1990.Google Scholar
  14. 14.
    Pronovost PJ, Bienvenu OJ. From shame to guilt to love. JAMA. 2015;314(23):2507–2508.CrossRefPubMedGoogle Scholar
  15. 15.
    Shanafelt TD. Enhancing meaning in work: a prescription for preventing physician burnout and promoting patient-centered care. JAMA. 2009;302(12):1338–1340.CrossRefPubMedGoogle Scholar
  16. 16.
    Weiss KB, Wagner R, Nasca T. Development, Testing and implementation of the ACGME Clinical Learning Environment Review (CLER) Program. J of Grad Med Educ. 2012;396-398.Google Scholar
  17. 17.
    Bennett CL, Finch A, Vuong K, McDonald D, Rennie S. Surgical resident duty hours. N Engl J Med. 2016;374(24):2399–2403.CrossRefPubMedGoogle Scholar
  18. 18.
    Kohn LT, Corrigan JM, Donaldson MS. To Err Is Human: Building a Safer Health System. Institute of Medicine (IOM). Washington, D.C.: National Academies Press; 2000.Google Scholar
  19. 19.
    National Initiative on Gender, Culture and Leadership in Medicine (C-Change). (accessed 17 February, 2017).
  20. 20.
    Pololi L, Evans A, Nickell L, Reboli A, Coplit L, Stuber M, Vasiliou V, Civian J, Brennan R. Assessing the learning environment for medical students: an evaluation of a novel survey instrument in four medical schools. Acad Psychiatry. 2016. doi: 10.1007/s40596-016-0620-1.PubMedGoogle Scholar
  21. 21.
    Pololi LH, Krupat E, Civian JT, Ash AS, Brennan RT. Why are a quarter of faculty considering leaving academic medicine? A study of their perceptions of institutional cultural and intention to leave in 26 representative medical schools. Acad Med. 2012;89:859–869.Google Scholar
  22. 22.
    Pololi LH, Evans AT, Civian JT, Gibbs BK, Coplit LD, Gillum LH, Brennan RT. Faculty vitality — Surviving the challenges facing academic health centers: a national survey of medical faculty. Acad Med. 2015;90:930–936.CrossRefPubMedGoogle Scholar
  23. 23.
    Pololi LH, Evans AT, Civian JT, Vasiliou V, Gibbs BK, Coplit LD, Gillum LH, Brennan RT. Mentoring faculty: A U.S. national survey of its adequacy and linkage to culture in academic health centers. J Contin Educ Health Prof. 2015;35:176–184.CrossRefPubMedGoogle Scholar
  24. 24.
    Pololi LH, Evans AT, Gibbs BK, Krupat E, Brennan RT, Civian JT. The experience of minority faculty who are underrepresented in medicine at 26 representative U.S. medical schools. Acad Med. 2013;88:1308–1314.CrossRefPubMedGoogle Scholar
  25. 25.
    Pololi LH, Civian JT, Brenan RT, Dotollo AL, Krupat E. Experiencing the culture of academic medicine: gender matters, a national study. J Gen Intern Med. 2013;28:201–207.CrossRefPubMedGoogle Scholar
  26. 26.
    Pololi L, Conrad P, Knight S, Carr P. A study of the relational aspects of the culture of academic medicine. Acad Med. 2009;84:106–114.CrossRefPubMedGoogle Scholar
  27. 27.
    Pololi L, Kern DE, Carr P, Conrad P, Knight S. The culture of academic medicine: faculty perceptions of the lack of alignment between individual and institutional values. J Gen Intern Med. 2009;24(12):1289–95.CrossRefPubMedPubMedCentralGoogle Scholar
  28. 28.
    Pololi LH, Jones SJ. Women faculty: an analysis of their experiences in academic medicine and their coping strategies. Gender Med. 2010;7:438–450.CrossRefGoogle Scholar
  29. 29.
    Pololi L, Cooper LA, Carr P. Race, disadvantage and faculty experiences in U.S. academic medicine. J Gen Intern Med. 2010;25:1363–9.CrossRefPubMedPubMedCentralGoogle Scholar
  30. 30.
    Conrad P, Carr P, Knight S, Renfrew MR, Dunn M, Pololi L. Hierarchy as a barrier to advancement for women in academic medicine. J Women’s Health. 2010;19(4):799–805.CrossRefGoogle Scholar
  31. 31.
    Carr P, Pololi L, Knight S, Conrad P. Collaboration in academic medicine: reflections on gender and advancement. Acad Med. 2009;84(10):1447–1453.CrossRefPubMedGoogle Scholar
  32. 32.
    Fredrickson BL. Love 2.0. New York, NY: Hudson Street Press/Penguin; 2013.Google Scholar
  33. 33.
    Connor KM, Davidson JRT. Development of a new resilience scale: the Connor-Davidson resilience scale (CD-RISC). Depress Anxiety. 2003;18:76–82.CrossRefPubMedGoogle Scholar
  34. 34.
    Through the Eyes of the Workforce: Creating Joy, Meaning, and Safer Health Care. Lucian Leape Institute. Boston, MA: National Patient Safety Foundation; 2013.Google Scholar
  35. 35.
    Hicks B. How can a deanery win the battle against bullying? Br Med J. 2003;326:S129.CrossRefGoogle Scholar
  36. 36.
    Rosenbaum JR, Bradley EH, Holmboe ES, Farrell MH, Krumholz HM. Sources of ethical conflict in medical housestaff training: a qualitative study. Am J Med. 2004;116(6):402–7.CrossRefPubMedGoogle Scholar
  37. 37.
    Wright SM, Kern DE, Kolodner K, Howard DM, Brancati FL. Attributes of excellent attending-physician role models. N Engl J Med. 1998;339:1986–1993.CrossRefPubMedGoogle Scholar
  38. 38.
    Skeff KM, Mutha S. Role models - guiding the future of medicine. N Engl J Med. 1998;399:2015–2017.CrossRefGoogle Scholar
  39. 39.
    Hicks L, Yin Y, Robertson D, Woodrow SI. Understanding the clinical dilemmas that shape medical students’ ethical development: questionnaire survey and focus group study. Br Med J. 2001;322:709–710.CrossRefGoogle Scholar
  40. 40.
    Wong RL, Sullivan MC, Yeo HL, Roman SA, Bell HR, Sosa JA. Race and surgical residency: results from a national survey of 4339 US general surgery residents. Ann Surg. 2013;257:782–787.CrossRefPubMedGoogle Scholar
  41. 41.
    Edmunds LD, Ovseiko PV, Shepperd S, Greenhalgh T, Frith P, Roberts NW, Pololi LH, Buchan AM. Why do women choose or reject careers in academic medicine? A narrative review of empirical evidence. Lancet. 2016.Google Scholar
  42. 42.
    Beagan BL. Is this worth getting into a big fuss over?’ Everyday racism in medical school. Med Educ. 2003;37:852–860.CrossRefPubMedGoogle Scholar
  43. 43.
    Oancia T, Bohm C, Carr T, Cujec B, Johnson D. The influence of gender and specialty on reporting of abusive and discriminatory behaviour by medical students, residents and physician teachers. Med Educ. 2000;34:250–256.CrossRefPubMedGoogle Scholar
  44. 44.
    Sullivan MC, Bucholz EM, Yeo H, Roman SA, Bell RH, Sosa JA. “Join the club:” effect of resident and attending social interactions on overall satisfaction among 4390 general surgery residents. Arch Surg. 2012;147:408–414.CrossRefPubMedGoogle Scholar
  45. 45.
    McManus IC, Sproston KA. Women in hospital medicine in the United Kingdom: glass ceiling, preference, prejudice or cohort effect? J Epidemiol Community Health. 2000;54:10–16.CrossRefPubMedPubMedCentralGoogle Scholar

Copyright information

© Society of General Internal Medicine 2017

Authors and Affiliations

  • Linda H. Pololi
    • 1
  • Arthur T. Evans
    • 2
  • Janet T. Civian
    • 3
  • Sandy Shea
    • 4
  • Robert T. Brennan
    • 5
  1. 1.National Initiative on Gender, Culture and Leadership in Medicine: C - Change, Brandeis Women’s Studies Research CenterBrandeis UniversityWalthamUSA
  2. 2.Hospital MedicineWeill Cornell Medical CollegeNew YorkUSA
  3. 3.Brandeis Women’s Studies Research CenterBrandeis UniversityWalthamUSA
  4. 4.Committee of Interns and ResidentsNew YorkUSA
  5. 5.Harvard T. H. Chan School of Public HealthHarvard UniversityBostonUSA

Personalised recommendations