Abstract
This study examined the impact of undergraduate research (UR) in engineering, focusing on three particular learning outcomes: communication, teamwork, and leadership. The study included 5126 students across 31 colleges of engineering. The authors employed propensity score matching method to address the selection bias for selection into (and differential availability of) UR programs. Engineering students who engage in UR tend to report higher skill levels, but when curriculum and classroom experiences are taken into account, there is no significant effect of UR on teamwork and leadership skills. Not accounting for college experiences such as curricular, classroom, and other co-curricular experiences may overestimate the positive relationship between UR participation and professional skills. After propensity score adjustment, we found that UR provided a significant predictor of communication skills; a finding that provides support for previous research regarding the importance of communication skills as an outcome of UR. The study highlights the importance of taking into account selection bias when assessing the effect of co-curricular programs on student learning. Implications of the study include expanding undergraduate research opportunities when possible and incorporating communication and leadership skill development into required course curriculum.
Similar content being viewed by others
Notes
Other conditioning methods commonly used to estimate the average treatment effect include: full matching, stratification on the propensity score, and covariate adjustment using the propensity score.
References
Abet.org. (2015). ABET—Accreditation Criteria and Supporting Documents. Retrieved December 1, 2014 from http://www.abet.org/accreditation-criteria-policies-documents/.
Armor, D. J. (1974). Theta reliability and factor scaling. In H. Costner (Ed.), Sociological methodology: 1973–1974 (pp. 17–50). San Francisco, CA: Jossey-Bass.
Astin, A. W. (1993). What matters in college. San Francisco, CA: Jossey-Bass.
Austin, P. C. (2011). An introduction to propensity score methods for reducing the effects of confounding in observational studies. Multivariate Behavioral Research, 46(3), 399–424.
Baruch, Y. (1999). Response rate in academic studies—A comparative analysis. Human Relations, 52, 421–438. doi:10.1177/001872679905200401.
Bernstein, Laura, Alexander, Debra, & Alexander, Ben. (2008). Generations: Harnessing the potential of the multigenerational workforce. The Catalyst, 37(3), 17–22.
Bowman, N. A. (2010). Can 1st-year college students accurately report their learning and development? American Educational Research Journal, 47(2), 466–496.
Bowman, N. A., & Seifert, T. A. (2011). Can college students accurately assess what affects their learning and development? Journal of College Student Development, 52(3), 270–290.
Boylan, M. (2009). Undergraduate STEM research experiences: Impact on student interest in doing graduate work in STEM fields. In R. G. Ehrenberg & C. V. Kuh (Eds.), Doctoral education and the faculty of the future (pp. 109–120). Ithaca, NY: Cornell University Press.
Brawner, C. E., Camacho, M. M., Lord, S. M., Long, R. A., & Ohland, M. W. (2012). Women in industrial engineering: Stereotypes, persistence, and perspectives. Journal of Engineering Education, 101(2), 288–318.
Carter, F. D., Mandell, M., & Maton, K. I. (2009). The influence of on-campus, academic year undergraduate research on STEM Ph.D. outcomes: Evidence from the Meyerhoff Scholarship Program. Educational Evaluation and Policy Analysis, 31(4), 441–462.
Cheslock, J. J., & Rios-Aguilar, C. (2011). Multilevel analysis in higher education research: A multidisciplinary approach. Higher education: Handbook of theory and research (pp. 85–123). Netherlands: Springer.
Cox, B. E., McIntosh, K., Reason, R. D., & Terenzini, P. T. (2014). Working with Missing Data in Higher Education Research: A Primer and Real-World Example. The Review of Higher Education, 37(3), 377–402.
Craney, C., McKay, T., Mazzeo, A., Morris, J., Prigodich, C., & de Groot, R. (2011). Cross-discipline perceptions of the undergraduate research experience. The Journal of Higher Education, 82(1), 92–113. doi:10.1353/jhe.2011.0000.
Dey, E. L. (1997). Working with low survey response rates: The efficacy of weighting adjustments. Research in Higher Education, 38(2), 215–227. doi:10.1023/a:1024985704202.
Eagan, M. K., Hurtado, S., Garcia, G., Herrera, F., and Garibay, J. (2010, June). Making a difference in science education for underrepresented students: The impact of undergraduate research programs. Paper presented at the annual forum of the Association for Institutional Research, Chicago, IL.
Espinosa, L. L. (2011). Pipelines and pathways: Women of color in undergraduate stem majors and the college experiences that contribute to persistence. Harvard Educational Review, 81(2), 209–241.
Fang, N. (2012). Improving engineering students’ technical and professional skills through project-based active and collaborative learning. International Journal of Engineering Education, 28(1), 26–36.
Goldstein, A. O., Calleson, D., Bearman, R., Steiner, B. D., Frasier, P. Y., & Slatt, L. (2009). Teaching advanced leadership skills in community service (ALSCS) to medical students. Academic Medicine, 84(6), 754–764.
Graham, J. W. (2009). Missing data analysis: Making it work in the real world. Annual Review of Psychology, 60(1), 549–576. doi:10.1146/annurev.psych.58.110405.085530.
Hackett, E. J., Croissant, J., & Schneider, B. (1992). Industry, academe, and the values of undergraduate engineers. Research in Higher Education, 33(3), 275–295.
Harder, V. S., Stuart, E. A., & Anthony, J. C. (2010). Propensity score techniques and the assessment of measured covariate balance to test causal associations in psychological research. Psychological Methods, 15(3), 234.
Ho, D. E., Imai, K., King, G., & Stuart, E. A. (2007). Matching as nonparametric preprocessing for reducing model dependence in parametric causal inference. Political Analysis, 15(3), 199–236. doi:10.1093/pan/mpl013.
Holland, J. L. (1966). The psychology of vocational choice. Waltham, MA: Blaisdell.
Holland, J. L. (1973). Making vocational choices: A theory of careers. Englewood Cliffs, NJ: Prentice-Hall.
Holland, J. L. (1985). Making vocational choices: A theory of vocational personalities and work environments (2nd ed.). Englewood Cliffs, NJ: Prentice-Hall.
Holland, J. L. (1997). Making vocational choices: A theory of vocational personalities and work environments (3rd ed.). Odessa, FL: Psychological Assessment Resources.
Jones, M. T., Barlow, A. E., & Villarejo, M. (2010). Importance of undergraduate research for minority persistence and achievement in biology. The Journal of Higher Education, 81(1), 82–115.
Knight, D. B. (2012). In search of the engineers of 2020: An outcomes-based typology of engineering undergraduates. In Proceedings of the 119th Annual Conference of the American Society for Engineering Education, San Antonio, TX.
Knight, D. B., Lattuca, L. R., Yin, A. C., Kremer, G., York, T., & Ro, H. K. (2012). An exploration of gender diversity in engineering programs: A curriculum and instruction-based perspective. Journal of Women and Minorities in Science and Engineering, 18(1), 55–78.
Lanehart, R. E., Rodriguez de Gil, P., Kim, E. S., Bellara, A. P., Kromrey, J. D., and Lee, R. S. (April, 2012). Propensity score analysis and assessment of propensity score approaches using SAS procedures. Paper presented at the annual meeting of SAS Global Forum, Orlando, FL.
Lattuca, L. R., Terenzini, P. T., Harper, B. J., & Yin, A. (2010). Academic environments in detail: Holland’s theory at the sub-discipline level. Research in Higher Education, 48(2), 251–282.
Lattuca, L. R., Terenzini, P. T., Knight, D. B., & Ro, H. K. (2014). 2020 Vision: Progress in Preparing the Engineer of the Future. Ann Arbor, MI: Author.
Lattuca, L. R., Terenzini, P. T., & Volkwein, J. F. (2006). Engineering Change: Findings from a Study of the Impact of EC2000, Final Report. Baltimore, MD: ABET Inc.
Lord, S. M., Layton, R. A., & Ohland, M. W. (2011). Trajectories of Electrical Engineering and Computer Engineering students by race and gender. Education, IEEE Transactions on, 54(4), 610–618.
May, G. S., & Chubin, D. E. (2003). A retrospective on undergraduate engineering success for underrepresented minority students. Journal of Engineering Education, 92(1), 27–39.
McKenna, A., Plumb, C., Kremer, G. E., Yin, A. C., and Ro, H. K. (2011). Approaches to engaging students in engineering design and problem solving. In Proceedings of the American Society for Engineering Education Annual Conference, Vancouver, Canada.
McKinney, L., & Sadler, T. D. (2010). Scientific research for undergraduate students: A review of the literature. Journal of College Science Teaching, 39(5), 43–49.
Morozov, A., Kilgore, D., Yasuhara, K., and Atman, C. (2008). Same courses, different outcomes? Variations in confidence, experience, and preparations in engineering design. In Proceedings of the American Society for Engineering Education Annual Conference. Pittsburgh, PA.
Myers, C. B. (2011). Union status and faculty job satisfaction: Contemporary evidence from the 2004 National Study of Postsecondary Faculty. The Review of Higher Education, 34(4), 657–684. doi:10.1353/rhe.2011.0028.
Neter, J., Wasserman, W., & Kutner, M. H. (1990). Applied linear statistical models: Regression, analysis of variance, and experimental design (3rd ed.). Burr Ridge, IL: Irwin.
Niehaus, E., Campbell, C. M., & Inkelas, K. K. (2013). HLM behind the curtain: Unveiling decisions behind the use and interpretation of HLM in higher education research. Research in Higher Education, 55(1), 101–122. doi:10.1007/s11162-013-9306-7.
Palmer, B., Terenzini, P. T., McKenna, A. F., Harper, B. J., and Merson, D. (2011). Design in context: Where do the engineers of 2020 learn this skill? In Proceedings of the American Society for Engineering Education Annual Conference. Vancouver, Canada.
Pascarella, E. T. (1985). College environmental influences on learning and cognitive development: A critical review and synthesis. In J. C. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 4, pp. 1–61). New York, NY: Agathon.
Pascarella, E. T., & Terenzini, P. T. (2005). How college affects students: A third decade of research (Vol. 2). San Francisco, CA: Jossey-Bass.
Porter, S. R., & Umbach, P. D. (2006). Student survey response rates across institutions: Why do they vary? Research in Higher Education, 47, 229–247.
Reason, R. D. (2009). An examination of persistence research through the lens of a comprehensive conceptual framework. Journal of College Student Development, 50(6), 659–682.
Ro, H. K., Terenzini, P. T., & Yin, A. C. (2013). Between-college effects on students reconsidered. Research in Higher Education, 54(3), 253-282.
Rubin, D. B., & Thomas, N. (1996). Matching using estimated propensity scores: relating theory to practice. Biometrics, 52, 249–264.
Russell, S. H., Hancock, M. P., & McCullough, J. (2007). Benefits of undergraduate research experiences. Science, 316, 548–549.
Sabatini, D. A. (1997). Teaching and research synergism: The undergraduate research experience. Journal of Professional Issues in Engineering Education and Practice, 123(3), 98–102.
Sadler, T. D., Burgin, S., McKinney, L., & Ponjuan, L. (2010). Learning science through research apprenticeships: A critical review of the literature. Journal of Research in Science Teaching, 47(3), 235–256.
Sankar, C. S., & Raju, P. K. (2011). Use of presage-pedagogy-process-product model to assess the effectiveness of case study methodology in achieving learning outcomes. Journal of STEM Education: Innovations and Research, 12(7), 45.
Schwartz, J. (2012). Faculty as undergraduate research mentors for students of color: Taking into account the costs. Science Education, 96(3), 527–542.
Seat, E., Parsons, J. R., & Poppen, W. A. (2001). Enabling engineering performance skills: A program to teach communication, leadership, and teamwork. Journal of Engineering Education, 90(1), 7–12.
Seymour, E., Hunter, A.-B., Laursen, S. L., & DeAntoni, T. (2004). Establishing the benefits of research experiences for undergraduates in the sciences: First findings from a three-year study. Science Education, 88(4), 493–534. doi:10.1002/sce.10131.
Shadish, W. R., Cook, T. D., & Campbell, D. T. (2002). Experimental and quasi-experimental designs for generalized causal inference. Boston, MA: Houghton Mifflin.
Shadish, W. R., & Steiner, P. M. (2010). A primer on propensity score analysis. Newborn and Infant Nursing Reviews, 10(1), 19–26.
Smart, J. C., Feldman, K. A., & Ethington, C. A. (2000). Academic disciplines: Holland’s theory and the study of college students and faculty. Nashville, TN: Vanderbilt University Press.
Smith, T. W. (1995). Trends in non-response rates. International Journal of Public Opinion Research, 7(2), 157–171. doi:10.1093/ijpor/7.2.157.
Smith, C. M., & Sodano, T. M. (2011). Integrating lecture capture as a teaching strategy to improve student presentation skills through self-assessment. Active Learning in Higher Education, 12(3), 151–162.
Steiner, P. M., & Cook, D. (2013). Matching and propensity scores. In T. D. Little (Ed.), The oxford handbook of quantitative methods: Foundation (Vol. 1, pp. 237–259). New York, NY: Oxford University Press.
Tinto, V. (1993). Leaving college: Rethinking the causes and cures of student attrition (2nd ed.). Chicago, IL: The University of Chicago Press.
Tomek, S. (2011). Developing a multicultural, cross-generational, and multidisciplinary team: An introduction for civil engineers. Leadership and Management in Engineering, 11(2), 191–196.
Tonso, K. L. (2006). Teams that work: Campus culture, engineer identity, and social interactions. Journal of Engineering Education, 95(1), 25–37.
Van Horn, P. S., Green, K. E., & Martinussen, M. (2009). Survey response rates and survey administration in counseling and clinical psychology: A meta-analysis. Educational and Psychological Measurement, 69, 389–403. doi:10.1177/0013164408324462.
Author information
Authors and Affiliations
Corresponding author
Electronic supplementary material
Below is the link to the electronic supplementary material.
Appendix
Appendix
See Table 8.
Rights and permissions
About this article
Cite this article
Carter, D.F., Ro, H.K., Alcott, B. et al. Co-Curricular Connections: The Role of Undergraduate Research Experiences in Promoting Engineering Students’ Communication, Teamwork, and Leadership Skills. Res High Educ 57, 363–393 (2016). https://doi.org/10.1007/s11162-015-9386-7
Received:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11162-015-9386-7