Skip to main content
SpringerLink
Log in

The Impact of a Goal-Setting Intervention for Engineering Students on Academic Probation

  • Published:
Research in Higher Education Aims and scope Submit manuscript

Abstract

Many degree-seeking college students struggle academically and ultimately never graduate. Academic challenges and persistence within the major are especially salient issues for students who major in science, technology, engineering, and mathematics. Academic probation serves as a means for informing students that they are at risk of dismissal, and many colleges and universities offer services to help students placed on probation to succeed academically. This paper presents two studies that examined the effectiveness of a goal-setting academic advising intervention for improving the grades of engineering students who were on academic probation; one study used a regression discontinuity design, and the other used an experimental design. The findings of both studies support the same overall conclusion: The intervention notably increased the grades of engineering students on probation who are beyond their first year of college, but it was not effective for students in their first year. This brief academic enhancement intervention appears to constitute a cost-effective strategy for bolstering the academic success of at-risk college students after their first year.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Similar content being viewed by others

Notes

  1. The process of initial placement into academic probation is straightforward, since this occurs immediately after—and only when—a student receives a semester GPA below 2.0. However, as described later in the paper, returning to academic good standing is more complicated; it requires the student to achieve not only at least a 2.0 GPA in that probationary semester, but also at least a 2.0 cumulative GPA within the institution. Students who do not successfully return to academic good standing at the end of the semester are dismissed from the College of Engineering (some of these students ultimately transfer to another college within the university). Importantly, the present study limited the analytic sample of each RD analysis to students who were enrolled in engineering during the semester in which they were on probation, which ensured that a sharp RD design is appropriate. For instance, the RD analyses that used Semester 3 GPA as the running variable only included students who were enrolled in an engineering major in Semester 4. This sampling is important not only because the performance improvement plan would have occurred for engineering students at the beginning of Semester 4, but also because students who were on probation in Semester 2 and did not return to good standing after Semester 3 would be excluded; thus, the analysis meets the definition of a sharp RD design. The sharp design pertains to being placed on probation and therefore receiving a request to participate in a performance improvement plan. Not all students actually completed this plan, and the available registrar data in Study 1 did not allow us to differentiate treatment completers from non-completers.

  2. The results of the instrumental variable analyses are available upon request from the lead author.

References

  • Ahmed, J. U., Chowdhury, M. H. K., Rahman, S., & Talukder, A. M. H. (2014). Academic probation: An empirical study of private university students. Research in Education,92(1), 1–17.

    Google Scholar 

  • Austin, M., Cherney, E., Crowner, J., & Hill, A. (1997). The forum: Intrusive group advising for the probationary student. NACADA Journal,17(2), 45–47.

    Google Scholar 

  • Brady, S. T., Fotuhi, O., Gomez, E., Cohen, G. L., Urstein, R., & Walton, G. M. (2019). A scarlet letter? Revising institutional messages about academic probation can mitigate students’ feelings of shame and stigma. Manuscript in preparation.

  • Brady, S. T., Kroeper, K. M., Henderson, A. G., Li, X. A., Ozier, E., Blodorn, A., Krol, N., Mathias, K., & Walton, G. M. (2017). Message intended is not message received: Shame, stigma, and disengagement in the academic probation notification process. Paper presented at the annual meeting of the Association for Public Policy Analysis & Management, Chicago, IL.

  • Burnette, J. L., O’Boyle, E. H., VanEpps, E. M., Pollack, J. M., & Finkel, E. J. (2013). Mind-sets matter: A meta-analytic review of implicit theories and self-regulation. Psychological Bulletin,139(3), 655–701.

    Google Scholar 

  • Business Roundtable & Change the Equation. (2014). Business roundtable/change the equation survey on U.S. workforce skills: Summary of findings. Retrieved from https://www.ecs.org/wp-content/uploads/2014-BRT-CTEq-Skills-Survey-Slides_0.pdf.

  • Calonico, S., Cattaneo, M. D., Farrell, M. H., & Titiunik, R. (2017). rdrobust: Software for regression-discontinuity designs. Stata Journal,17, 372–404.

    Google Scholar 

  • Calonico, S., Cattaneo, M. D., & Titiunik, R. (2014). Robust data-driven inference in the regression-discontinuity design. Stata Journal,14, 909–946.

    Google Scholar 

  • Cataldi, E. F., Green, C., Henke, R., Lew, T., Woo, J., Shepherd, B., et al. (2011). 2008–2009 Baccalaureate & beyond longitudinal study (B&B: 08/09). First look (NCES 2011-236). Washington, DC: National Center for Education Statistics, U.S. Department of Education.

    Google Scholar 

  • Cattaneo, M. D., Idrobo, N., & Titiunik, R. (2018). A practical introduction to regression discontinuity designs: Part I. In R. M. Alvarez & N. Beck (Eds.), Quantitative and computational methods for social science. Cambridge: Cambridge University Press.

    Google Scholar 

  • Chen, X. (2013). STEM attrition: College students’ paths into and out of STEM fields (NCES 2014-001). Washington, DC: National Center for Education Statistics, U.S. Department of Education.

    Google Scholar 

  • Chen, X., & Weko, T. (2009). Students who study science, technology, engineering, and mathematics (STEM) in postsecondary education (NCES 2009-161). Washington, DC: National Center for Education Statistics, U.S. Department of Education.

    Google Scholar 

  • Cheryan, S., Ziegler, S. A., Montoya, A. K., & Jiang, L. (2017). Why are some STEM fields more gender balanced than others? Psychological Bulletin,143(1), 1–35.

    Google Scholar 

  • Chi, O. L., & Dow, A. W. (2014). Improving balance in regression discontinuity design by matching: Estimating the effect of academic probation after the first year of college. Paper presented at the annual meeting of the Society for Research on Educational Effectiveness, Washington, DC.

  • College Transition Collaborative. (2018). Academic probation and the role of notification letters. Stanford, CA: Author. Retrieved from http://collegetransitioncollaborative.org/content/sass_toolkit_researchbrief_final.pdf.

  • Donovan, J. J., & Williams, K. J. (2003). Missing the mark: Effects of time and causal attributions on goal revision in response to goal-performance discrepancies. Journal of Applied Psychology,88(3), 379–390.

    Google Scholar 

  • Dweck, C. S. (2006). Mindset: The new psychology of success. New York, NY: Ballatine.

    Google Scholar 

  • Earl, W. R. (1988). Intrusive advising of freshmen in academic difficulty. NACADA Journal,8(2), 27–33.

    Google Scholar 

  • Feldman, R. S. (Ed.). (2018). The first year of college: Research, theory, and practice on improving the student experience and increasing retention. New York: Cambridge University Press.

    Google Scholar 

  • Fletcher, J. M., & Tokmouline, M. (2017). The effects of academic probation on college success: Regression discontinuity evidence from four Texas universities (IZA DP No. 11232). Bonn: IZA—Institute of Labor Economics.

    Google Scholar 

  • Friedman, B. A., & Mandel, R. G. (2009). The prediction of college student academic performance and retention: Application of expectancy and goal setting theories. Journal of College Student Retention: Research, Theory & Practice,11(2), 227–246.

    Google Scholar 

  • Frölich, M., & Huber, M. (2017). Including covariates in the regression discontinuity design (IZA DP No. 11138). Bonn: IZA—Institute of Labor Economics.

    Google Scholar 

  • Gahagan, J., & Hunter, M. S. (2006). The second-year experience: Turning attention to the academy’s middle children. About Campus,11(3), 17–22.

    Google Scholar 

  • Gordon, V. N., Habley, W. R., & Grites, T. J. (Eds.). (2011). Academic advising: A comprehensive handbook. New York: Wiley.

    Google Scholar 

  • Hayes, A. F. (2013). Introduction to mediation, moderation, and conditional process analysis: A regression-based approach. New York: Guilford.

    Google Scholar 

  • Heerman, C. E., & Maleki, R. B. (1994). Helping probationary university students succeed. Journal of Reading,37(8), 654–661.

    Google Scholar 

  • Holdren, J. P., & Lander, E. (2012). Engage to excel: Producing one million additional college graduates with degrees in science, technology, engineering, and mathematics. Washington, DC: President’s Council of Advisors on Science and Technology.

    Google Scholar 

  • Hollenbeck, J. R., Williams, C. R., & Klein, H. J. (1989). An empirical examination of the antecedents of commitment to difficult goals. Journal of Applied Psychology,74(1), 18–23.

    Google Scholar 

  • Humphrey, E. (2005). Project success: Helping probationary students achieve academic success. Journal of College Student Retention: Research, Theory & Practice,7(3), 147–163.

    Google Scholar 

  • Hwang, M. H., Lee, D., Lim, H. J., Seon, H. Y., Hutchison, B., & Pope, M. (2014). Academic underachievement and recovery: Student perspectives on effective career interventions. The Career Development Quarterly,62(1), 81–94.

    Google Scholar 

  • Isaak, M. I., Graves, K. M., & Mayers, B. O. (2006). Academic, motivational, and emotional problems identified by college students in academic jeopardy. Journal of College Student Retention: Research, Theory & Practice,8(2), 171–183.

    Google Scholar 

  • James, C., & Graham, S. (2010). An empirical study of students on academic probation. Journal of the First-Year Experience & Students in Transition,22(2), 71–91.

    Google Scholar 

  • Johnson, V. E. (2003). Grade inflation: A crisis in higher education. New York: Springer.

    Google Scholar 

  • Kamphoff, C. S., Hutson, B. L., Amundsen, S. A., & Atwood, J. A. (2007). A motivational/empowerment model applied to students on academic probation. Journal of College Student Retention: Research, Theory & Practice,8(4), 397–412.

    Google Scholar 

  • Kirk-Kuwaye, M., & Nishida, D. (2001). Effect of low and high advisor involvement on the academic performances of probation students. NACADA Journal,21(1–2), 40–45.

    Google Scholar 

  • Kot, F. C. (2014). The impact of centralized advising on first-year academic performance and second-year enrollment behavior. Research in Higher Education,55, 627–649.

    Google Scholar 

  • Kuh, G. D., & Hu, S. (2001). The effects of student-faculty interaction in the 1990s. Review of Higher Education,24(3), 309–332.

    Google Scholar 

  • Lee, M.-J. (2016). Matching, regression discontinuity, difference in differences, and beyond. New York: Oxford University Press.

    Google Scholar 

  • Lindo, J. M., Sanders, N. J., & Oreopoulos, P. (2010). Ability, gender, and performance standards: Evidence from academic probation. American Economic Journal: Applied Economics,2(2), 95–117.

    Google Scholar 

  • Locke, E. A. (1964). The relationship of intentions to motivation and affect. Unpublished doctoral dissertation, Cornell University, Ithaca, NY.

  • Locke, E. A., & Latham, G. P. (1990). A theory of goal setting and task performance. Upper Saddle River: Prentice-Hall.

    Google Scholar 

  • Locke, E., & Latham, G. (1994). Goal-setting theory. In J. B. Miner (Ed.), Organizational behavior 1: Essential theories of motivation and leadership (pp. 159–183). New York: Routledge.

    Google Scholar 

  • Locke, E. A., & Latham, G. P. (2002). Building a practically useful theory of goal setting and task motivation: A 35-year odyssey. American Psychologist,57(9), 705–717.

    Google Scholar 

  • Locke, E. A., & Latham, G. P. (2006). New directions in goal-setting theory. Current Directions in Psychological Science,15(5), 265–268.

    Google Scholar 

  • Mathies, C., Gardner, D., & Bauer, K. W. (2006). Retention and graduation: An examination of students who earn academic probation. Paper presented at 2006 SAIR Forum, Arlington, VA.

  • Mayhew, M. J., Rockenbach, A. N., Bowman, N. A., Seifert, T. A., & Wolniak, G. C., with Pascarella, E. T. & Terenzini, P. T. (2016). How college affects students (Vol. 3): 21stcentury evidence that higher education works. San Francisco, CA: Jossey-Bass.

  • McCrary, J. (2008). Manipulation of the running variables in the regression discontinuity design: A density test. Journal of Econometrics,142(2), 698–714.

    Google Scholar 

  • McGrath, S. M., & Burd, G. D. (2012). A success course for freshmen on academic probation: Persistence and graduation outcomes. NACADA Journal,32(1), 43–52.

    Google Scholar 

  • McShane, S. L., & Von Glinow, M. A. Y. (2005). Organizational behavior. New York: McGraw-Hill.

    Google Scholar 

  • Mellor, D. T., Brooks, W. R., Gray, S. A., & Jordan, R. C. (2015). Troubled transitions into college and the effects of a small intervention course. Journal of College Student Retention: Research, Theory & Practice,17(1), 44–63.

    Google Scholar 

  • Mento, A. J., Steel, R. P., & Karren, R. J. (1987). A meta-analytic study of the effects of goal setting on task performance: 1966–1984. Organizational Behavior and Human Decision Processes,39(1), 52–83.

    Google Scholar 

  • Molina, A., & Abelman, R. (2000). Style over substance in interventions for at-risk students: The impact of intrusiveness. NACADA Journal,20(2), 5–15.

    Google Scholar 

  • Morisano, D., Hirsh, J. B., Peterson, J. B., Pihl, R. O., & Shore, B. M. (2010). Setting, elaborating, and reflecting on personal goals improves academic performance. Journal of Applied Psychology,95(2), 255.

    Google Scholar 

  • Padgett, R. D., & Keup, J. R. (2011). 2009 national survey of first-year seminars: Ongoing efforts to support students in transition. Columbia, SC: National Resource Center for the First-Year Experience and Students in Transition, University of South Carolina.

  • Preacher, K. J., & Hayes, A. F. (2004). SPSS and SAS procedures for estimating indirect effects in simple mediation models. Behavior Research Methods, Instruments, and Computers,36, 717–731.

    Google Scholar 

  • Preuss, M., & Switalski, R. (2008). Academic probation intervention through academic assistance advising. Retrieved from ERIC database. (ED502891).

  • Ramirex, G. M., & Evans, R. J. (1988). Solving the probation puzzle: A student affirmative action program. NACADA Journal,8(2), 34–45.

    Google Scholar 

  • Richardson, M., Abraham, C., & Bond, R. (2012). Psychological correlates of university students’ academic performance: A systematic review and meta-analysis. Psychological Bulletin,138, 353–387.

    Google Scholar 

  • Robbins, S. B., Lauver, K., Le, H., Davis, D., Langley, R., & Carlstrom, A. (2004). Do psychosocial and study skill factors predict college outcomes? A meta-analysis. Psychological Bulletin,130(2), 261–288.

    Google Scholar 

  • Rojas, J. I., Knauft, D. A., Broder, J. M., & Campbell-Burden, B. (2002). Academic counseling for students within a college of agricultural and environmental sciences. NACTA Journal,46(3), 16–20.

    Google Scholar 

  • Schudde, L., & Scott-Clayton, J. (2016). Pell grants and performance-based scholarships? An examination of satisfactory academic progress requirements in the nation’s largest need-based aid program. Research in Higher Education,57(8), 943–967.

    Google Scholar 

  • Schwebel, D. C., Walburn, N. C., Klyce, K., & Jerrolds, K. L. (2012). Efficacy of advising outreach on student retention, academic progress and achievement, and frequency of advising contacts: A longitudinal randomized trial. NACADA Journal,32(2), 36–43.

    Google Scholar 

  • Scrivener, S., & Weiss, M. J. (2009). More guidance, better results? Three-year effects of an enhanced student services program at two community colleges. New York: MDRC.

    Google Scholar 

  • Smith, C. P., & Winterbottom, M. T. (1970). Personality characteristics of college students on academic probation. Journal of Personality,38(3), 379–391.

    Google Scholar 

  • Sneyers, E., & De Witte, K. (2018). Interventions in higher education and their effect on student success: A meta-analysis. Educational Review,70(2), 208–228.

    Google Scholar 

  • Snyder, T. D., de Brey, C., & Dillow, S. A. (2018). Digest of Education Statistics 2016 (NCES 2017-094). Washington, DC: National Center for Education Statistics, U.S. Department of Education.

    Google Scholar 

  • Sorrentino, D. M. (2006). The SEEK mentoring program: An application of the goal-setting theory. Journal of College Student Retention: Research, Theory & Practice,8(2), 241–250.

    Google Scholar 

  • Tinto, V. (2012). Completing college: Rethinking institutional action. Chicago: The University of Chicago Press.

    Google Scholar 

  • Tobolowsky, B. F. (2008). Sophomores in transition: The forgotten year. In B. O. Barefoot (Ed.), The first year and beyond: Rethinking the challenge of collegiate transition (New Directions for Higher Education) (Vol. 144, pp. 59–67). San Francisco, CA: Jossey-Bass.

    Google Scholar 

  • Trombley, C. M. (2000). Evaluating students on probation and determining intervention strategies: A comparison of probation and good standing students. Journal of College Student Retention: Research, Theory & Practice,2(3), 239–251.

    Google Scholar 

  • Tubbs, M. E. (1986). Goal setting: A meta-analytic examination of the empirical evidence. Journal of Applied Psychology,71(3), 474–483.

    Google Scholar 

  • Upcraft, M. L., Gardner, J. N., Barefoot, B. O., & Associates. (2005). Challenging and supporting the first-year student: A handbook for improving the first year of college. San Francisco, CA: Jossey-Bass.

    Google Scholar 

  • U.S. Census Bureau. (2014). 2010–2014 American Community Survey 5-Year Estimates. Washington, DC: Author.

    Google Scholar 

  • Vander Schee, B. A. (2007). Adding insight to intrusive advising and its effectiveness with students on probation. NACADA Journal,27(2), 50–59.

    Google Scholar 

  • Varney, J. (2007). Intrusive advising. Academic advising today. Retrieved from http://www.nacada.ksu.edu/Resources/Academic-Advising-Today/View-Articles/Intrusive-Advising.aspx.

  • Voyer, D., & Voyer, S. D. (2014). Gender differences in scholastic achievement: A meta-analysis. Psychological Bulletin,140(4), 1174–1204.

    Google Scholar 

  • Wlazelek, B. G., & Coulter, L. P. (1999). The role of counseling services for students in academic jeopardy: A preliminary study. Journal of College Counseling,2(1), 33–41.

    Google Scholar 

  • Young, J. W., with Kobrin, J. L. (2001). Differential validity, differential prediction, and college admission testing: A comprehensive review and analysis (Research Report No. 2001-6). New York: College Board.

  • Young-Jones, A. D., Burt, T. D., Dixon, S., & Hawthorne, M. J. (2013). Academic advising: Does it really impact student success? Quality Assurance in Education,21(1), 7–19.

    Google Scholar 

  • Zimmerman, B. J., Bandura, A., & Martinez-Pons, M. (1992). Self-motivation for academic attainment: The role of self-efficacy beliefs and personal goal setting. American Educational Research Journal,29(3), 663–676.

    Google Scholar 

Download references

Acknowledgement

The authors thank the students and faculty in the Center for Research on Undergraduate Education at the University of Iowa as well as Rocío Titiunik at the University of Michigan for their helpful feedback on this project.

Author information

Authors and Affiliations

  1. University of Iowa, Iowa, IA, USA

    Nicholas A. Bowman, Nayoung Jang, D. Martin Kivlighan III & Nancy Schneider

  2. Indiana University, Bloomington, IN, USA

    Xiaomeng Ye

  3. Department of Educational Policy and Leadership Studies, University of Iowa, N438A Lindquist Center, Iowa, IA, 52242, USA

    Nicholas A. Bowman

Authors
  1. Nicholas A. Bowman
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nayoung Jang
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. D. Martin Kivlighan III
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Nancy Schneider
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Xiaomeng Ye
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Nicholas A. Bowman.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Appendix

Appendix

See Table 6.

Table 6 Descriptive statistics for all variables in both studies
Full size table

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Bowman, N.A., Jang, N., Kivlighan, D.M. et al. The Impact of a Goal-Setting Intervention for Engineering Students on Academic Probation. Res High Educ 61, 142–166 (2020). https://doi.org/10.1007/s11162-019-09555-x

Download citation

  • Received:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11162-019-09555-x

Keywords

Search

Navigation