Introduction

Even before the global pandemic necessitated a radical shift in course delivery in the spring of 2020, online learning had become fundamental to postsecondary education. In the U.S., online learning has increased at a substantial rate over the last decade, and this is expected to continue: the 7% annual growth rate for online enrollments stands in marked contrast to the decline found in general post-secondary enrollments (Allen & Seaman, 2016). Pre COVID-19, between 28 and 36% of all post-secondary students in the U.S. enrolled in at least one online course annually (Allen & Seaman, 2016; Lederman, 2019; NCES, 2013) and online learning was a critical part of two-thirds of post-secondary institution’s long-term strategy (Allen & Seaman, 2010). Initially led by community colleges (Pearson, 2011; Shea & Bidjerano, 2014), online learning is now a reality for virtually all undergraduate institutions. Yet, because of the potentially high rates and costs of online attrition, there remains a need for institutions to be able to predict the potential persistence of online undergraduate students to foster inclusion and direct efforts towards ameliorating dropout (Hachey et al., 2013; Lee & Choi, 2011). Hence, enrollment and retention research–and the identification of influential factors to include in pre-enrollment predictive models–is critical.

Background: how do outcomes differ for online versus face-to-face instruction on average?

Course-level learning outcomes

There is support that students can learn as much online as they do face-to-face via comparison of course-level factors. An oft-cited meta-analysis of 232 online learning studies (Bernard et al., 2004) concluded no significant difference between student outcomes online versus face-to-face. After controlling for G.P.A, Driscoll et al. (2012) found grade difference of online students versus their face-to-face counterparts were statistically insignificant. Numerous other studies and meta-analyses support that there are no positive or negative effects on outcomes online versus face-to-face as measured by exam scores and aggregate course grades—the “no significant difference” claim (Ashby et al., 2011; Bowen et al., 2014; Brown, 2012; Cavanaugh & Jacquemin, 2015; Dell et al., 2010; Enriquez, 2010; Fish & Kang, 2014; Holmes & Reid, 2017; Jones & Long, 2013; Lack, 2013; Lyke & Frank, 2012; McCutcheon et al., 2015; McDonough et al., 2014; Meder, 2013; Mollenkopf et al., 2017; Nemetz et al., 2017; Rich & Dereshiwsky, 2011; Schmidt, 2012; Stack, 2015; Stocker, 2018; Tallent-Runnels et al., 2006; Wagner et al., 2011; Wu, 2015; Wuellner, 2013; Ya Ni, 2013; Zhao et al., 2005). Thus, collectively, there is a general consensus in the literature of little to no difference in student performance between instructional mediums for courses where both mediums are available.

Nonetheless, in an early review by Zhao et al. (2005) and supported by more recent findings, although aggregated data show no significant difference in outcomes between online and face-to-face learning, there continue to be some differences reported within studies, often connected to student characteristics. Several studies report higher exam scores or final grades in online courses compared to face-to-face courses (Holbert, 2017; Jorczak & Dupuis, 2014; Soffer & Nachmias, 2018). A meta-analysis of 176 studies from the Department of Education (2010) suggests that online learning outcomes are superior to those found in face-to-face courses. However, Jaggars and Bailey (2010) question these findings, as the only studies in the meta-analysis that were fully online and semester long (seven in total) did not find a difference between online vs. face-to-face course outcomes. Atchley et al. (2013) found differences in student outcomes between online vs. face-to-face courses, with online students earning more A, D and F grades, and Buchanan and Palmer (2017) report that median learning outcomes were similar but online assessments had larger standard deviations and higher F grades; yet, for both of these studies, differences did not necessarily mean better or worse overall outcomes between modalities.

Fendler et al. (2018) contend that while no significant difference may exist between instructional modes at the aggregate level, the choice between online and face-to-face course modes may matter at the individual level. A meta-review by Nguyen (2015) contends that characteristics such as gender, race/ethnicity, and ability moderate the learning outcomes for online learning. In line with this, Figlio et al. (2013) report that students with specific characteristics (Latinx, men, low achievers) do poorly online in comparison to face-to-face. Xu and Jaggars (2011b) report lower grades for online English and Math “gatekeeper” courses and further (2013, 2014) identify lower grades in online versus face-to-face course sections for specific populations (i.e., men, Black Students, younger students and students with lower G.P.A.s). Tanyel and Griffin (2014), in a 10-year longitudinal study, found final grades for students in face-to-face sections were significantly higher than online sections of the same course. Bettinger et al (2017) report that online learning produced lower grades on average and was more likely to negatively impact future enrollment than in-person courses. Similar findings of lower grades and negative impact on later enrollment are reported for both baccalaureate and community college students across a variety of disciplines (Alpert et al., 2016; Burns et al., 2013; Faidley, 2018; Francis et al., 2019; Garman, 2012; Hart et al., 2018; Keramidas, 2012). It is important to note that none of these studies compared outcomes within students, rather they compared outcomes in online and face-to-face courses by different students. While the inferences we can make about which characteristics put students at highest risk online from these particular studies are limited, they do suggest that a consideration of student characteristics maybe important when investigating outcomes.

Attrition rates

Course completion (i.e., successful finish of a course and final grade earned for which a student receives credit from the institution) is often used to measure the effectiveness of online courses (Boston & Ice, 2011). Attrition rates among online learners fluctuate between 40 and 80% (Smith, 2010) and research shows that online attrition may be 7–20 percentage points higher than the attrition found in face-to-face courses ( Anderson & Kim, 2006; Boston & Ice, 2011; Garman, 2012; Hachey et al., 2013; Jaggars et al., 2013; Johnson & Mejia, 2014; Kinder, 2013; Morris & Finnegan, 2009; Muse, 2003; Nora & Snyder, 2009; Patterson & McFadden, 2009; Smith, 2016; Smith & Ferguson, 2005; Tanyel & Griffin, 2014). Online dropout rates have been linked by some researchers to academic non-success (Angelino et al., 2007; Atchely et al., 2013; Carr, 2000; Diaz, 2002; Gregory, 2016; Jaggars & Bailey, 2010; Rovai, & Downey, 2010; Tyler-Smith, 2006; Wojciechowski & Palmer, 2005). In contrast to most of the literature, Moore and Fetzner (2009) present a review of six higher education institutions with 80% or greater online retention rates, Ashby et al. (2011) report better completion online, and others have found no differences in the completion rates between online and face-to-face students (Soffer & Nachman, 2018; Wilson & Allen, 2018). Despite the few noted exceptions, overwhelmingly, aggregate findings in the literature have demonstrated that students enrolled in online courses are more likely to drop the course than those enrolled in a face-to-face course. Yet, one major limitation of this research is that studies have largely not addressed differences in the characteristics of students who enroll in online courses (versus face-to-face courses).

College access and persistence

At least 74% of all U.S. undergraduate students have been found to have one (or more) non-traditional characteristic (U.S. Department of Education, 2015). [Non-traditional/underserved often includes students with any of the following characteristics: minorities; low-income; rural/inner-city; first generation; academically unprepared; the most common definition of non-traditional student characteristics is: delayed enrollment (age > 24); no high school diploma; part-time enrollment; financially independent; have dependents; single parent status; working full-time while enrolled (NCES, 1996, 2002)]. Online courses are often believed to increase college access, particularly for non-traditional and/or underserved students (Epper & Garn, 2003; Jaggars & Bailey, 2010; Johnson & Mejia, 2014; Parsad et al., 2008; Picciano et al., 2010). Many students who take online courses do so because they need the flexibility that these courses offer due to a wide range of life challenges that make it difficult to attend face-to-face courses (Brown, 2012; Burnette, 2015; Daymont & Blau, 2011; Fox, 2017; Jaggars, 2014; Jaggars & Bailey, 2010; Jaggars, 2013; Lei & Gupta, 2010; Pontes et al., 2010; Skopek & Schuhmann, 2008). Yet, there is little evidence to support the claim that online courses increase access (Jaggars, 2011), although improved access is often assumed (Allen & Seaman, 2010).

Moreover, it is still unclear how online offerings impact college persistence, as results have been very mixed (Jaggars, 2011; Shea & Bidjerano, 2014; Wladis et al., 2016). In some large-scale multi-institution studies (Jaggars & Xu, 2010; Xu & Jaggars, 2011a, 2011b), results suggest that online coursework may hinder subsequent semester re-enrollment and students who took a higher proportion of online courses were less likely to graduate or transfer to a senior college. Xu and Jaggars (2013) found that for community college students, students who were men, younger, Black, or had lower G.P.A.’s, struggled in online courses. Similar findings for community college students are reported by Huntington-Klein et al. (2017). Shea and Bidjerano (2019) report that minority community college students with higher online loads were more likely to drop out of college than non-minority students. Additionally, Smith (2016), using multi-institution data and controlling for student self-selection bias, found results for 4-year colleges that confirm Jaggars and Xu’s earlier findings of an overall negative impact of online enrollments on course and college retention.

However, these initial studies have not been confirmed in other recent research on academic momentum and degree completion which attempt to control for the student characteristics which often differ between online and face-to-face learners or that include nuanced analyses on online course taking and graduation attainment. Johnson and Mejia (2014), assessing all community colleges in the California system and controlling for demographics and G.P.A., found that in the short term, student outcomes were worse in online courses versus face-to-face courses. Yet, they also found that students who took at least some online courses were more likely than those who took only face-to-face courses to earn an associate degree or to transfer to a 4-year institution. Shea and Bidjerano (2014, 2016), using national data (rather than the state-wide datasets that other large-scale studies utilize) and controlling for student characteristics, also found that students (even those less academically prepared) who had taken some of their early courses online, attained degrees at significantly higher rates than those who did not. More recently, Shea and Bidjerano (2018) report that community college students who enrolled in online course loads up to 40% had better chances of completing their college degree (although it is important to note that beyond 40% online enrollment, the enhanced rate of degree completion was shown to begin to rapidly decline). They also found, on average, online course completion significantly increased the odds of completing a community college degree and transfer to a 4-year institution; more specifically, online course taking was shown to improve the degree completion/transfer of low G.P.A. part-time students relative to their full-time counterparts (Shea & Bidjerano, 2017, 2019). Also, using national data, Sublett (2019a) reports that community college students who engaged in online learning obtained their BA degrees faster than their peers who did not.

James et al. (2016) in a large scale, multi-institutional study including community colleges and 4-year institutions, report students taking a mix of online and face-to-face courses had higher retention and passing grades than students attending fully online or fully face-to-face, but differences were explained by student characteristics rather than delivery mode and thus, showed little evidence that the act of taking online courses itself placed students at greater risk of dropout. Similarly, Wladis et al. (2016), using data from a large, urban, east coast, multi-institution university system, report that it cannot be assumed that students who withdraw after taking online courses do so because of the outcomes of the online course; rather, other student characteristics (which drive the students to take online courses in the first place) may be significant in determining college persistence. Recent findings, which all attempt to control for student characteristics, suggest that restricting access to online courses could reduce academic momentum and student progress toward a degree. However, given the conflicting results, more research is needed (particularly accounting for student characteristics) before firm conclusions can be drawn about how enrollment in online courses impacts college persistence and degree completion.

Study purpose

This review initially arose from the question “What factors are important to include in any archetypical pre-enrollment predictive model of undergraduate online learning enrollment, retention and success? To date, a widely empirically tested, archetypical predictive model of retention and success does not exist for undergraduate online learning. Traditional models of persistence and retention (i.e., Bean & Metzner, 1985; Nora & Cabrera, 1996; Tinto, 1975, 1986, 1993, 2016) may not be applicable to online learning because these models emphasize the role of social integration, a process very different for the online student (Boston & Ice, 2011; Hachey et al., 2013; Rovai, 2002, 2003).

Scholars have posited online retention models, although none have been completely validated. Kember (1995) proposed a two-prong model of persistence for students in online courses based on how well students socially integrate school with their lives and the number of external demands they face. Kember’s own research found that there are likely other factors not included in the model (it did not account for 20% of the variance in student persistence). Further, Kember’s model has been empirically challenged, with Woodley et al. (2001) finding little internal consistency in the sub-scales for key constructs in the model. Rovai (2003) proposed a model of online student retention that synthesized traditional departure models with research on the skills and needs of online students; it included student characteristics, skills, and external factors as the major inputs that lead to internal factors affecting student persistence. Studies have validated and extended some of the external and internal factors identified in Rovai’s model (Lehman & Conceição, 2013; Packham et al., 2004), although other specific sub-factors that were identified have not been supported (Lee et al., 2013). More critically, both Kember and Rovai’s models are old and do not reflect recent knowledge about online learning, particularly related to modern social media phenomena (Lint, 2013).

Park (2007) proposed a model of online dropout based on Rovai’s model and a review of the online learning literature at that time. Efforts to validate Park’s model (Park & Choi, 2009) used a small sample and did not test all components. Money and Dean (2019) presented a model with major categories of antecedents and processes that they suggest may directly or indirectly impact online outcomes. Aligning with factors in Park’s model, they placed individual antecedents in seven broad categories and included broad categories of student internal processes (cognitive, affective, behavioral, and social). This model has yet to be validated and while the broad categories do reflect the literature, the model does not seem to account for the totality of potential predictor factors.

Research on students who succeed in the online environment has identified internal affective qualities such as self-discipline, independence, and motivation (Dabbagh, 2007; Johnson & Mejia, 2014; Stevens & Switzer, 2006) or skills such as time management and computer literacy (Dutton et al., 2002; Johnson & Galy, 2013), which (although not detailed) would likely fall under the internal process category in Money and Dean’s model. However, these internal qualities and skills may be no less applicable to success in the face-to-face environment than in the online environment or they may be less predictive than other factors (Kauffman, 2015; Park & Choi, 2009). Further, it is not well understood how such qualities and skills can be identified pre-enrollment; the effectiveness of screening tools is often assumed, but surveys developed to test student readiness for online learning are often costly, time consuming and may not be as effective in predicting student retention and success in online courses as other measures (Wladis & Samuels, 2016).

A cursory examination of early reviews devoted to online learning points to two broad conclusions. First, there are a plethora of variables that have been investigated as impacting retention and success in online learning; as Tyler-Smith (2006) contends, the reasons for attrition in online courses are numerous and complex. While many of the factors reported as impacting the online student’s decision to drop out likely contribute to online attrition, these variables may not easily be obtained pre-enrollment, nor may they account for most of the variability in retention and success. Second, discussions of early online learning research reveal a “lack of rigor” in research design (e.g., Jaggars & Bailey, 2010; Lack, 2013; Wu, 2015). This probably stems from the lack of controlled experimental designs. For example, in an oft-cited meta-analysis focused on learning outcomes (Means et al., 2010), the authors focus on 176 articles in their study, but a critique of this work (Jaggars & Bailey, 2010) notes that only seven of the studies included experimental/quasi-experimental designs and reflect conditions in typical postsecondary education. This may provide a partial explanation for why there is little consensus for which factors are significant and lead to persistence for online students and further, why there is currently no archetypical predictive model.

The ideal approach to analyzing the effects of the online environment on attrition would be one in which students were randomly assigned to either an online or a face-to-face section of a course. Some studies have attempted to do this with mixed results (i.e., Alpert et al., 2016; Arias et al., 2018; Figlio et al., 2013; Stack, 2015), though most randomized studies typically compare hybrid courses to face-to-face courses (see Bowen et al., 2014; Vernadakis et al., 2012). This is problematic because most online courses at colleges are fully online [80% or more online] rather than hybrid [30–79% online] (Instructional Technology Council, 2010; Lutzer et al., 2007), and because hybrid students seem to be comparable to face-to-face students in terms of demographics, course persistence and success (Jaggars & Xu, 2010; Xu & Jaggars, 2011a).

The dearth of large-scale randomized studies is likely due to the difficulties of recruiting students to participate in a study in which they could be randomly placed in a fully online or a face-to-face course. This is because a critical aspect of online learning is that students choose to enroll in the online modality (Fendler et al., 2018). Lack (2013) and Paulson and McCormick (2020) note that the differences in pre-existing background characteristics of students taking online versus face-to face courses are not trivial; the characteristics that drive students to enroll in online courses are likely correlated to course outcomes (Muljana & Luo, 2019; Nguyen, 2015; Smith, 2016; Wuellner, 2013). It has been posited that it is the characteristics of students who select online classes, rather than the instructional methodology itself, which impacts dropout (Jaggars & Bailey, 2010; McDonough et al., 2014; Wladis et al., 2015a).

A review of the literature from 1999 to 2009 conducted by Lee and Choi (2011) identified 69 possible factors from 35 empirical studies that may impact the online student’s decision to drop-out. However, Lee and Choi deliberately exclude findings related to student demographics (such as age, gender, or ethnicity), citing that the literature at the time offered inconsistent results. They are not incorrect; much of the research on the impact of student characteristics on online retention and success is conflicting (Jones, 2010). Yet, because findings are mixed does not mean that student characteristics should be completely ignored, nor does it mean that findings and related theorizing cannot be informative. This connects back to the earlier “lack of rigor” contention; because students opt into online courses and our lack of knowledge of what matters, many of the empirical studies on retention and success lack sufficient controls (Cavanaugh & Jacquemin, 2015; Lack, 2013; Wladis et al., 2016; Wu, 2015). This clarified for us that an integrative review of the literature focused on the question of “What student characteristics lead to online enrollment, retention, success and later college persistence?” is needed, both to foster better controls in future research and to help clarify potential factors for inclusion in an archetypical pre-enrollment predictive model of undergraduate online learning enrollment, retention and success.

Method of the review

Our goal was to be as inclusive as possible, providing a comprehensive view of the undergraduate literature for the last decade (January 2010–March 2021). In line with the methodology employed by Wallace (2003) and the guidelines for an integrative review as outlined by de Souza et al. (2010) and the PRISMA statement (Page et al., 2021), we present a wide-ranging overview that includes findings from previous reviews and research summaries, and any empirical research that addresses student factors in online learning that affect enrollment, retention, success and/or college persistence at institutions that also offer face-to-face courses in the U.S. (this review excludes research conducted at online-only institutions). The reasons for the inclusion of these types of work are two-fold. First, previous reviews and policy reports offer conclusions that are often used by researchers and policy makers as the basis for current interventions and research directions, and therefore, deserve to be noted. Second, we include all research studies that may offer insight, regardless of sample size or methodology or type. We do note multi-institutional or national datasets present findings that are likely more generalizable; therefore, these results may be more critical to the development of any archetypical predictive model (See the Appendix, Tables 5, 6).

To conduct the search (see Fig. 1), relevant articles on undergraduate students enrolled in online learning in the U.S. were first located through a search of publicly available literature published between January 2010 and March 2021. Using key words, searches were conducted in three educational online databases: ERIC, Education Full Text and PsychINFO; these databases were chosen because they include adult education and they are large and commonly searched educational databases devoted to peer-reviewed literature. The key words were: Higher Education or College or University or Postsecondary [Boolean operater AND] Online Learning [the Boolean operator OR] Distance Education [Boolean operater AND] Attrition [Boolean operators AND/OR] Retention [Boolean operators AND/OR] Persistence [Boolean operators AND/OR] Dropout [Boolean operators AND/OR] Learning Outcomes. Our search of the three online educational databases initially gathered 682 articles. These articles were then screened to remove duplicates and articles that fell outside the scope of the search criteria; screening of the initial collection of articles based on the described criteria reduced the total educational database gathered works to 17. For the initial screening, articles had to meet the following criteria: published 2010 or later; focus is on online learning/online courses, with online learning being classified as taking place over the Internet (i.e., exclude articles on self-paced video-only; hybrid or blended courses); focus is on typical, semester long, for-credit undergraduate courses at institutions that offer both online and face-to-face courses/programs (i.e., exclude articles on graduate populations and/or MOOCS and/or about online-only institutions); focus is on outcomes as measured by grade and/or completion and/or future enrollment (exclude articles reporting only perceived learning, satisfaction or engagement by faculty/students and/or only student affective factors/motivation data and/or only self-reported data); specifically reports the impact of student demographics/characteristics on enrollment, course success, course retention or future online persistence; enrollment takes place in the United States and the work is published in English.

Fig. 1
figure 1

Flowchart outlining the search protocol adopted in this integrative review

Full size image

Adopting Wallace’s (2003) comprehensive methodology, our search was supplemented by an ancestral/snowball review of the initial articles found and also works cited in several meta-reviews and summaries on research in online learning (i.e., Hart, 2012; Jaggars et al., 2013; Lee & Choi, 2011; Money & Dean, 2019; Muljana & Luo, 2019; Sublett, 2019b; Wu, 2015). Additional articles were gathered through manual searches in Google Scholar of the first five pages brought up in our searches [with the key words: “enrollment”, “attrition” in online learning]. Gathered articles were screened by our original criteria, and then we also applied an ancestral/snowball review to these articles. Finally, abstracts were manually reviewed based on our initial search criteria for articles published from January 2010 to March 2021 in the following journals (which focus on/often include the topic of online learning): American Journal of Distance Education; Computers & Education; Internet and Higher Education Journal; Online Journal of Distance Learning Administration; Online Learning (Journal of the Online Learning Consortium); Briefs from the Community College Research Center; Journal of Computing in Higher Education; Journal of Online Learning and Teaching; and The Journal of Educators Online. [Note: The Journal of Online Learning and Teaching merged with Online Learning, so an independent search of the former was conducted from 2010–2015]. As the search process progressed, duplications were removed. Note: although pre-dating 2010, we have chosen to include the meta-reviews of Bernard et al., (2004), Tallent-Runnels et al. (2006) and Zhao et al. (2005) due to the applicability of the findings and to provide critical historical context. This inclusion and exclusion process brought the number of articles detailed in the results section of this review to 83 (See study summaries in the Appendix, Tables 2, 3, 4, 5, 6).

Results: what are the characteristics of undergraduate students who enroll in online courses and their relation to online retention and success?

Except for emergency conditions such as pandemic driven necessity, students select to enroll in online courses (versus face-to-face courses). The research suggests that students who tend to enroll in online courses are likely on average different from those who enroll in face-to-face courses in terms of academic preparation/G.P.A., ethnicity/English language skills, gender, age, family/employment, part-time enrollment, non-traditional student characteristics and other characteristics such as socioeconomic status, prior online experience & credit accumulation. The literature on each of these factors is discussed and we share what is known about their relationship with undergraduate student enrollment and success in online courses.

Academic Preparation and G.P.A.

Enrollment

Several large-scale and multi-institution studies suggest that undergraduate students who enroll in online classes generally have higher G.P.A.s. than those who only take face-to-face courses (Ary & Brune, 2011; Jaggars & Xu, 2010; Xu & Jaggars, 2011a*). In more recent multi-institution studies, Johnson and Mejia (2014*) found that online course enrollment is more prevalent among more academically prepared students and Cavanaugh and Jacquemin (2015) report online students having higher G.P.A.s despite similar re-enrollment credit accumulation. On the other hand, some single course studies (Driscoll et al., 2012; Faidley, 2018; Helms, 2014; Johnson & Palmer, 2015) found that students who chose online courses had lower G.P.A.s and performed more poorly than face-to-face peers. Additionally, both a 10-year longitudinal study and a study using national data report that online students may be no more prepared or even less academically prepared than students who only take face-to-face courses (Shea & Bidjerano, 2014; Tanyel & Griffin, 2014).

Outcomes

Academic preparation (usually measured by G.P.A., SAT score and/or class rank), has been found to be a mediating variable affecting online grades and dropout rates (Bettinger et al., 2017; Figlio et al., 2013; Gregory, 2016; Harrell & Bower, 2011; Ice et al., 2012; Johnson & Mejia, 2014; Lee & Choi, 2011; Lint, 2013; Ryabov, 2012; Wladis et al., 2016; Xu & Jaggars, 2011a*), although it is unclear the extent to which this is any different in online courses than in face-to-face courses. Remedial placement (a strong indicator of academic preparedness) has been found to negatively impact online success as measured by lower grades and higher dropout (Jaggars & Xu, 2010; Xu & Jaggars, 2013, 2014*). Recently, Francis et al., (2019*) reported that online students in developmental math coursesFootnote 1 had worse outcomes than comparable face-to-face students, including lower grades and lower pass rates. Research also suggests that taking online developmental courses may negatively impact future online enrollment (Xu & Jaggars, 2011b*). Using multi-institutional data, Johnson and Mejia (2014*) found that students who were less academically prepared performed worse in online courses. Along the same line, Cavanaugh and Jacquemin (2015) report that students with higher G.P.A.s perform better in online courses compared to a face-to-face format. Overall, findings substantiate earlier assertions by Lee and Choi (2011), who in their review of the literature from 1999 to 2009, cite eight empirical studies that indicated that students with higher levels of academic preparation were less likely to drop-out of online courses.

Research has shown that G.P.A was a highly significant predictor of grades, for both online and face-to-face formats (Burns et al., 2013; Fendler et al., 2018; Wilson & Allen, 2018). Still, we note that there are nuances in the research. Figlio et al. (2013) found no significant difference between online and face-to face course formats among students with higher prior G.P.A.s. However, among those with lower G.P.A.s, students in the online course scored significantly lower on in-class exams than did those in the face-to-face section. G.P.A. has been noted as having a significant relationship with online course dropout (Harrell & Bower, 2011; Helms, 2014). Cochran et al. (2014) found that students with lower G.P.A.s were more likely to withdraw from online courses than those with higher G.P.A.s. Similarly, Wladis et al., (2016*) report that the weakest G.P.A. was associated with higher risk online, though the effect size in this study was small and students with G.P.A.s in the 2.0–3.0 range did not have significantly different risk online than students with higher or lower grades. Additionally, Hachey et al., (2013*) found that students in the 2.0–3.5 G.P.A. range had the highest proportional difference in attrition between online and face-to-face courses. For students with no prior online course experience, G.P.A. was a good predictor of future online course outcomes; but (corroborating earlier research) for students with previous online course experience, prior online course outcomes were a better predictor of future online course grades and retention than G.P.A. (Hachey et al., 2014, 2015*). While the literature indicates that G.P.A. can be a predictor of online course retention, there is no evidence that G.P.A. is any better at predicting student outcomes in online courses than in face-to-face courses.

Ethnicity & ESL

Enrollment

Available data generally seems to suggest that there is a proportional disparity in online enrollment by ethnicity, but the relationship between ethnicity and online success is less clear. Online learners are less likely than face-to-face learners to be Latinx or Black (Ashby et al., 2011; Cavanaugh & Jacquemin, 2015; Conway et al., 2011; Kaupp, 2012; Ortagus, 2017; Xu & Xu, 2019*). Jaggars and Xu (2010*) and Xu and Jaggars (2011a*, 2011b*), assessed students who took online courses at community and technical colleges in Virginia and Washington State, and reported that online students tended to be White. Similarly, Johnson and Mejia (2014*) report that online enrollments were proportionally lower for Black and Latinx California community college students than White students, with Latinx students lagging all other student groups in online enrollment. Within the New York SUNY system, Shea and Bidjerano (2017*) report that White community college students were much more likely to take both online and face-to-face courses than to be face-to-face only students. Using national data, Shea and Bidjerano (2014) found that the odds of taking an online course were lower for Black students in comparison to White Students. Specific to STEM online courses, Wladis et al. (2015b) found that Latinx and Black STEM majors were significantly less likely to take online courses than their White student peers. Further, using national data, results indicate that Black and Latino men students are particularly proportionally underrepresented in STEM online courses (Wladis et al., 2015c*). Additionally, ESL and immigrant status have been suggested as potentially having a negative effect on enrollment in online courses, as some research has shown that online students tend to be native English speakers (Xu & Jaggars, 2011a*).

Outcomes

Research regarding the impact of ethnicity on online success is mixed, with most studies failing to differentiate between overall relationships which may exist both online and face-to-face, versus those that may be specific to the online environment. Figlio et al. (2013) randomly assigned students in a single course and report that the face-to-face format produced more favorable outcomes than the online format for students who were lower-achieving, men, and Latinx (we note this study tested one narrow definition of online learning among a small sample from a single class at a single university). Several studies using statewide data from California community colleges found that Latinx and Black students in online classes had higher withdrawal rates and lower grades than their peers in face-to-face classes (Johnson & Mejia, 2014; Kaupp, 2012*); however, we note that online and face-to-face students may not be comparable on key characteristics. Ice et al. (2012) report race and ethnicity as a factor in online retention and success. Krajewski (2015*) found that non-White students were 2.4 times less likely to complete an online course than White Students and further, White students were 2.5 times more likely to earn a C or above in comparison to non-White students. Similarly, Xu and Jaggars (2013*) using multi-institutional datasets, found that Black students had poorer performance (i.e., lower grades, higher withdrawal) in comparison to White students in online courses. In the other direction, Yeboah and Smith (2016) found a positive relationship for minority students (defined as African American, Hispanic, Asian, Native American or Pacific Islander, including international students) between the number of online courses taken and academic performance. The results found in these studies may simply be reflecting general trends, and not ones that are specific to the online environment.

In contrast, Xu and Jaggars (2014*) report that for students of different ethnicities, although all types of students were more likely to drop out from an online course than a face-to-face course, the size of this difference did not significantly vary across ethnic groups. Similarly, Strang (2017) found that there was no interaction between ethnicity and course medium, suggesting that gaps in success rates between Latinx and White students were no greater online than face-to-face. Shea and Bidjerano (2017*), analyzing community college student data from the New York SUNY system, found that minority status was not a significant predictor of differences in G.P.A. between face-to-face and online courses. Lei and Gupta (2010) posited that the online environment can build a bridge between academia and students from diverse cultures which can lead to better academic outcomes. Supporting this, Wladis et al., (2016*) found that foreign-born students (with English as a second language) in a large, urban, U.S. university system were at less risk of dropout online than native-born students.

Gender

Enrollment

There is strong evidence that women (who enroll in higher education in greater numbers than men) are also represented to an even greater degree in the online environment. Numerous studies, which confirm a plethora of earlier research, indicate that a larger proportion of women students enroll in online courses (Ashby et al., 2011; Cavanaugh, & Jacquemin, 2015; Faidley, 2018; Helms, 2014; James et al., 2016; Pao, 2016; Ryabov, 2012; Streich, 2014*; Wilson & Allen, 2018. Within the New York SUNY system, Shea and Bidjerano (2017*) report that women community college students were much more likely to take both online and face-to-face courses than to be face-to-face only students. Jaggars and Xu (2010*), Johnson and Mejia (2014*), Smith (2016) and Xu and Jaggars (2011a*, 2011b*), all using different multi-institution state datasets, report that online courses are significantly more popular among women students in comparison to men. Shea and Bidjerano (2014), using national data, confirm that there is an overrepresentation of women among students who take online courses. Also using national data, Wladis et al. (2015b*, 2015c*) found that women STEM majors are significantly more likely than men STEM majors to take online courses, even when academic preparation, socioeconomic status (SES), citizenship and English-as-second-language status was controlled.

Outcomes

Research on the relationship between gender and online course retention and success is inconclusive. Ice et al. (2012) report gender as a factor in online retention and success. Wagner et al. (2011) and Gregory (2016*) found interaction of gender and course delivery method, with lower average grades for men online than in face-to-face courses. Using multi-institutional datasets, Xu and Jaggars (2013*) and Johnson and Mejia (2014*) found that women community college students had better online course persistence and grades than men students. Cochran et al. (2014) found that men were more likely to withdraw from online courses than women. Other studies report no relationship between gender and online course grades or completion rates (Burns et al., 2013; James et al., 2016; Krajewski, 2015; Park & Choi, 2009; Strang, 2017). Shea and Bidjerano (2017*), analyzing community college student data from the New York SUNY system, report gender was not a significant predictor of differences in G.P.A. between face-to-face and online courses. Specific to STEM online courses, one study found that women students did significantly worse online than would be expected based on their comparable face-to-face STEM outcomes [and yet, still no worse than men students] (Wladis et al., 2015a*). Palacios and Wood (2016*) found that community college men had higher success face-to-face in comparison to men online, even when accounting for ethnicity. While some studies cite no differences and others have found women outperform men, we found only one study (Fendler et al., 2018) that reports men obtaining better grades than women in online courses (at the 10% significance level); this study reported no gender differences observed in face-to-face courses.

Age

Enrollment

Research suggests that many students who take online courses do so because they need the flexibility that these courses offer due to life challenges that make it difficult to attend face-to-face courses (Jaggars & Bailey, 2010; Lei & Gupta, 2010; Pontes et al., 2010). Studies have found that online learners are more likely to be older (> 24), married with families, employed and with other responsibilities (Ashby et al., 2011*; Cavanaugh & Jacquemin, 2015; Driscoll et al., 2012; Faidley, 2018; Jaggars & Xu, 2010*; Johnson & Mejia, 2014*; Ortagus, 2017; Pao, 2016; Rich & Dereshiwsky, 2011; Smith, 2016; Tanyel & Griffin, 2014; Wilson & Allen, 2018; Xu & Jaggars, 2011a*). Within the New York SUNY system, Shea and Bidjerano (2017*) report that older community college students were much more likely to take both online and face-to-face courses than to be face-to-face only students. The current literature overwhelmingly confirms an early research review by Tallent-Runnels et al. (2006), which puts the average age of online students above the traditional college age range of 18–24 years.

Outcomes

Most large-scale studies indicate that on average students in online courses are older (> 25) and that older students tend to be more successful in the online environment than younger students, both in terms of grades and completion rates (Garman, 2012; Ice et al., 2012; Kinder, 2013*). Older students (> 25) have been reported to adapt more readily to online courses than younger students, and to benefit from higher levels of critical thinking, responsibility, and maturity (Kinder, 2013; Wuellner, 2013; Xu & Jaggars, 2013, 2014*). Johnson and Mejia (2014*) report that older students (> 24) did better than younger students in online courses, with a 4.6 percentage point achievement gap. In another multi-institutional study, James et al. (2016) report that among students taking only online courses, older students had better retention. Krajewski (2015*) reports that for every year of age, students were 1.1 times more likely to complete an online course. Xu and Jaggars (2014*) found that older online students did better than younger online students (but not to comparable face-to-face younger students). Specific to STEM online courses, Wladis et al., (2015a*) found that older students did significantly better online than would be expected based on their outcomes in comparable face-to-face STEM courses.

Park and Choi (2009), in a small-scale study, did not find age differences between those who dropped out and those who were retained in online courses. Similarly, Burns et al. (2013) and Strang (2017) found no impact of age on online performance. In a few exceptions to general trends, Francis et al., (2019*) found that older students had lower grades than both face-to-face peers and younger students in either delivery modality and Johnson and Galy (2013) report lower online grades for older Hispanic students in comparison to younger Hispanic students. Studies which attempted to compare trends by age online versus face-to-face seem to suggest that age is a factor. Yet, as with other characteristics, it is often difficult to determine the extent to which this relationship is specific to the online environment, or simply reflective of overall academic trends by age.Footnote 2

Family, employment

While most studies seem to point to greater success for older students, the mixed findings may be due to the added complexity of many older students lives: they are more likely to work, be married, have children and have other responsibilities that compete with time to devote to academics and impact decisions to dropout (Jaggars et al., 2013*; Lee & Choi, 2011; Smith, 2016). Students with the additional complexities of work, family commitments, and other life responsibilities have been shown to have significantly greater preference for the flexibility and convenience of online courses (Brown, 2012; Pontes et al., 2010). Enrollment in online courses helps these students better manage their busy schedules and provides better work/life balance (Helms, 2014; Jaggars, 2013*; Wladis et al., 2016*).

Nonetheless, time commitments outside of school also seem to impact their online academic performance: Wladis et al., (2016*) found that students with insufficient time for their academics (i.e., those with “time poverty” due to family and work responsibilities) may be more likely to drop an online course than a face-to-face class, all else being equal. Additionally, they found evidence that the number of credits/courses taken (increasing time poverty) may be a significant predictor of differential online versus face-to-face course retention, with students taking more credits/courses at higher risk of online dropout. They further found that parents of young children (under age six) were particularly at risk of online dropout; in contrast, Gregory (2016*) did not find an impact for having a dependent child (but being married made it more likely to earn an “A” online). Park and Choi (2009) found no difference in completion results with adult learners taking online courses related to their work. There is almost no research investigating the extent to which increased work and family commitments may simultaneously relate to online course enrollment and outcomes in online versus face-to-face courses, despite the extensive research showing that online students are significantly more likely to have work and family commitments. Thus, this is a significant area of research where more research is needed.

Part-time enrollment

Several studies have found that students who enrolled in online classes were employed for more hours than their face-to-face counterparts (Driscoll et al., 2012; Xu & Jaggars, 2011a, 2011b*) and as a result are more likely to attend part-time, which may impact enrollment and persistence. Part-time enrollment, for example, may be a weak proxy variable for time poverty, which may significantly relate to both online enrollment and course outcomes. Johnson and Mejia (2014*) report that part-time students do significantly worse in online courses. Similarly, Krajewski (2015*) found that full-time students were 2.1 times more likely to complete an online course than part-time students. Research suggests that without adequate support (e.g., childcare; financial aid to reduce work hours), the flexibility that online courses offer may not be enough to compensate for the time demands of non-academic responsibilities (Wladis et al., 2016*). Still, for students juggling school, family, and work obligations, the ability to maintain a full-time course load by taking some online courses each term may outweigh any theoretical risks of higher attrition online in terms of actual degree attainment (Johnson & Mejia, 2014*). Perhaps in support of this, within the New York SUNY system, Shea and Bidjerano (2017*) found that full-time community college students were much more likely to take both online and face-to-face courses than to be face-to-face only students.Footnote 3

Non-traditional characteristics

As noted earlier, there are seven characteristics typically considered non-traditional: delayed enrollment (> age 24); no high school diploma; part-time enrollment; financially independent; have dependents; single parent status; working full-time while enrolled (NCES, 1996, 2002). While some of these characteristics were addressed above, studies have often looked at these characteristics as a group, as there is evidence that online learners are more likely to possess non-traditional student characteristics (Pao, 2016; Pontes et al., 2010; Rovai & Downey, 2010; Xu & Jaggars, 2013). Jaggars and Xu (2010*), Xu and Jaggars (2011a*, 2011b*) and Streich (2014*), using multi-institutional state data, report that being financially independent, having dependents and/or being employed was associated with online course taking. Shea and Bidjerano (2014), using national data, report an overrepresentation of online students in comparison to face-to-face students in six out of the seven categories of non-traditional characteristics. Wladis et al. (2015b*, 2015c*), also using national data, found that having non-traditional student characteristics strongly increased the likelihood of enrolling in a STEM course online, more so than any other characteristic, with online enrollment probability increasing steeply as the number of non-traditional factors increased. Further, they found that the impact of non-traditional factors on online enrollment was significantly stronger for STEM than non-STEM majors. Wladis et al. (2015b*) contend that as non-traditional students are more likely to be women and non-White, that non-traditional characteristics may serve as a mediating variable for differences in online enrollment by gender and ethnicity. Further, military students, also shown to often have non-traditional characteristics, have overwhelmingly moved to online learning modalities, yet there is little specific research on this population (Ford & Vignare, 2015) [Outside our search parameters due to self-report methodology, we do note the only found research related to military students (Wright, 2015; See the Appendix, Table 7) which suggests a connection between military duty, online enrollment, and dropout.]

There is concern about the online course performance of underprepared and/or traditionally underserved students who are already at higher risk of course dropout (Jaggars & Bailey, 2010*). Evidence that non-traditional students tend to enroll in online courses, combined with research reporting that non-traditional students are more likely to be non-White and women, as well as significantly more vulnerable to negative academic outcomes (NCES, 1996, 2002) suggests that non-traditional characteristics may be a mediating variable for differences in online outcomes by gender and ethnicity (Wladis et al., 2015c*). Yet, data on the effect of all non-traditional characteristics on online enrollment and persistence to date is inconsistent across the literature (Wladis et al., 2015c*) or generally missing. The high proportion of non-traditional students who enroll online compared to face-to-face students should make us cautious about generalizing inferences based on simple comparisons of online versus face-to-face students, as these two populations are significantly different; imposing norms developed with traditional students on non-traditional students may lead to invalid inferences.

Other characteristics (SES, prior online experience & credit accumulation)

Other characteristics that warrant further exploration include socioeconomic status (SES), prior online course taking experience, and credit accumulation. SES has been posited as an important mediating variable when considering online course enrollment and ethnicity (Wladis et al., 2015b*). Ortagus (2017) found increasing participation of low-income students in comparison to middle/high income students in online courses, though low-income students were also less likely to enroll completely online. Shea and Bidjerano (2017*) found that community college students within the New York SUNY system who were Pell Grant recipients were much more likely to take both online and face-to-face courses than to be face-to-face only students. Additionally, several large-scale and national data research studies have reported that in comparison to face-to-face enrollment, students enrolling online are more likely to have applied for, or received, financial aid (Jaggars & Xu, 2010; Shea & Bidjerano, 2014; Smith, 2016; Streich, 2014; Xu & Jaggars, 2011a*).

Krajewski (2015*) did not find financial aid was related to online completion but did report that Pell grant recipients were 2.7 times less likely to achieve a C or better in an online course. Similarly, Gregory (2016*) reports that students ineligible for Pell Grants were more likely to earn As, Bs and C’s in online courses than those who were eligible. However, Burns et al. (2013) report that Pell Grant eligibility was not a significant factor in online grades. Cochran et al. (2014) found that students with loans were more likely to withdraw from online courses than those without loans. Wladis et al. (2016) and James et al. (2016) report that while household income and Pell Grant eligible were strongly correlated with course and college outcomes generally, it was not relevant to the online environment specifically. This does not eliminate SES as a factor, yet it may simply be that lower-income students need significant support regardless of course modality.

Both prior online experience and college experience generally may have an impact on online success. Several studies report that students with a greater number of completed higher education credits had better performance in terms of grades and/or retention online (Cochran et al., 2014; Fendler et al., 2018; Fetzner, 2013*; Wilson & Allen, 2018), though Burns et al. (2013) did not find an effect for class status. Krajewski (2015*) found that students in their first term were 2.6 times less likely to complete an online course than students who completed at least one semester. We note it is difficult to determine whether these patterns are specific to the online medium, or just reflective of a relationship between course outcomes and college level.

Previous online course experience may particularly have a significant impact on future online success (Burns et al., 2013; Lint, 2013*). Wang et al. (2013) report that online experience directly effects self-regulation, which impacts course performance. Hachey et al., (2012*) report that knowing a student’s prior online course success explains 13.2% of the variation found in retention and 24.8% of the variation found in online success, over and above GPA alone, making prior experience a significant predictor variable, which is more specific than prior academic outcomes more generally. Students with no previous online course experience (including new freshman) or those with prior unsuccessful online experience (i.e., course withdrawal or low grade) have been found to be at greatest risk (Cochran et al., 2014; Goomas & Clayton, 2013*; Hachey et al., 2012*, 2014*; Ice et al., 2012; Jaggars, 2011*), suggesting that this may be an important factor to consider in future models.

Limitations

In applying Wallace (2003) and de Souza’s et al. (2010) integrative methodology, we attempted to cast a wide net, conducting a comprehensive review of the literature in the U.S. for the past decade related to student characteristics which may impact the decision to enroll and subsequently succeed in online learning. To this end, we employed multiple search techniques. As all-encompassing as our method tried to be, it still cannot be certain that all relevant literature has been gathered within our defined search parameters. Further, aspects of the search criteria utilized in our protocol in their own way serve as limitations; by definition, setting exclusions means that some data is lost. We note that while this review covered a large time span, it does not include articles prior to 2010 barring a few exceptions, so all knowledge gained in previous decades are not accounted for in this study. And, while we selected databases and additional meta-reviews and journals to manually search that were likely to contain articles relevant to this review, there may have been other databases and journals that were not included that could have contributed to our findings. Additionally, we acknowledge that our search criteria of excluding studies outside the U.S. or not written in English likely filtered out studies pertinent to online student characteristics and in this way, does not allow generalizability of our findings to global contexts. Finally, our findings may only be applied to undergraduate students at institutions that offer both online and face-to-face courses, thus excluding graduate and online-only sectors in higher education. That said, the multiple approaches utilized for searching and the protocol adopted did provide a relatively large number of articles gathered and discussed in this review and thus, serves to shed some light on the online learning literature in the U.S. that focuses on undergraduate student characteristics and the potential relationships to enrollment and outcomes.

Concluding thoughts and future directions

The results of this integrative review, though in many ways inconclusive, are also informative. We identified ten student characteristics which most past research suggests is relevant in describing online undergraduate students (G.P.A./academic preparation; ethnicity; ESL; gender; non-traditional status; age; family responsibilities; employment; student level; and SES). Thus, this review provides more systematic evidence that certain student characteristics are a factor significantly differentiating those who enroll online versus those who enroll face-to-face. Further, while Money and Dean (2019) provide several likely broad categories of antecedents in their model of student differences for learning in online education, this review enlarges their limited sub-category discussion of demographic attributes that are likely to have an impact on outcomes, expanding to a more comprehensive set of categories.

To accurately assess whether a factor puts an undergraduate student at greater risk in the online environment, it is essential to analyze the interaction between that factor and course medium, while simultaneously controlling for the characteristics of students who select online courses (Wladis et al., 2016). However, clear evidence of the potential impact of the characteristics of undergraduate students who select online courses has yet to be determined; the research for many of the variables influencing outcomes is indeed mixed (See Table 1). Further, most of the studies found have limited potential for more generalized inferences (e.g., because they only considered patterns in online courses and direct face-to-face controls; because they compared the same courses taken by different students without thoroughly controlling for all of the factors that correlate with student selection into online courses). That said, there seems to be evidence that the ten student variables identified are sufficiently correlated with both online enrollment and academic outcomes such that they cannot completely be ignored in any online enrollment/retention model. Thus, this review supports previous assertions that several student characteristics have emerged which need further empirical testing, both regarding their impact and the predictive power these factors may have in terms of identifying students at highest risk online (Hachey et al., 2014; Street, 2010).

Table 1 Overall summary description of findings related to identified student characteristics
Full size table

The characteristics we identify above, plus the prior online course experience factor, combined with the additional factors (69 additional factors, including items related to course design and institutional support) identified in the review by Lee and Choi (2011) offer insight into the creation of an archetypical predictive model of retention and success for undergraduate online learning. However, before such a model can be constructed, further work is needed. First, more large-scale, well controlled empirical research is needed to narrow down which student characteristics truly do impact outcomes in the online course modality and specifically, in what direction and to what extent. Many of the supports or interventions which are relevant to all students (e.g., supporting low-income students, students with lower levels of college preparation, minoritized students, etc.) remain equally relevant to online students, as this literature review demonstrates. What is less clear is which specific student characteristics may be most relevant to predicting differential outcomes online versus face-to-face. While some of the studies in this review have attempted to investigate this question, they are in the minority, and many of them suffer from critical limitations of the groups to which inferences can be generalized.

In the absence of large-scale randomized control trials (RCTs) and clearer evidence, this study suggests that future observational studies should be careful to systematically measure and control for all of the student-level demographic variables identified in this review which have been shown to differ between online and face-to-face students. While many of the characteristics of undergraduate students who enroll in online courses are readily available through routine and/or automatized institutional research data collections, others (e.g., work and family commitments) are less readily available, but could become readily available if they were automatically included in routine data collection efforts (e.g., colleges could ask these questions on routine admissions surveys, or federal forms such as the FAFSA could collect more specific information about ages and numbers of children, for example).

In addition, both relevant student characteristics and those possible factors identified by Lee and Choi (2011) need to be further assessed to find out which of the multitude are most predictive of academic outcomes. An archetypical model with 79+ variables is overwhelming and likely of little practical use to advisors and policy makers at undergraduate institutions; such a model need not include every possible variable but only those that capture the most variability in outcomes to be accurate enough to be predictive of those at highest risk. Finally, effective and targeted interventions need to be developed and empirically tested to support those undergraduate students identified as being at most risk of drop-out in the online environment. Because of the critical role that online courses may be playing in providing students access to college who might otherwise not have been able to take particular courses face-to-face, it is essential that we consider ways to provide all students with the supports that they need to succeed in this environment. Until this work is accomplished, there is likely to be little reduction in online attrition.