Keywords

1 Introduction

Science, technology, engineering, and mathematics (STEM) workers are no longer only bench scientists and engineers with bachelor’s and graduate degrees, but also include engineering technicians, systems administrators, computer specialists, and others for whose skills can be obtained at the sub-baccalaureate level. Students who have a preexisting interest and ability in STEM, but who may not represent the traditional profile of STEM workers must be strongly encouraged to persistent in STEM careers. Electronic mentoring (e-mentoring) is one very effective practice for supporting secondary and postsecondary underrepresented students’ persistence in STEM majors [1]. As more students use on-line learning for instruction, enhancing on-line support services such as e-mentoring has direct and indirect outcomes for student engagement and retention in STEM majors.

The Georgia STEM Access Alliance (GSAA) is a collaborative project between The University of Georgia (UGA) and Georgia Institute of Technology (Georgia Tech) to develop an e-mentoring student resource model that connects underrepresented students with mentors across distinct secondary and postsecondary institutions. The GSAA project, branded as BreakThru, was a five-year grant funded by the National Science Foundation. The purpose of this paper is to examine the BreakThru e-mentoring model for scalability purposes. Two aspects of a STEM e-mentoring program were examined: (1) the use of virtual environments and social media platforms; and (2) the development of e-mentoring relationships.

1.1 Participants

BreakThru student participants (n = 188) were enrolled in an e-mentoring program to increase the persistence of underrepresented secondary and postsecondary individuals in STEM majors. Student enrollment was restricted based on available grant resources. In addition, all mentors and mentees (students) were provided a financial incentive to participate in the program. Secondary mentees were selected from three distinct districts (i.e., rural, urban, suburb) with student enrollments of 500, 3,000, and 45,000 respectively. Postsecondary students were selected from three institutions including one open-enrollment two-year college with approximately 26,000 students, and two research universities with student enrollments of 35,000 and 21,000. Table 1 provides evidence that more males from the secondary schools and more females from the postsecondary institutions self-selected to participate in the BreakThru e-mentoring program. In addition, the race/ethnicity demographics (Table 1) illustrate that a larger number of minority students were represented in BreakThru.

Table 1. Student Gender by Race/Ethnicity across all years. *Majority = White or Asian; Minority = Black, Hawaiian, Hispanic, Other, Two or more races, Native American/Alaskan Native. ** Ethnicity data was not reported for 2 secondary females, 3 postsecondary males, and 3 postsecondary female so they are not included in the majority/minority data, but are included in the totals for gender data.
Full size table

Mentors (n = 33) were recruited from postsecondary faculty, graduate students, secondary teachers, and business leaders. The secondary mentors included 69 % females, 62 % majority racial/ethnic populations, and 4 % individuals with disabilities. The postsecondary mentors represented 53 % females, 62 % majority racial/ethnic populations, and 10 % individuals with disabilities.

1.2 BreakThru E-Mentoring Program

The key components to the BreakThru e-mentoring program included provision of online learning and training practices, access to virtual environments and use of social media platforms to promote networks of support, and virtual linkage to STEM resources. Essential to the mentor and student engagement was their collaborative use of the on-line STEM learning modules. During the development year of the grant, the project staff developed 12 discrete online learning modules. Four of the modules were identified as critical modules and were program requirements: accommodations, time management, introduction to STEM, and self-determination. In addition to these critical modules, early in the program mentors reported that they were encouraging their mentees to complete the math/science test anxiety module. All modules included universally-designed online, mobile device, and Second Life formats.

1.3 BreakThru Data Collection

Institutional descriptive data and monthly programmatic reports were collected over the five-year span. An on-line instrument was administered to all the students at the end of each of the semesters to provide detailed intervention data (i.e., engagement and quality of mentoring). All mentors were required to complete a survey to provide feedback about each of their assigned mentees. The primary purpose of the mentor survey was to investigate the number of mentoring sessions, the communication platform mediums used for mentoring, and the length of mentoring sessions when certain mediums were used. Investigating the usage patterns of different communication platforms during e-mentoring provides a means of understanding the specific resources critical for such a practice.

1.4 Institutional Demographics

The BreakThru student demographics in Table 1 should be interpreted in relation to the demographics of the various participating institutions. The secondary institutions varied greatly in minority student enrollment (rural = 86 %; urban = 80 %; suburban = 54 %), but have similar proportions of students with disabilities (SwD) (rural = 11 %; urban = 15 %; suburban = 11 %). The post-secondary institutions have more distinct profiles of minority enrollment (two-year = 68 %; research universities = 20 %) and populations with disabilities (two-year = 1.9 %; research = 3.0-3.6 %). The total SwD enrolled in STEM majors (two-year = 2.5 %; research = 3.0 %; 9.2 %) compared to the percentage of SwD STEM graduating (two-year = 1.1 %; research = 1.5 %; 6.5 %) illustrates a significant need for student on-line resource support for these underrepresented populations.

2 BreakThru Model

We investigated the BreakThru model specific to the incremental cost of the model over time (i.e., five years) and the specific resources (i.e., grant leadership; design; and program administration) critical for student retention in the mentoring process (See Fig. 1). Year one of the grant was devoted to the development of the virtual platforms, on-line resources, and recruitment. No students or mentors were enrolled during year one of the project. The relationship of the following model variables to student retention was investigated: (1) participant demographics: gender; disability, institution; and race/ethnicity; (2) virtual resources; and (3) mentorship intervention (i.e., engagement and quality).

Fig. 1.
figure 1

BreakThru model

Full size image

2.1 Allocation of Resources

The resources allocated to support the BreakThru model are categorized as representing leadership, program management, and virtual design (see Fig. 2). The leadership resources were assigned to the grant administrative and research roles, the program management to the e-mentoring activities, and design to the development and management of the virtual world and social media platforms. The increase in leadership resources during years four and five was a function of grant research requirements. Design resources decreased over the five years of the project since platforms were created primarily during year one with iterative changes and maintenance being the focus over years two through five. The design products (i.e., virtual modules, on-line resources), are student support resources currently available on-line for use in replicating the model [2].

Fig. 2.
figure 2

Grant budget

Full size image

The program management resources represent the scalable portion of the BreakThru e-mentoring program (see Fig. 3). Over the five years of the project, we tracked four essential components of developing and maintaining an e-mentoring program: communication activities; incentives; program evaluation; and staffing requirements. Some of the increase in program evaluation resources were a function of grant requirements and would not be essential for replicating this model.

Fig. 3.
figure 3

Program budget for e-mentoring activities

Full size image

2.2 Student Recruitment and Retention

BreakThru served 91 secondary students, 56 two-year postsecondary students, and 38 research institution post-secondary students. Fifty-seven percent of the BreakThru students continued in the program for multiple years. Enrollment in the BreakThru secondary and postsecondary institutions is provided in Fig. 4. The recruitment for secondary, and postsecondary students required very different communication strategies related to the unique institutional demographics and policies. Overall, we found one of the best recruiting communication strategies required the development of a short, visual (e.g., YouTube, video) message stressing virtual mentoring, avoiding the use of terms such as disability or remedial in the messaging. More resources were allocated during year one and two of the project to meet our grant goals specific to student enrollment (see Fig. 2, communication). However, at the end of the 2013-2014 academic year, the grant goal for enrollment of postsecondary students at each of the research institutions was approximately 30 % under the grant goal. Therefore, additional recruitment resources (i.e., staff time) were reallocated during the fourth year (see Fig. 2).

Fig. 4.
figure 4

BreakThru participant enrollment across institutions and years

Full size image

Primarily two types of recruiting were used in the project, active and passive strategies. Active recruitment required staff to meet face-to-face (FtF) with students to describe the mentoring activities. In this sense, active recruitment became the first stage toward the development of a relationship between the student and the program. We found it useful to conceptual active mentoring as enlisting faculty and advisors to invite students to participate in e-mentoring activities. Approximately 75 % of the students were actively recruited into participating in the BreakThru e-mentoring by faculty members or secondary teachers. One active recruiting method that did not yield positive results was eliciting the Disability Service Office as a recruitment resource. Unlike active methods, passive recruitment methods rely on other media, and do not involve direct interaction between a staff member and a potential participant. Examples include flyers, social media messages, or blanket emails sent to a group of students. One drawback is that passive recruiting requires a student to be in the right place at the right time in order to see a message on a plasma screen, social media platform, or a flyer. We found the passive strategy of using social media platforms (e.g., website, Facebook, Skype) not very effective for recruitment.

Patterns in retention vary across the different types of institutions (see Fig. 5). Of the secondary schools, the suburban school system, the largest district population, had more student involvement and retained these individuals consistently across all years of the project. It is important to note that both the urban and rural secondary school districts demonstrated greater difficulty accessing virtual and social media platforms essential for the mentoring process as a result of school district broadband restrictions and home resources (e.g., internet access). Of the postsecondary institutions, the two-year institution was the most active in recruitment and retention of students for the program during year one as result of institutional program support. The two-year college students who continued in the program past year one remained active during the other three years of the program. While the research institutions did not have high initial recruitment figures, they recruited more students each year, and the majority of the students did stay throughout the four years with plans for graduate degrees. Therefore, the BreakThru model was effective for both two-year and research institutions.

Fig. 5.
figure 5

Student retention in BreakThru e-mentoring across institutions and years. Note: this visualization shows total number of students and does not distinguish between students who graduated and students who chose to continue

Full size image

2.3 Diversity of Population

Race, gender, and disability have different relationships with individual interest and retention in the BreakThru e-mentoring program. The majority of students at both the secondary and postsecondary levels interested in and remaining active throughout the four years of BreakThru were minority students, and of this population the African American students represented the largest group at both the secondary and postsecondary institutions (see Table 1). As noted earlier, males participated more in the secondary schools and females at the postsecondary institutions (see Table 1). The population of students with learning disabilities, attention deficit/hyperactivity disorder (ADHD), and autism spectrum disorder (ASD) represented the largest numbers of individuals at both the secondary and postsecondary levels who self-selected to participate and remain active in e-mentoring activities (Fig. 6).

Fig. 6.
figure 6

Students with disabilities

Full size image

2.4 Virtual Platform Usage

The e-mentoring sessions were provided through either digital voice communication platforms (e.g., Second Life voice, smart phone, video calls) or text-based communication platforms (e.g., emails, social media posts, Second Life chat posts, text message conversation threads). The use of Second Life was encouraged as part of the grant goals and activities. The e-mentoring island in Second Life was a virtual platform where individuals interacted with each other through avatars. Avatars communicated through voice (sue of a microphone) or by chat threads (written communication). To ensure that mentors and mentees had access to communication platforms other than the e-mentoring island, participants were provided options for social network sites such as Facebook, Google, Skype, Twitter, YouTube and/or a virtual learning environment on the program website, including virtual learning modules, a blog, and other support resources. Mentoring pairs were encouraged to find a platform that was beneficial to their mentoring activities. We examined the reflections of the secondary and postsecondary participants across these different communication platforms as reported by mentors through a monthly survey tool collected at the end of each academic semester (see Table 2). The percentages represent the total number of responses received for a given item out of the total number of responses received for that survey. These totals are aggregated across all data collection time points and are disaggregated by mentor/mentee responses and secondary/post-secondary responses.

The majority of the participants chose digital tools that were easily accessed on their smart phone. For instance, the chat feature on social media sites such as Facebook, were often used as a way to type quick messages between participants. Those messages, whether real-time or asynchronous, allowed the participants to engage in unscheduled, quick-response, and private one-on-one mentoring with little to no fiscal resources required by the program. No significant differences were noted across gender or race/ethnicity.

Second Life was the only social media platform that incurred a cost to the project in either design or management. In addition, the learning curve for using Second Life was steep, and it was not a tool preferred by the students. The cost of developing and managing Second Life resources was disproportionate to student or mentor usage. We do not support Second Life as a communication platform for e-mentoring at either the secondary or postsecondary levels. Second-Life is just one example of why platform-dependent resources may not be a wise investment.

Table 2. Survey Responses to “Select all the ways you communicate with your mentor/mentee.” Note: time points represent multiple time points not unique participants.
Full size table

2.5 Student Engagement

Student engagement was measured by the number of modules completed across the four years of the project (see Fig. 7). We chose module usage since a metric such as the number of meeting times across virtual platforms did not appear to provide a reliable or valid index of program engagement. Figure 7 represents the module completion as reported by the mentor surveys, and as can be seen in the figure, there was a low response rate on the surveys. This low response rate speaks to the need for automatic real-time data collection methods rather than relying on self-report in surveys at the end of each semester.

Fig. 7.
figure 7

Module completion reported across 2012-2014

Full size image

Interestingly, we did find that of the participants reporting module completion, there appeared to be two very different ways that mentee and mentors engaged with the modules. One group of participants completed more than the required critical modules, but this group of students primarily remained in the program for only one year. The second group of students completed only the four critical modules, but this group of students remained in the program longer, many for all four years. The two different groups of students appear to be focused on different e-mentoring goals. As we noted in a previous study, many of the BreakThru participants reported a number of positive benefits in addition to persisting in STEM majors. For instance, many of the participants reported the development of trusting and supportive relationships [3].

2.6 Quality of E-Mentoring

The quality of the BreakThru e-mentoring program was measured on the mentee survey by the question, “I am satisfied with my mentoring experience”. Mentees active in the program were requested to complete the mentee survey each semester. Out of the 188 mentees who participated across the four years of the program, only 33 % completed at least one mentee survey. However, the vast majority (85 %) of those mentees completing a survey either strongly agreed (53 %) or agreed (33 %) that they were satisfied with their mentoring experience. The mentees most recent survey was used in the analyses. The low response rate for evaluating the quality of the program again speaks more to the need for automatic data collection methods rather than relying on self-report in surveys at the end of each semester.

3 Findings

The scalability of the BreakThru model was investigated specific to the incremental cost over time (i.e., five years) and the specific resources (i.e., grant leadership; design; and program administration) critical for student retention in the mentoring process. Results indicate that such an e-mentoring model is effective in the recruitment and retention of students in STEM majors and provides implications for the use of virtual student support resources for faculty and graduate students providing on-line instruction. The student retention rate (57 % over five years) suggests that the model was effective in recruiting and retaining students at both the secondary and postsecondary levels. As noted earlier, those students who did not continue in BreakThru seemed to treat the e-mentoring program as a short-term training exercise rather than an opportunity to develop a long-term relationship. Underrepresented populations (i.e., minority, students with cognitive disabilities) were the largest group of individuals self-selecting to participate in the program.

Two aspects of a STEM e-mentoring program were specifically examined: (1) the use of virtual environments and social media platforms; and (2) the development of e-mentoring relationships (i.e., quality, engagement). The mentors’ monthly surveys provided the evidence that text-based tools were often the most frequently used e-mentoring tools. It appears that one of the main reasons for using text-based tools was the practicality, user friendliness, and familiarity with the platforms. Email and chat platforms offered instant access to the participants, and their monthly surveys revealed that mentors and mentees spoke regularly though these mediums. Ease of use, availability, and disability accommodation needs all played a role in the determination of what type of communication tool to use and when to use it. Age, race/ethnicity, or gender did not appear to factor into the choice of a specific communication platform.

Race, gender, and disability often have different effects on the ability of individuals to engage with on-line student support services. While differences were noted across gender self-selection across secondary and postsecondary institutions, more underrepresented students, particularly African-American individuals, self-selected to participate in BreakThru regardless of type or level of institution. The largest number of postsecondary students self-selecting the program attended the two-year college. However, the retention rates were highest for the students attending research institutions.

Data collection within the BreakThru project has been one of the primary ongoing challenges for the project. As indicated previously, mentors and mentees communicated through multiple on-line and off-line platforms, thus collecting log-files through a single platform was infeasible. This led us to rely on surveys as the primary method for collecting data on the mentoring relationship. Unfortunately, asking students to complete long surveys at the end of the semester, near the finals period, resulted in low response rates.

We believe that streamlined, real-time, automatic data collection will be critical for both future research, and evaluating the implementation of BreakThru in other institutions. The learning modules now contain a survey at the end of each module with two questions which can be paraphrased as: (1) Was this helpful? (2) Please comment. This is a more convenient method for obtaining feedback than waiting months until the end of the semester to administer a survey. However, there is still the problem that if a student finds the module unhelpful, they may be more likely to quit in the middle, than finish the module and answer the survey question. When we examined the unique IP accesses to different modules, we found patterns that indicated students may not be completing all of the modules that they start. Also, we noted that if a student began the module on one computer and finished it on another computer, the hit count was affected. Thus page hits are a very limited measure of engagement. Nonetheless, it is useful to notice that some modules had a very high drop-off between first and last pages, while other modules have a higher percentage of visitors reaching the last page. For example, twenty-one students completed the end-of module survey for Classroom Accommodations during 2013-2014, and all of them either agreed or strongly agreed that the module was helpful; however, there are 3237 hits on the first page of the module, and 1335 hits on the last page. In contrast, Time-Management for High School Students had stronger retention between the first page (1339) and last page (1109), but only 10 students completed the end-of-module survey. Incorporating modules into a smartphone app, or an institution’s learning management system will provide e-mentoring programs more accurate matching of students to learning module activities.

Collecting data on mentor-mentee virtual meetings is even more difficult. Initially, some mentors tended to report a pair of text messages as a meeting, while others reported a 30- minute phone conversation as a meeting. This confusion was resolved by the second year of the project as we defined a meeting using a digital voice communication platform (e.g., Second Life, video chat, smart phone) for a length of time. However, training mentors in the proper definition of a “meeting” so they can answer survey questions consistently, is not a good use of time. Mentor training time would be better spent learning how to help students. A more convenient method might be to incorporate a question into the welcome page of a smartphone app: “Have you had contact with your mentor in the last week?” We are designing new applications and procedures for data collection that capture real-time responses.