Journal of Formative Design in Learning

, Volume 2, Issue 1, pp 56–67 | Cite as

Efficacy of Online Laboratory Science Courses

  • Rebecca J. Rowe
  • Lori Koban
  • Amy J. Davidoff
  • Kathryn H. Thompson
Article

Abstract

The overall goal of this study was to evaluate student experiences in online laboratory courses in order to inform the design and improvement of lab activities in a distance education program. Students were surveyed about their satisfaction and perceptions of usability and learning in both hands-on (at home) and computer-based simulation (virtual) labs in a variety of natural science courses. We also attempted to evaluate the effectiveness of several online chemistry courses taught with either hands-on kits or virtual laboratory activities, and examined the performance of students concurrently enrolled in lecture and laboratory chemistry courses versus those enrolled in a lecture only courses. The majority of survey respondents felt their online laboratory experience was the same as or better than their prior experiences in the traditional setting. Survey data also show that students believe their laboratory experiences reinforced and improved their understanding of concepts presented in lectures and the textbook, and thus may have helped them perform better on course assessments. Our data on performance suggest that students enrolled in online science courses do as well or better than their peers enrolled in traditional courses. The data also suggest that students who take lecture and laboratory concurrently outperform their lecture-only peers, independent of course (i.e., general or organic chemistry) or delivery method (i.e., online or traditional).

Keywords

Online learning Hands-on laboratory Virtual laboratory Inquiry-based Student performance Student perception 

Introduction

Decades of research and practice on teaching and learning science have provided educators with a framework based on inquiry—learning science by engaging in the scientific process (Bransford et al. 2000; National Research Council 1996, 2000, 2005). This includes laboratory experiences that provide students with the opportunity for hands-on manipulation of materials and equipment to learn the techniques (practical skills) of acquiring data, as well as the methods of experimental design, analysis/evaluation and problem solving, and the broader outcomes of reinforcing conceptual knowledge and improving scientific communication skills (National Research Council 2000; NSTA 2007). Goals for laboratory experiences have been published for the disciplines of biology, chemistry, and physics (American Association of Physics Teachers 1998; American Chemical Society 2011; Dikmenli 2007; Woodfield et al. 2005), and there is agreement among these groups that face-to-face laboratory experiences reinforce the topics and concepts introduced in the lecture and foster critical thinking skills. Bruck and Towns (2013), in their national survey of undergraduate chemistry faculty, reported that both general and organic chemistry teachers believe that laboratory activities support the lecture content. Students concurrently enrolled in lecture and laboratory sections tend to have better learning outcomes and retention when compared to their lecture-only peers (Matz et al. 2012).

With the rapid and extensive adoption of online education, research has begun to explore the nature and effectiveness of teaching and learning in the digital world. Several reports have appeared in the literature providing evidence that online education engages students (Chen et al. 2010) and provides an environment that promotes student learning (Allen and Seaman 2010; Means et al. 2010). Adoption of the online format to teach laboratory science courses has been more challenging because of the perceived need to provide an equivalent laboratory experience. However, the potential cost-savings of online learning is affecting nearly every scientific field, despite the view that traditional lab experiences are needed to provide the practical skills to conduct advanced (i.e., graduate level and professional) research (Rivera 2014; Waldrop 2013). And others argue that many students in the traditional laboratory setting come to lab unprepared, and then complete the exercise/experiment without ever really thinking about what they are learning (Josephsen and Kristensen 2006; Woodfield and Catlin 2004).

Alternative laboratory experiences have been used by some instructors as a supplement in conjunction with face-to-face laboratories (e.g., Makransky et al. 2016; Woodfield et al. 2005). The laboratory component can be either a “virtual laboratory” using computer-based simulations or a “hands-on laboratory” where the students perform actual experiments outside the classroom setting (at home). For example, online simulations have been used to familiarize students with their chemistry labs before they come to the physical laboratory to perform experiments (Dalgarno et al. 2009). Delivery of a distance education laboratory component through these modalities (virtual or at-home) has several advantages, which include essentially unlimited access and the ability to repeat the experiments (Conway-Klaassen et al. 2012). Studies of online labs used mainly as supplemental instruction have provided evidence that these may be helpful for increasing lecture test scores, improving students’ attitudes and preparedness for the hands-on lab, and strengthening conceptual knowledge (Dalgarno et al. 2009; Liu 2006; Woodfield and Catlin 2004, 2005).

Virtual laboratory experiences can range from simple videos and games to graphing and 2D simulations to interactive 3D virtual reality experiences. Simulations, as mathematical models of processes in the physical world, allow users to manipulate parameters and can be used by faculty to customize laboratories in various disciplines—e.g., virtual dissections, chemical reactions, laws of motion experiments (Welsch 2015). In addition to the advantages of cost savings, improved safety, and lower environmental impact (i.e., no hazardous waste disposal), virtual chemistry labs can help students visualize structures and processes at the molecular level (Johnstone 1982) and allow types of experiments not possible in a standard undergraduate laboratory—e.g., quantum chemistry (Woodfield and Catlin 2004, 2005). Most studies evaluating simulations have focused on conceptual understanding, and the evidence is promising for advancing this goal (Honey and Hilton 2011). Students using SimuLab in an inorganic chemistry course found it to be a motivating tool that enhanced their skills and helped them see the practical application of their knowledge (Josephsen and Kristensen 2006). A recent study modeled the impact of 3D virtual reality-based instruction in an introductory chemistry course and found this to be effective in enhancing spatial ability and self-efficacy, with positive impacts on achievement (Merchant et al. 2012).

Distance education has employed hands-on labs that can be experienced at home. These labs are favored for teaching manipulative, practical skills and exposing students to open-ended situations that foster inquiry and design skills (Ma and Nickerson 2006). Hands-on kits available from multiple vendors can provide students with standards-based experiments while providing universities advantages in terms of cost and safety, with manufacturers usually assuming liability (Welsch 2015).

Attempts to compare the effectiveness of these alternatives, especially with to face-to-face laboratory experiences, have been complicated, in part because studies have not used standardized educational objectives as the criteria for measuring success, and in part because of small sample sizes (Ma and Nickerson 2006). Nevertheless, a number of studies in various disciplines have reported results suggesting that a virtual or at-home laboratory can be comparable to the traditional laboratory in terms of learning outcomes (Casanova et al. 2006; Dobson 2009; Feig 2010). Brinson (2015) completed a comprehensive review of empirical studies comparing the effectiveness of traditional and non-traditional (online) labs, using a model he developed for categorizing learning outcomes. While learning outcomes and evaluation tools varied considerably among studies, a majority of these suggested that non-traditional labs are equally or more effective relative to traditional labs (in all outcome categories, independent of evaluation instrument), with conceptual knowledge being the most frequently used measure of effectiveness.

In the present study, we were interested in evaluating which features of our virtual and hands-on laboratory experiences are important in adding pedagogical value to online science courses. We used a formative evaluation to survey student satisfaction and perceptions regarding usability, content delivery, and learning in a university program offering a variety of online laboratory science courses (i.e., biology, general chemistry, organic chemistry, microbiology, anatomy, physiology, and physics). We also attempted to evaluate the effectiveness of online chemistry courses taught with either remote hands-on or virtual laboratory activities and examined the performance of students concurrently enrolled in a lecture and laboratory course versus those enrolled in a lecture-only course.

Methods

The overall goal of this study was to evaluate student experiences in online laboratory courses in order to inform the design and improvement of online/distance education laboratory experiences. These courses are part of a distance education program of science prerequisites for the health professions and were developed to incorporate the same science standards and best practices used in face-to-face laboratory science courses—i.e., with experiences that are engaging and grounded in scientific thinking and practice. In order to evaluate these new course designs, we surveyed students’ perceptions of the effectiveness, convenience, quality, and usability of the laboratory experiences.

Evaluating Student Perceptions

Survey Development and Structure

We developed a survey to evaluate student satisfaction and perceptions regarding content delivery and learning in online laboratory courses. The survey is presented in Appendix A. The study design and survey were approved by the Institutional Review Board. Survey questions were developed by the authors and reviewed by university curriculum personnel in order to verify the clarity of the questions and answers. The survey questions emerged from those aspects of a science laboratory experience that have been found to be important (American Association of Physics Teachers 1998; Bruck and Towns 2013; Dikmenli 2007; Woodfield et al. 2005). Generally, the questions explored the usability of the online laboratory and the extent of the connection between the lab and the lecture. The survey consisted of 20 items, including a mixture of Likert scale responses, Yes/No, and fill-in questions. Background information was also collected including some demographic factors and items exploring computer use.

Pre-survey reviews used to validate our instrument included individuals with experience in curriculum/course design and faculty with experience in laboratory sciences. For example, survey questions were reviewed by members of the university curriculum staff to verify the clarity of question and answer options and the face validity of the items. Following the initial review, faculty with experience in laboratory courses was asked to complete the survey as if they were students completing the survey. This simulation occurred without any questions or concerns arising. General descriptive statistics for the survey were generated using Survey Monkey Gold. The last survey question was an open response question asking students to “please provide additional comments or feedback about your experience in the laboratory.” The open responses were manually coded and categorized into themes by the corresponding author. Reliability of the survey responses was addressed by asking similar but unique questions and evaluating consistency among those questions. This is described in more detail in the “Results” sections pertaining to individual questions.

Survey Enrollment

In order to ensure a significant number of respondents and gather reliable data, we sent out the survey to all students who successfully completed one of the online science laboratory courses offered through the distance education program during the period of June 1, 2010, to May 31, 2013. These are undergraduate courses in the following disciplines: biology, general chemistry, organic chemistry, microbiology, anatomy, physiology, and physics. The students were taking these online courses in order to fulfill major requirements for their home institutions or as prerequisites for their desired professional programs. There were 2972 students in this cohort. Students were contacted through e-mail with an explanation of the purpose and process of the survey which included the following information: Principal Investigator’s contact information, short introduction as to the purpose of the study, end date for data collection, and the link to the survey. Students were informed that the survey responses would be anonymous, and only aggregate data would be presented. All participation was voluntary. The survey took approximately 20 min to complete. The survey remained open for 2 weeks for each student.

Survey Analyses

General descriptive statistics for the survey were generated using Survey Monkey Gold. The last survey question was an open response question asking students to “please provide additional comments or feedback about your experience in the laboratory.” The open responses were manually coded and categorized into themes by the corresponding author. We extracted a subset of the survey results from the chemistry courses (General Chemistry I and II and Organic Chemistry II), in order to control for inter-instructor and inter-discipline variability (Tables 4 and 5 in Appendix E). These data were tabulated separately for comparison with both the full aggregated data set, as well as the performance data (see “Results” section).

Evaluating Student Performance

In order to assess the relationships among student perceptions and student performance, further analyses were carried out with students enrolled in general chemistry and organic chemistry courses. This enabled us to enhance the reliability of our data by controlling for inter-discipline and inter-instructor course variability, since these courses were all designed and delivered by the same faculty member (corresponding author). This restricted sample included 160 students who successfully completed an online course, either General Chemistry I (70 students) or Organic Chemistry II (90 students), during the past 3 years, and students who completed Organic Chemistry II taught in a traditional manner at another university (107 students). From this convenience sample, we obtained data for the following variables: Final Laboratory Grade, Final Lecture Grade, Laboratory Type, and Course Format (i.e., online or traditional).

Hands-On and Virtual Laboratories

The study was designed, in part, to explore the efficacy of laboratory experiences in online chemistry courses using a hands-on (at-home) laboratory format or a virtual laboratory format. Hands-on laboratory activities were part of the online general chemistry course, in which students conducted the experiments, unsupervised, using the chemicals and equipment in custom-built at-home laboratory kits. The students used their observations to write laboratory reports and answer assessment questions, similar to those activities associated with traditional laboratory experiments, but unsupervised directly by a content expert. Virtual laboratory activities were part of the online organic chemistry course in which students completed the experiments on their computers (again with no direct expert supervision). The students built reaction apparatus, ran chemical reactions, and analyzed reaction products using spectroscopy, all through an interactive computer-based simulation program (Woodfield and Catlin 2004) which we refer to as “virtual” laboratory experiments. The students collected and analyzed data to write laboratory reports and answer assessment questions.

Performance Analyses

Student performance in lecture and laboratory was analyzed using linear regression analysis and non-parametric testing. In addition, student performance in lecture was compared across all three chemistry course formats, with or without laboratory, using a two-way ANOVA with Tukey follow-up tests for multiple comparisons.

Results

Demographics of Survey Respondents

Surveys were sent to students who completed the online undergraduate science laboratory courses between the years 2010 and 2013. Of the total 2972 surveys emailed to the students, 90 student emails were undeliverable. Of the remaining 2882 students presumed to have received the survey, 386 responded, yielding a 13.4% response rate (see Appendix B for full results of demographic and computer use questions). Only 22.2% of respondents were taking these courses to complete an undergraduate or post-baccalaureate degree, whereas over half (54.5%) were in a graduate or professional program. The remaining 23.3% were taking the courses to satisfy prerequisite requirements for graduate or professional programs or for self-enrichment. Over 70% were non-traditional students (defined by the National Center for Education Statistics (NCES; https://nces.ed.gov/pubs/web/97578e.asp) as those aged 25–39) and females outnumbered males two to one.

The first two survey questions asked students to select the last online course they completed through their distance education program and to select the corresponding laboratory component (i.e., hands-on or virtual laboratory activities). Students were instructed to answer the remaining survey questions based on the course they selected. In the last question of the survey, which was an open response question, students were given the opportunity to provide additional comments or feedback about their laboratory experience. Of the 386 students who responded to this survey, 189 (49%) provided feedback or comments. These open responses were manually coded and categorized into themes by the corresponding author. The themes that emerged were effectiveness, quality, and convenience.

Of the courses involved in the survey, five courses utilized hands-on lab kits (at-home labs) and five used computer-based virtual labs. For the subset of data extracted and used to control for inter-discipline and inter-instructor variability, students were taking either online general chemistry, in which they engaged in hands-on, unsupervised, at-home laboratory activities, or online organic chemistry, in which they conducted virtual experiments on their computers, again with no direct expert supervision.

Description of Laboratory Experiences for Online Courses

Hands-on General Chemistry Laboratory

The students in online general chemistry used custom-built “LabPaqs,” purchased from Hands on Labs (HOL; http://holscience.com/). The experiments selected for the LabPaqs provide the student with a “comprehensive hands-on laboratory experience” that is academically aligned with the laboratory experience obtained by students at traditional colleges and universities according to the HOL website. These LabPaqs are designed to reinforce the concepts and topics described in the lecture component that are usually difficult for the students to grasp. For example, students are able to determine the stoichiometry of a precipitation reaction using techniques normally used in the traditional lab setting, which include the scientific method (including recording observations), correct use of laboratory glassware and equipment including digital balance, gravity filtration, laboratory calculations, and waste disposal. This one experiment allows the student to demonstrate competency in several key concepts, such as stoichiometry, limiting reactant, theoretical, actual, and percent yields that are used throughout the course.

Owing to the nature of these at-home labs, the students are engaged in the process and can reflect on what they are doing and why. There is a greater risk with the hands-on labs that the experiment will not go as planned owing to experimenter error. The LabPaqs are designed to contain enough chemicals and supplies to allow the student to repeat the experiment. When a student’s experiment does not go as planned, this can be turned into a positive learning experience by which the student can gain valuable experience and confidence which will be helpful down the road.

Virtual Organic Chemistry Laboratory

The students in Organic Chemistry II used simulated/virtual lab software that is installed on their computers. The experiments are “realistic and sophisticated” (Woodfield et al. 2005) and place the students in a 2D virtual laboratory where they are able to make decisions that will determine the outcome of the experiments (Woodfield and Catlin 2004). This important attribute of the virtual laboratory software places the responsibility of the decision-making process on the student, which is comparable to the decision-making process of students in the traditional laboratory setting.

One important aspect of the simulated organic chemistry experiments is that they are designed to work as long as the student follows the experimental procedure. Therefore, it is important to keep the student engaged by giving them the opportunity to reflect on why they are doing the experiment, what they should expect the outcome to be, and how they can ensure the expected outcome is achieved. Keeping the students engaged in the process and allowing them time to reflect are instrumental in the development of their independent critical thinking skills. To keep the students engaged, the procedure is written in such a way to allow students to record their observations as they are happening. This is accomplished by asking the student to run thin-layer chromatography (TLC) and describe how the reaction is progressing from reactant to product. Another aspect of student engagement occurs during the work-up of the experiment. During the liquid-liquid extraction, the student needs to record which layer contains the desired product. This is important to know since saving the wrong layer would result in isolating a product that is not the desired product. Knowing where the desired product is at all times allows the student to show they understand how the structure of the product directly affects its polarity and thus increases the solubility in one layer (solvent) over the other.

What happens when an experiment does not go as planned in either the simulated organic chemistry laboratory or the traditional organic chemistry laboratory? This is where the reflection comes in. In the simulated laboratory, students have the luxury of reflecting on what went right and what did not go as planned. They are able to go over their experiment and determine the step where the reaction went off the projected path. They are then able to re-do the experiment and correct their mistakes on the spot. This reduces the student’s frustrations and creates a positive learning experience. In the traditional laboratory, students have very few options when their experiments do not go as planned. Traditional organic laboratories have significant time constraints. Students have a set amount of time to start the experiment, work-up and analyze the data, and clean up. They do not have the luxury of starting over if their experiment did not go as planned. This can lead to frustration and a poor lab grade, especially since the student does not have time to figure out what went wrong.

Student Perceptions

The following results for student perceptions of their online course experiences are based on the survey data from all of our program’s science courses with laboratory that includes General Biology I and II, General and Organic Chemistry I and II, Physics I, Microbiology, Anatomy, and Physiology. Table 1 summarizes responses collated from Likert-formatted questions, and Table 2 illustrates responses from a fill-in question format.
Table 1

Student perceptions of usability and learning

Survey questions

Numbera

Strongly agree or agree

Neither agree nor disagree

Strongly disagree or disagree

Usability

 • The laboratory software was easy to install/download.

368

88.0%

4.1%

7.8%

 • The laboratory software was easy to use.

365

88.5%

4.4%

7.1%

 • The laboratory instructions were easy to understand.

317

88.3%

5.4%

6.3%

 • Time required to complete online lab assignments comparable to time spent in traditional laboratory.

381

48.3%

12.1%

39.6%

Learning enhancement

 • Lab experiments reinforced topics from lecture and text.

385

83.4%

9.4%

7.3%

 • Laboratory experiments helped me with the understanding of lecture concepts.

385

72.7%

15.6%

11.7%

 • Lecture material helped me understand the purpose of the lab experiment

383

81.7%

9.9%

8.4%

 • Completing the lab assignments helped me to do better on lecture quizzes and exams.

385

61.3%

21.0%

17.7%

aThe sample size for each question is 386 (the total number of students who took the survey) minus the number of students who either did not answer the question or who said the question did not apply

Table 2

Perceptions of students with both online and traditional lab experiences

Survey question

Numbera

“Much better than” or “better than”

“About the same as”

“Much worse than” or “not as good as”

My online lab experience was ___________my experience in a traditional classroom lab

384

29.7%

36.7%

33.6%

aThe sample size for each question is 386 (the total number of students who took the survey) minus the number of students who either did not answer the question or who said the question did not apply

Student Perceptions of the Usability of Online Laboratories

We constructed several survey questions (see Appendix A for entire survey questionnaire) to assess student perceptions of the ease of use (i.e., usability) of the online course laboratory activities. Based on our survey results (Table 1), more than 88% of the respondents who conducted course experiments using either a virtual laboratory or a hands-on lab kit felt that the virtual labs were easy to install and use, and the hands-on lab instructions were easy to understand.

One survey question provided students the opportunity to report any challenges they experienced accessing the virtual laboratory experiments or online laboratory manual for the hands-on experiments. Of the 379 responses to this question, 40 students (10.6%) reported they experienced one of the following challenges: unable to install the virtual laboratory software or to download the hands-on lab manual, unable to open the laboratory software or laboratory manual once it was installed, or computer froze or crashed when opening the software or laboratory manual. Seventy-six percent of the students reported that they did not experience any technical difficulties. The remaining 13.4% reported experiencing different technical challenges than those stated above. These responses are summarized in Appendix C. Since these challenges varied with no appreciable trend for any one challenge, they were not explored further.

Of the 189 responses to the open response question, the majority of these students were enrolled in a laboratory course that utilized online (virtual) experiments, rather than at home, hands-on experiments (see Appendix D for results of open response question). Nineteen responses were about the usability of the virtual laboratory software: nine students mentioned that it was easy to use, two mentioned that it was not easy to use, three mentioned that there was a small learning curve, and five mentioned that the software worked well in general, but suffered a bit from poor image quality or mouse sensitivity. Regarding whether or not the hands-on laboratory instructions were easy to understand, ten of the students’ responses to the additional-comment question were relevant: four said they were easy to understand, four said they were easy to understand but would like more instructions, and two said they were not easy to understand.

Overall, students found the laboratory instructions readily accessible and experienced few technical problems, independent of whether their course utilized hands-on or virtual laboratory activities.

Perceptions of Online vs Traditional Laboratory Science Courses

Two survey questions were designed to assess the students’ experiences with the laboratory associated with their online course as compared to their experiences in traditional laboratory courses. Most of the students (97.4%) responded that they had taken a college or university science course with a laboratory in a traditional classroom setting so they could make this comparison.

Perceptions about the time required to complete an experiment varied widely. Approximately half of the students felt that the time required to complete a laboratory experiment for their online course was comparable to the time required to complete an experiment in a traditional laboratory setting (Table 1). While nearly 40% did not agree with this, the reason for disagreement was not surveyed. Nineteen students commented that the online (virtual) laboratories took less time than traditional laboratories. Reasons provided by the students for the time-savings were twofold: first, timers could be accelerated so that reactions could be sped up; second, time required for setting up and cleaning up experiments was reduced.

When students compared their laboratory experiences in their online courses to their traditional laboratory courses, two thirds reported that the online laboratory experience was as good or better than their traditional laboratory experience (Table 2). Eighty-three students who responded to the open response question specifically commented on their virtual and traditional laboratory experiences. Of these, the majority (73%) said that their virtual laboratory experience was as good or better than their traditional laboratory experience. Recurring reasons for this included the ability to repeat experiments, the ease of changing parameters, the stress-free environment, the ability to perform the experiment without wasting materials, the ability to focus exclusively on content instead of logistical details, and generally feeling the laboratory was more interactive. Several comments in favor of traditional laboratories cited the advantage of having peers present and a laboratory assistant available for interactions, and a preference for hands-on learning.

Perceptions of Synergism Between Lecture and Laboratory Concepts

As shown in Table 1, the majority of students (83.4%) indicated that the online laboratory experiments reinforced the topics discussed during the online lecture or in the textbook. These survey questions were reinforced by other answers to the “Learning enhancement” questions in Table 1, where a majority (72.7%) also felt that completing the laboratory experiments helped them better understand the lecture material. Furthermore, most students (81.7%) believed that the lecture material helped them understand the purpose of the laboratory experiments, and many (61.3%) perceived that completing the laboratory assignments helped them to do better on lecture quizzes and exams. These findings are further supported by responses to the open response question, where 12 students specifically commented on how well the online laboratories reinforced the lecture concepts, and 3 students mentioned that the lecture reinforced the concepts of and/or prepared them for the online laboratory.

In order to control for inter-instructor variability and discipline-based variability, we extracted the survey results from students who were enrolled in chemistry courses. These courses were all developed by and taught by the lead author of this paper, and performance data are also presented in Figs. 1, 2, and 3. Analyzing the subset of results from the survey data of the chemistry courses provided some interesting outcomes. The data querying students about whether they felt labs enhanced their learning of the lecture and textbook materials by reinforcing concepts and making connections varied between general chemistry courses (with hands-on laboratory exercises) and the organic chemistry course (with the virtual laboratory component). A large majority of students in the virtual organic chemistry laboratory (76–88%) reported that the labs improved their understanding of the lecture material and vice versa, including that their performance on quizzes and exam were enhanced (Table 5 in Appendix E). These findings are similar in students who completed either general chemistry course (both of which utilized hands-on laboratory experiments), with the notable exception that only 52% of the general chemistry students felt that the labs improved their assessment scores (Table 4 in Appendix E). When performance data was analyzed between students who took these courses either with or without a laboratory component, we found that the final course grade for organic chemistry students was similar in both groups, whereas having a lab improved student performance in general chemistry course (whether they completed the lab at home or in a traditional manner in a classroom with other students and the professor (Fig. 3).
Fig. 1

Regression analyses of lecture and laboratory performance in the online and traditionally taught chemistry courses. a Online general chemistry with “hands-on” laboratory experiments (n = 70 students). b Online organic chemistry with “virtual” laboratory experiments (n = 90 students). c Traditionally taught organic chemistry with “traditional” laboratory experiments (n = 107 students). Data represent individual student performances in lab (final lab grade) and final lecture exam. Inserts depict slope and y-intercept of the regression line and R 2 values for each data set

Fig. 2

Quadrant analyses. a Comparing laboratory grades and lecture grades based on upper and lower limits using the medians to draw the demarcation lines for students in an online general chemistry course with hands-on laboratory experiments (n = 70 students). b Comparing laboratory grades and lecture grades based on upper and lower limits using the medians to draw the demarcation lines for students in an organic chemistry course with virtual laboratory experiments (n = 90 students). See text for quadrant definition details

Fig. 3

Final lecture grades were compared between two online courses (general chemistry and organic chemistry) and a course taught in a traditional format (organic chemistry). Student grades were also divided based on concurrent enrollment in lecture and laboratory versus lecture only. Data represent mean ± SE, n = 70–107 students/course. Asterisk indicates significant difference among final lecture grades of students taking the lecture only. Bars indicate significant differences between groups. Data were analyzed with a two-way ANOVA with Tukey multiple comparisons follow-up tests (significance was set at α = 0.05)

Survey Validation and Reliability

As stated in the methods, prior to administering our survey, we vetted it through professionals with knowledge of online curriculum and laboratory sciences, as a means of validating our survey with some fidelity. It is recognized that there are more rigorous methods to validate surveys, but since our questions were straightforward and we feel confident that they are reasonable for an initial screening of student perceptions regarding their experiences with online courses. The reliability of our survey results was assessed by the consistency among the responses to questions in the “Usability” and “Learning enhancement” questions (Table 1) and “Open response” question (Appendix D). By assessing the consistency among three Likert scale questions (each indicating that 88% of the students report ease of software installation/use and instruction) and while the majority responses in the Open response question indicated that they did not experience any technical difficulties (76%). Furthermore, there is good corroboration among the three Likert scale questions in the Learning enhancement category that interrogate whether the lab and lecture components reinforce one another (73–83% agreed strongly or agreed), and these were confirmed in the open responses. Collectively, we have confidence that our survey captured the fundamentals of student perception of usability of software and learning enhancement of laboratory activities.

Student Performance

Results describing student performance are based on the convenience sample data. This sample reflects a subset of students who completed a chemistry course, either online general chemistry I, online organic chemistry II, or organic chemistry II taught in a traditional classroom setting. This subset was chosen so that we could evaluate performances while reducing variability due to differences among science disciplines and instructors. This subset of students was further subdivided in terms of the format of the laboratory experiments, such that student taking the online organic chemistry course used a virtual laboratory program while those taking the online general chemistry course had hands-on (at-home) laboratory experiments. The organic chemistry course taught in a face-to-face campus setting utilized traditional hands-on laboratory experiments. Another advantage for using this particular subset of students is that all of the courses were developed and taught by the same instructor, so that instructor variability could be minimized.

Student Performance in Online Chemistry Courses; Lecture and Laboratory Components

Data from 70 students in an online general chemistry course, who used a hands-on laboratory at home, were analyzed to examine the relationship between Final Lecture Grade and Final Laboratory Grade (Fig. 1a). A similar analysis was carried out with data from 90 students in an online organic chemistry course, in which students used software for a virtual laboratory (Fig. 1b). These analyses were then compared to regression results based on data from 107 students in a traditional (face-to-face) organic chemistry course (Fig. 1c).

Regression analyses of these data indicate a strong positive correlation between student performance in lecture and laboratory in the traditional organic chemistry course and a modest positive correlation between these variables in the online organic chemistry course (see R 2 values in Fig. 1c, b, respectively). On the other hand, lecture grades were not well correlated with laboratory grades in the online general chemistry course (see R 2 value in Fig. 1a). Therefore, we further analyzed the data from the online courses by separating the student performances by quadrants. In Fig. 2, each variable was broken into an upper and lower half. In both scatterplots, the vertical line was placed at the median Final Laboratory Grade, and the horizontal line was placed at the median Final Lecture Grade. Figure 2a illustrates data from the students who conducted hands-on laboratory experiments, and Fig. 2b scatterplot illustrates data from a course with virtual laboratory experiments. Both scatterplots contain representation in all four quadrants. The four quadrants are marked LL, LU, UL, and UU. The first letter in each pair indicates whether the data points are in the lower or upper half of the Final Laboratory Grade, and the second letter pertains to the Final Lecture Grade. For example, a data point in the UL quadrant is above the median Final Laboratory Grade and below the median Final Lecture Grade.

For each laboratory type, the quadrant distribution is shown in Table 3. Nonparametric testing indicates that the percent distribution for the quadrants for hands-on laboratories is not statistically different than the percent distribution for the quadrants describing virtual laboratories, \( {\chi}_3^2=2.937,p=0.4 \) For example, there was no statistical difference between highest performing students in both the lecture and lab (UU) when comparing students using a hands-on lab (30%) to those using a virtual lab (40%).
Table 3

Quadrant distribution (see text for definitions of quadrant labels)

 

LL

LU

UL

UU

Hands-on

44%

11%

14%

30%

Virtual

43%

9%

8%

40%

Student Performances in Online and Traditional Environments: Impact of Laboratory

To assess the influence of laboratory activities on final lecture grades, and compare those results for both online and traditional courses, data from three subsets of student outcomes were analyzed (online general chemistry, online organic chemistry, and traditional organic chemistry).

Our data indicate that students performed better in courses with hands-on or traditional laboratory experiments compared to lecture only (Fig. 3). A similar trend was observed with the virtual laboratory, but the data were not significant (p = 0.09). It should be noted that the overall grades for the online chemistry course with a virtual laboratory were higher than any of the other courses. In addition, even in a lecture-only course, students enrolled in an online organic chemistry course outperformed students in the other courses (gray bars).

Discussion

Based on a survey of student perceptions, this study provides support for the notion that natural science courses can be effectively taught using an online/distance education platform. Laboratory experiences provide additional value to these online courses and can be delivered digitally (virtual labs) or using hands-on lab kits for “at-home” convenience. Our summative student data on performance suggest that students enrolled in online science courses do as well or better than their peers enrolled in traditional courses.

Student Perceptions of Online Laboratory Experiences

The findings presented here provide useful information on how to design and deliver a positive online laboratory experience. Laboratory software must be easy to install, user-friendly, and intuitive, yet challenging and able to provide experiences similar to the traditional laboratory. Likewise, digital laboratory manuals that accompany hands-on lab kits must also be user-friendly and intuitive. It is vital to bear in mind that online students are completing their laboratory assignments (either virtual or hands-on) without direct supervision from content experts.

The majority of survey respondents felt their online laboratory experience was the same as or better than their prior experiences in the traditional setting. One reason that emerged from students’ comments was the convenience of accessing the laboratories whenever and wherever they were needed. In many cases, this is what made it possible for students to complete their required or prerequisite laboratory science courses. Student comments were particularly positive about the virtual laboratory for several reasons, all associated with the ease of setup and use. These virtual labs are time saving compared to traditional labs that require physical setup and cleanup, and can be conducted multiple times while changing the parameters in an efficient manner. This allows students to spend more time understanding the objectives of the laboratory experiences. Laboratory experiments that involve timed processes, such as reactions, can be accelerated in the virtual format. Recurring comments also indicate that students find virtual laboratory experiments to be more interactive and engaging.

Overall, the students’ responses regarding the quality of the online laboratory experience were positive. These included having a stress-free environment, the ability to perform the experiments without wasting materials, the ability to focus exclusively on content instead of logistical details, and generally feeling the laboratory was more aligned with the lecture material. Some students mentioned having a preference for hands-on learning, and this can be accomplished in an online course by using at home lab kits. Other comments, however, did cite the advantage of having direct interactions with both peers and the professor/lab instructor in the traditional science laboratory setting.

In terms of effectiveness, our data show that students believe that their laboratory experiences helped them understand the concepts presented in lectures and that the lecture material helped them grasp the purpose of the laboratory experiments. Furthermore, students stated that completing the laboratory experiments reinforced specific topics covered in the lectures and textbook readings, and thus may have helped them perform better on course assessments (quizzes and exams). Instructors in online science courses, like their counterparts using face-to-face instruction, use the laboratory experience both to illustrate particular techniques and methods, and to provide students with an experiential way to learn specific course concepts. This reinforcement provides an opportunity for students to develop a deeper understanding and better retention of course material.

The open-ended response question also gave the students the opportunity to inform us of several areas where we could improve our online courses in order to enhance the student’s online experience. These include real-time technical support and the lack of more tutorials that help to explain difficult topics. Real-time technical support is a hard issue to address. This is owing to the fact that our students are able to take online courses from anywhere in the world. With all of the different time zones, there is a possibility that all technical support offices are closed and not accessible to all of our students when needed. Solutions to the most common problems, such as forgotten passwords, or other common issues can be found in the Frequency Asked Questions (FAQ) on the program’s home page. To address student needs for more tutorials on difficult subject matter, we can turn to the textbook publishers. With the advances of electronic textbooks (e-books), these tutorial or videos are embedded directly in the e-book. In addition to these, the publishers often provide other online resources that can be included into the learning management system (LMS). Adoption of books that are content rich and have adaptive learning would address these needs and enhance the online experience. The biggest concern expressed by online students is the lack of peer-to-peer and student-instructor interactions. Ways that we are planning to address this concern include the use of chat rooms or study lounges, virtual office hours, and discussion boards.

Overall Student Performance in Online vs Traditional Laboratory Courses

The data presented in Fig. 1 indicate a positive correlation between lecture and laboratory grades for students enrolled in either an online or traditional organic chemistry course, compared to students enrolled in an online general chemistry course (R 2 values = 0.508, 0.729, and 0.106, respectively). We speculate this finding most likely reflects a process of student learning and maturation (as opposed to being subject matter or format specific). Students usually take a year of general chemistry with corresponding laboratory before they take organic chemistry. This allows students in general chemistry an opportunity to figure out how to make use of the laboratory experience to help them understand the topics discussed in the lecture, and vice versa. Therefore, students taking organic chemistry have already developed the skills needed to appreciate how laboratory experiments are designed to reinforce course concepts discussed in lecture, whereas students in general chemistry are just learning how to make this connection.

A closer inspection of the R 2 values for Fig. 1b, c shows the correlation between lecture and laboratory performance is stronger for students in the traditional course than those taking organic chemistry online, with a virtual laboratory (same instructor for both courses). This is perhaps explained by the fact that the instructor in the traditional organic chemistry course has the ability to directly supervise students in the laboratory. This allows for immediate feedback on the progress of the experiment and an opportunity to address concerns or questions that may arise during the lab period. Students taking the laboratory online do not have the ability to obtain immediate feedback from their instructor or their peers. Instructors for online laboratories must rely on their students to concisely explain (after the fact) any issues they had during the laboratory. Several students’ comments in the open response survey question stated they preferred the traditional laboratory over the virtual laboratory simply because of the availability of the laboratory instructor and peers.

Student Performance: Concurrent Enrollment in Lecture and Laboratory vs Lecture Only

The data presented in Fig. 3 show that students who take lecture and laboratory concurrently outperform their lecture-only peers, independent of course (i.e., general or organic chemistry) or delivery method (i.e., online or traditional). These data are in agreement with the observations of Matz et al. (2012) who compared students taking general chemistry with or without a concurrent laboratory. When the material in the lecture and laboratory reinforce each other, students gain a deeper understanding, retain the material longer, and perform better on the assessments.

Interestingly, we found that the students enrolled in the online organic chemistry course performed significantly better than those in either the traditional organic chemistry course or the online general chemistry course (Fig. 3). Most of the students taking the online science courses are nontraditional students who have earned a BA/BS degree, and those taking organic chemistry all must have already taken at least one prior science course (i.e., general chemistry). Therefore, these results are perhaps not surprising. It is likely that these students are more mature, more motivated, and have already demonstrated success in science. The traditional organic chemistry students have also had general chemistry (and maybe other science courses), but they are typically younger and have not yet earned a degree. The online general chemistry students may or may not have had any science courses before taking that course (even though they are likely to have earned a bachelor’s degree).

Study Limitations

The authors did not have access to the grades for all the survey participants; therefore, we restricted the analyses of the online natural science courses to student perceptions of usability and effectiveness of their laboratory experiences. The survey question asking students to compare their online experience relative to their traditional lab experience was too general—i.e., it did not include specific aspects of the lab (e.g., quality, effectiveness, ease of use, etc.). Assessment of student performance was limited to chemistry courses in which lecture and laboratory grades were available (taught by the corresponding author). Additionally, it would be an improvement in the study design if a comparison between virtual, hands-on and traditional laboratory experiments could be conducted for the same online lecture course.

Conclusions

This study supports the notion that effective, efficient teaching and learning instruction is possible outside the traditional laboratory setting. For students taking online science courses with laboratory, it was important to determine whether students’ experiences and perceptions of the laboratory component of the course were similar to those in the traditional classroom/laboratory setting. Most of the students surveyed are nontraditional students who have limited educational options because of geography, employment, and/or family obligations. Online science courses with laboratory provide a flexible option for completing science prerequisite courses for programs in the health professions.

This study also provides evidence that online laboratory science courses can meet the educational needs of these students. Our data show that the majority of students conducting laboratory experiments through online science courses felt that the online laboratories were easy to use and comparable to the experiments in a traditional laboratory setting, but also, they saved valuable time. Students perceived that the laboratories helped them to do well in the lecture portion of the course, and that the lecture, in turn, helped them understand the purpose of the laboratories. These perceptions are supported by summative laboratory and lecture assessment scores. Finally, content analysis of the students’ open-ended comments revealed an overall satisfaction with the effectiveness, quality, and convenience of the laboratory experience.

Notes

Acknowledgements

The authors would like to thank Mary L. Schwanke, Ph.D. for her expert editing and advice during the writing of this manuscript.

Compliance with Ethical Standards

Conflict of Interest

The authors declare that they have no conflict of interest.

Supplementary material

41686_2017_14_MOESM1_ESM.docx (117 kb)
ESM 1 Included in the Supplemental Material file are Appendices A through E. Appendix A contains the Survey Questions, Appendix B contains Results for Demographics and Computer Use Questions, Appendix C contains Other Challenges Experienced Accessing the Laboratory Component and Appendix D contains Additional Students Comments to the open response question Appendix E contains Table 4 General Chemistry Students Perceptions of Usability and Learning and Table 5 Organic Chemistry Students Perceptions of Usability and Learning. (DOCX 117 kb)

References

  1. Allen, I. E., Seaman, J. (2010). Class difference$: online education in the United States. Sloan Consortium. https://www.onlinelearningsurvey.com/reports/class-differences.pdf. Accessed June 2006.
  2. American Association of Physics Teachers. (1998). The goal of introductory laboratories. College Park. Retrieved from http://www.aapt.org/Resources/policy/goaloflabs.cfm. Accessed April 2016.
  3. American Chemical Society. (2011). Importance of hands-on laboratory activities. Retrieved from https://www.acs.org/content/acs/en/policy/publicpolicies/education/computersimulations.html. Accessed September 2017.
  4. Bransford, J. D., Brown, A. L., & Cocking, R. R. (2000). How people learn: Brain, mind, experience, and school (expanded ed). Washington, D.C.: National Academy Press.Google Scholar
  5. Brinson, J. R. (2015). Learning outcome achievement in non-traditional (virtual and remote) versus traditional (hands-on) laboratories: A review of the empirical research. Computers and Education, 87, 218–237.  https://doi.org/10.1016/j.compedu.2015.07.003.
  6. Bruck, A. D., & Towns, M. (2013). Development, implementation, and analysis of a national survey of faculty goals for undergraduate chemistry laboratory. Journal of Chemical Education, 90(6), 685–693.  https://doi.org/10.1021/ed300371n.CrossRefGoogle Scholar
  7. Casanova, R. S., Civelli, J. L., Kimbrough, D. R., Heath, B. P., & Reeves, J. H. (2006). Distance learning: A viable alternative to the conventional lecture-lab format in general chemistry. Journal of Chemical Education, 83(3), 501.CrossRefGoogle Scholar
  8. Chen, P.-S. D., Lambert, A. D., & Gudiry, K. R. (2010). Engaging online learners: The impact of web-based learning technology on college student engagement. Computers & Education, 54, 1222–1232.CrossRefGoogle Scholar
  9. Conway-Klaassen, J. M., Wiesner, S. M., Desens, C., Trcka, P., & Swinehart, C. (2012). Using online instruction and virtual laboratories to teach hemostasis in a medical laboratory science program. Clinical Laboratory Science, 25(4), 224–229.Google Scholar
  10. Dalgarno, B., Bishop, A. G., Adlong, W., & Bedgood Jr., D. R. (2009). Effectiveness of a virtual laboratory as a preparatory resource for distance education chemistry students. Computers & Education, 53(3), 853–865.CrossRefGoogle Scholar
  11. Dikmenli, M. (2007). Biology student teachers’ ideas about purpose of laboratory work. Asia-Pacific Forumon Science Learning and Teaching, 10(2), 1–14 Article 9. https://www.ied.edu.hk/apfslt/v10_issue2/dikmenli/dikmenli7.htm.Google Scholar
  12. Dobson, J. (2009). Evaluation of the virtual physiology of exercise laboratory program. Advances in Physiology Education, 33, 335–342.CrossRefGoogle Scholar
  13. Feig, A. (2010). An online introductory physical geology laboratory: From concept to outcome. Geosphere, 6(6), 1–10.CrossRefGoogle Scholar
  14. Hands-On Lab Website. (n.d.). http://holscience.com/.
  15. Honey, M. A., & Hilton, M. L. (2011). Learning science through computer games and simulations. Washington, D.C.: National Academic Press.Google Scholar
  16. Johnstone, A. H. (1982). Macro-and micro-chemistry. School Science Review, 64(227), 377–379.Google Scholar
  17. Josephsen, J., & Kristensen, A. K. (2006). Simulation of laboratory assignments to support students’ learning of introductory inorganic chemistry. Chemical Education Research and Practice, 7(4), 266–279.CrossRefGoogle Scholar
  18. Liu, X. (2006). Effects of combined hands-on laboratory and computer modeling on student learning of gas laws: A quasi-experimental study. Journal of Science Education and Technology, 15(1), 89–100.CrossRefGoogle Scholar
  19. Ma, J., & Nickerson, J. V. (2006). Hands-on, simulated, and remote laboratories: A comparative literature review. ACM Computing Surveys (CSUR), 38(3), 7.CrossRefGoogle Scholar
  20. Makransky, G., Thisgaard, M. W., & Gadegaard, H. (2016). Virtual simulations as preparation for lab exercises: Assessing learning of key laboratory skills in microbiology and improvement of essential non-cognitive skills. PLoS One, 1-11, e0155895.  https://doi.org/10.1371/journal.prone.0155895.CrossRefGoogle Scholar
  21. Matz, R. L., Rothman, E. D., Krajcik, J. S., & Banaszak, M. M. (2012). Concurrent enrollment in lecture and laboratory enhances student performance and retention. Journal of Research in Science Teaching, 49(5), 659–682.CrossRefGoogle Scholar
  22. Means, B., Toyama, Y., Murphy, R., Bakia, M., and Jones, K. (2010). Evaluation of evidence-based practices in online learning: a meta-analysis and review of online learning studies. Washington DC: US Department of Education, Office of Planning, Evaluation, and Policy Development. https://www2.ed.gov/rschstat/eval/tech/evidence-based-practices/finalreport.pdf. Accessed September 2017.
  23. Merchant, Z., Goetz, E. T., Keeney-Kennicutt, W., Kwok, O., Cifuentes, L., & Davis, T. J. (2012). The learner characteristics, features of desktop 3D virtual reality environments, and college chemistry instruction: A structural equation modeling analysis. Computers & Education, 59, 551–568.CrossRefGoogle Scholar
  24. National Center for Education Statistics (NCES). (n.d.). https://nces.ed.gov/.
  25. National Research Council. (1996). National Science Education Standards. Washington, D.C.: The National Academies Press.Google Scholar
  26. National Research Council. (2000). Inquiry and the National Science Education Standards: A guide for teaching and learning. Washington, D.C.: The National Academies Press.Google Scholar
  27. National Research Council. (2005). How students learn: History, mathematics, and science in the classroom. Washington, D.C.: The National Academies Press.Google Scholar
  28. National Science Teachers Association. (2007). NSTA position statement: the integral role of laboratory investigations in science instruction. http://www.nsta.org/about/positions/laboratory.aspx. Accessed September 2017.
  29. Rivera, C. (2014). For some students, virtual labs replace hands-on science experiments. Los Angeles Times. Retrieved from http://www.latimes.com/local/education/la-me-college-labs-20141115-story.html. Accessed September 2017.
  30. The National Center for Education Statistics (NCES). (n.d.). Retrieved from https://nces.ed.gov/pubs/web/97578e.asp.
  31. Waldrop, M. M. (2013). Education online: The virtual lab. Nature, 499, 268–270 http://www.nature.com/news/education-online-the-virtual-lab-1.13383.CrossRefGoogle Scholar
  32. Welsch, D. (2015). The effectiveness of online labs. Online Learning Tips, American Public University System. https://onlinelearningtips.com/2015/06/the-effectiveness-of-online-science-labs/. Accessed September 2017.
  33. Woodfield, B. F., & Catlin, H. R. (2004). The virtual ChemLab project: A realistic and sophisticated simulation of inorganic qualitative analysis. Journal of Chemical Education, 81(11), 1672–1678.CrossRefGoogle Scholar
  34. Woodfield, B. F., Andrus, M. B., Anderson, T., Miller, J., Simmons, B., & Stanger, R. (2005). The virtual ChemLab project: A realistic and sophisticated simulation of organic synthesis and organic qualitative analysis. Journal of Chemical Education, 82(11), 1728–1735.CrossRefGoogle Scholar

Copyright information

© Association for Educational Communications & Technology 2017

Authors and Affiliations

  • Rebecca J. Rowe
    • 1
  • Lori Koban
    • 2
  • Amy J. Davidoff
    • 1
  • Kathryn H. Thompson
    • 1
  1. 1.University of New EnglandBiddefordUSA
  2. 2.University of Maine FarmingtonFarmingtonUSA

Personalised recommendations