Introduction

Over the last quarter century, rapid advances in medical technology have provided modern patients with an unprecedented range of options for diagnosis and treatment. At the same time, information technology has enabled instant public access to a wide variety of highly specialized medical information. These changes have altered the way in which patients participate in their own health care, presenting both new opportunities and new demands. Many patients and their families suddenly find themselves responsible for choosing among multiple treatment options, some more aggressive or experimental than others, based on medical advice and technical information that is often highly complex and sometimes contradictory.14 It is increasingly necessary for patients to have a high degree of scientific literacy to successfully navigate the array of health care options available to them.

It is not clear that today’s patients are prepared to meet these new challenges. In fact, there is ample evidence that the scientific literacy of the general public remains poor.29 In a 2001 survey, nearly two-thirds of Americans could not properly describe the scientific method as theory testing, experimentation or rigorous comparison.22 A 2005 telephone survey of 957 adults indicated that a large fraction of Americans have misconceptions regarding cancer;10 41% of respondents felt that “treating cancer with surgery can cause it to spread throughout the body,” and 27% responded that “the medical industry is withholding a cure for cancer from the public in order to increase profits.” In a 1995 study, Miller and colleagues presented 2006 adults with two hypothetical studies to test a new drug to treat high blood pressure.20 Participants were asked whether it was better to “give the drug to 1000 people with high blood pressure and see how many experience lower blood pressure levels” or to “give the drug to 500 people with high blood pressure and not give the drug to another 500 people with high blood pressure and see how many people in both groups experience lower blood pressure levels.” A follow up question probed why each respondent though it was better to test the drug in this way. In this study 69% of respondents selected the two-group design; however, upon probing, it became clear that most did so for incorrect reasons—because they feared that the drug might kill people and fewer people would be harmed if the drug were given to only half the study population. Only 26% correctly selected the two-group design and articulated the need for a comparison between two groups.

The problem of poor scientific literacy is not limited to those with a minimal educational background. Despite a number of calls to the educational community to develop programs to improve scientific and technological literacy, the problem persists even among those with a high school or college education.22 In 1989, the American Association for the Advancement of Science put forth a set of recommendations regarding science literacy for all citizens,26 and later provided specific guidelines for what students should be expected to know and do in science, mathematics, engineering and technology (SME&T) at various grade levels.2 In general, recommendations call for approaches that emphasize hands-on problem solving, collaborative group work and inquiry based science to produce citizens who understand the nature of technology and its interaction with science and society. Similarly, the National Academy of Sciences Report on Transforming Undergraduate Education in SME&T calls for diverse opportunities for all undergraduates to study SME&T early in their academic careers.28 Yet today, only about 20% of Research-I and -II universities provide opportunities for active learning or real-world problem solving in a substantial number of introductory science courses.8 At the undergraduate level, most science education courses for non-science majors do not discuss the societal implications of science, and there tend to be few offerings focused on technological literacy.5

In a society that relies increasingly on the use of technology in healthcare, limited scientific literacy is cause for concern. The American Medical Association’s Ad Hoc Committee on Health Literacy for the Council on Scientific Affairs1 reports that “patients with inadequate health literacy have a complex array of communications difficulties, which may interact to influence health outcome. These patients report worse health status and have less understanding about their medical conditions and treatment.” Individuals who lack basic scientific and technological literacy have difficulty obtaining and understanding basic health information to make informed decisions about their own medical care; furthermore, the lack of understanding of basic scientific experimentation can affect decisions of whether to participate in medical research or to select new treatments based on clinical research studies.

Most definitions of scientific literacy encompass both an appreciation of the nature, aims and limitations of science, and an understanding of important scientific ideas.16 Hazen and Trefil make a distinction between doing science and using science, defining scientific literacy as “the knowledge you need to understand public issues.”12 Miller20 has suggested that civic scientific literacy represents a level of understanding sufficient to comprehend scientific arguments put forth in the popular media and requires three components: (1) a vocabulary of basic science concepts sufficient to read the newspaper (scientific content), (2) an understanding of the process of scientific inquiry (nature of science), and (3) an understanding of the impact of science and technology on individuals and on society (scientific context).

Recent studies indicate that courses which integrate science, technology and society can provide meaningful gains in civic scientific literacy and can reduce gaps between high and low academic level students.9,23 Using learning science guidelines4 we designed and taught a novel project-based course to increase civic scientific literacy associated with healthcare among undergraduate non-science majors. The course, entitled BME 301: Bioengineering and World Health, uses the development of new biomedical technologies as a platform. The objective of this study was to assess the effectiveness of the course with respect to the three metrics of civic scientific literacy: (1) biomedical content knowledge, (2) knowledge of the scientific method, and (3) societal context. Improvement in each area was accessed by conducting small group interviews in which students read and discussed a news account describing the development of a new biomedical technology.

Methods

Course Design

The content of the semester-long Bioengineering and World Health course answered four questions: (1) What are the problems in health today and how do these differ throughout the world? (2) Who pays to solve problems in health care? (3) How can technology be used to solve world-health problems? (4) How do technologies move from bench to bedside? The course examined case studies of vaccination to prevent infectious disease, imaging to detect cancer early, and implantable devices to treat heart disease.

Recent work supporting the scientific basis of learning has been summarized by Bransford et al.4 and applied to the design of learning environments from four different perspectives: (1) learner, (2) content, (3) assessment, and (4) community. In designing the course, consideration was given to the four centers of the learning environment, as outlined below.

Learner-Centered Environment

Research has demonstrated that students bring to the classroom knowledge and experiences that affect how they learn and interpret new information.4 Taking into account the need for a learner-centered environment, class activities were designed to uncover this background knowledge and provide opportunities for students to challenge previous notions and practice new ways of thinking. Learner centered activities included daily requirements for student-generated ideas through homework and an online polling system to elicit student feedback and to initiate class discussions.

Knowledge-Centered Environment

Knowledge-centered environments are built on the theory that “expertise” requires a depth of factual knowledge as well as the ability to approach and organize knowledge into a coherent whole, as opposed to many disconnected concepts.4 Much of the course content was organized using uniquely designed case studies whose data comes from sources such as the World Health Organization, the World Trade Organization, the Center for Medicare and Medicaid Services, the Centers for Disease Control and Prevention, the Pew Global Attitudes Project and Medline. Science education research clearly illustrates the advantages of developing curriculum that provides students an opportunity to reason and infer with real data.7,25

Assessment-Centered Environment

Research has demonstrated that the ability to monitor one’s own learning greatly facilitates one’s ability to apply knowledge in new situations.4 An assessment-centered environment provides opportunities for continuous feedback and revision. While the course included traditional summative assessment in the form of homework and exam grades, formative, or daily assessment, was provided through collaborative in-class activities as well as class discussion. In addition, interactive spreadsheets provided a technology supported tool to assess student understanding of complex quantitative concepts.

Community-Centered Environment

A community-centered environment focuses on establishing a connection between knowledge acquired in the classroom and its real-world applications.4 Media presentations, some produced locally and others produced professionally, were used daily to immerse students in the local, national, and international research communities. An interactive website was used to house all lectures, assignments, resources, and to facilitate communication.3

In an effort to align the various components of the learning environment, a series of interactive class activities was developed. Table 1 provides three examples of activities and the learning environments that each incorporates. Details of each activity are provided below.

Table 1 Components of the learning environment included in the design of activities
Full size table

Example One—Pizza Pan Immunology

The course explored the development of vaccines to prevent infectious disease. Students must be able to apply knowledge of how the immune system works to understand and discuss these issues. Yet the course had no prerequisites and most of the students had no biology beyond high school. To provide an engaging background on immunology, while simultaneously promoting and assessing student understanding, a lecture providing an overview of the immune system was developed around a simple and inexpensive interactive learning tool (Fig. 3) consisting of pizza pans and magnets representing bacteria, viruses, and components of the immune system. The lecture was structured around a series of questions that asked students to demonstrate the relationship between various components. Working in small groups, students arranged the magnets and displayed their answers, allowing the instructor to visually assess student understanding. This approach increased student engagement as compared to traditional lecture-based approaches and greatly facilitated understanding.

Example Two—Can Screening for a Rare Disease Cause Greater Harm than Good?

It is difficult for students to understand that screening the general public for a rare disease can identify many more false positives than individuals with disease. In a traditional course, one defines sensitivity, specificity, and predictive value. However, even if students can calculate these quantities, they do not always understand what the numbers mean from the perspective of patients or society. BME 301 initiated this unit by showing a short NBC News clip describing a new blood test to screen for ovarian cancer (OvaCheck).24 Each student then received a sealed envelope stamped CONFIDENTIAL and was told that one envelope contains a guarantee of 100% on the next exam; students were cautioned that some envelopes contain automatic deductions. They each made a decision about whether to open the envelope, representing the choice to be screened for ovarian cancer. Some students opened the envelope, most found no change to their score and a few received a decrease. The class discussed what these outcomes represent—and quickly came to the idea of true negatives and false positives. Students were then provided data to calculate sensitivity and specificity for OvaCheck. Guided class discussion examined why sensitivity and specificity do not tell the whole story, especially when screening for a rare disease, and the class was given data to calculate the predictive value of the OvaCheck test.

Example Three—Debating Ethics

Throughout BME 301, scientific information was placed in societal context, taking into account ethical and economic perspectives. At the conclusion of the vaccination unit, the class was given a reading assignment describing ethical dilemmas associated with testing HIV vaccines in Uganda.27 In class, student volunteers role-played each of the people in the article. The rest of the class acted as the residents of a Ugandan village who must decide whether to participate in a vaccine trial, even if no treatment is offered to those who develop AIDS. A lively debate ensued in which students used their knowledge of the science of immunology, the engineering of vaccines, the ethical principles in the Belmont Report, and the economic realities of healthcare funding in the developing world.

Course Project

Throughout the course, students worked in small groups to complete a research project focused on a disease and biomedical technology of their choice. The goal of the project was to design a clinical trial to test the new technology and to write a clinical research protocol and informed consent document following IRB guidelines. By dividing the project into weekly assignments, students were provided opportunities for assessment and feedback throughout the semester. Weekly assignments included researching the epidemiology and pathophysiology of the disease, identifying the limitations of current diagnostic and therapeutic approaches for the disease, researching the scientific basis of the new technology and its potential advantages and risks, identifying the ideal group of study subjects and control group to test the new technology, calculating a sample size for the study and describing how the clinical data would be analyzed. Students presented an overview of their proposed clinical trial to a mock IRB group.

Design of Student Assessment

To assess gains in civic scientific literacy produced through class activities, a method was designed to assess students’ abilities to critically discuss science news accounts from the popular press. In an IRB-approved study, groups of students were invited to read and discuss an article from the popular press describing a new technology that was not covered as a part of the class. Structured, small group interviews were conducted, video taped and coded to quantify the number of times students correctly introduced terms related to the development and assessment of new health care technologies. The terms (Table 2) were associated with the three broad dimensions of civic scientific literacy as derived from Miller20: biomedical content knowledge, knowledge of the scientific method, and societal context. There is general agreement among scholars that biomedical content knowledge and knowledge of the scientific method are reliable measures of civic scientific literacy.20 While some debate the merits of measuring societal context, this stems from a difficulties related to implementation in cross national surveys.20

Table 2 Terms used to assess students’ civic scientific literacy
Full size table

Each group of students read and discussed a news story from USA Today entitled “Researchers: New Vaccine Can Stop Lung Cancer.”17 This article describes a novel technologic approach to a common cancer, but reports results from a sample size of only 43 patients. An interviewer led the discussion by asking five questions: (1) What are some of the difficulties researchers must overcome in attempting to develop vaccines against cancer? (2) Why would it take so long for a promising new vaccine of this type to reach the market? (3) Does the sample size in this study have any bearing on how the results should be interpreted? (4) Discuss the statement in the article that “the vaccine has no side effects.” (5) If efforts were made to make this vaccine available to lung cancer patients worldwide, what problems might be encountered?

Videotaped discussions were analyzed for students’ introduction of content-specific dialogue into the conversation. Terms were considered to be introduced if students used the term correctly or correctly described the concept denoted by the term. Two different observers coded these videotapes at two different times for a total of four separate codings per discussion interview. Inter-rater and intra-rater reliability were high. Discrepancies were discussed until consensus was reached.

Interviews were conducted with small groups of students from three pools: (1) an experimental group of students enrolled in BME 301 in the Spring of 2004 and 2005, (2) a control group of non-BME majors who had not taken BME 301, and (3) a control group of BME majors who had not taken BME 301. Interviews with BME 301 students were conducted at different points throughout the 16-week semester during which students were enrolled in the course. All groups were treated in the same manner for the discussion interviews and students could participate in only one interview. Student major, current grade point average (GPA), number of semester credit hours of SME&T courses completed, and total number of semester credit hours completed at this university were collected for all students.

Results

The course was taught in the Spring of 2004 with an enrollment of 57 students and in the Spring of 2005 with an enrollment of 65 students. We assessed the effectiveness of the course with respect to the three components of civic scientific literacy by conducting small group interviews in which students read and discussed a news account describing the development of a new biomedical technology. Structured, small group interviews were conducted, video taped and coded to quantify the number of times students correctly introduced terms related to the development and assessment of new health care technologies. Table 3 details the characteristics of student participants in the experimental group of BME 301 students and the two control groups of Biomedical Engineering (BME) and non-BME majors. Within each group, no association was found between a student’s grade point average (GPA) and the number of terms they introduced in the discussion interview to assess civic scientific literacy.

Table 3 Characteristics of subjects
Full size table

Figure 1 compares the average number of terms introduced in each interview for students in the experimental group interviewed at three different time points in the semester and for students in the two control groups. Students in the non-BME major control group introduced the fewest number of terms. For students enrolled in BME 301, an increase in civic scientific literacy was observed throughout the semester; the number of terms introduced doubled from the mid-point of the course to the conclusion of the course. At the conclusion of the course, the average number of terms introduced by BME 301 students exceeded even the average number of terms introduced per interview in the BME major control group. The number of terms introduced by BME 301 students at the conclusion of the course was significantly larger than those for the control groups of both BME majors (p < 0.045) and non-BME majors (p < 0.000011), and for BME 301 students at the midpoint of the course (p < 0.012).

Figure 1
figure 1

Average total number of terms per interview group for students in the control group of non-BME majors (white bar), students in the control group of BME majors (black bar) and BME 301 students interviewed in weeks 7–9, weeks 10–12, and weeks 14–16 (gray bars). Error bars represent ±1 standard deviation

Full size image

We examined whether the number of terms that each student introduced was related to the total number of semester credit hours they had completed in SME&T courses (Fig. 2). Interestingly, for students in the non-BME major control group, the number of terms introduced per student did not depend strongly on the number of total SME&T hours completed. For BME majors, the average number of terms was much higher for students who had completed more than 60 semester credit hours of SME&T courses, indicating the increase in civic scientific literacy associated with increased BME coursework. Both groups of BME 301 students, those who had completed less than 60 SME&T hours and those who had completed more than 60 SME&T hours, achieved civic scientific literacy levels which were comparable to that of BME majors who had completed more than 60 SME&T hours. These data suggest that a single, well designed, introductory course can have a significant influence on undergraduates’ civic scientific literacy (Fig. 3).

Figure 2
figure 2

Mean number of total terms per student for students who had taken fewer than 60 semester credit hours (SCH) of SME&T courses (gray bars) and more than 60 semester credit hours of SME&T courses (black bars) for all three groups of students. Error bars represent ±1 standard deviation

Full size image
Figure 3
figure 3

Pizza pan immunology in use

Full size image

We examined the influence of the course on the three dimensions of civic scientific literacy, relative to the two control groups (data not shown). Compared to non-BME majors, the BME 301 students introduced significantly more biomedical content (p < 0.0000003), societal context (p < 0.0007) and more scientific process (p < 0.038) terms. Compared to BME majors, the BME 301 students introduced significantly more biomedical content terms (p < 0.011). Differences in the scientific process and societal context terms for BME 301 students were not significantly different than those for BME majors. Biomedical content assessed here was primarily associated with the application and assessment of biomedical technologies, topics which are important for health literacy but often not explicitly covered in the BME major curriculum.

Discussion

The scientific literacy of our population has important implications for sound scientific policy decisions, including funding for and the regulation of new health care technologies. The number of public policy issues requiring some level of science literacy is increasing, including stem cell research, emerging infectious diseases, childhood vaccines and controversy over recalled prescription drugs. Further, the dramatic growth of managed health care plans over the last two decades in the US has led to increasing need for health literacy, as patients are increasingly expected to navigate their own way through a complex health care system.14

Current science education methods do not produce students with sufficient civic scientific literacy. A recent study compared whether the use of scientific jargon (scientese) increased the persuasiveness of advertisements for unproven medical treatments.11 Scientese increased message persuasiveness; however, the strength of this effect was unexpectedly the same for both undergraduate science and non-science majors. Similarly, we found that the civic scientific literacy of non-BME majors who had not taken BME 301 did not differ for those students who had taken fewer or more than 60 semester credit hours of SME&T courses.

Promoting scientific literacy requires a new way of teaching.18 A recent national survey of attitudes toward an understanding of science in the UK showed that for those with little scientific knowledge, medical research is judged as far more interesting and scientific than anything else, and these citizens tend to take medical research as a model for all science.29 Science education reforms to address civic scientific literacy should focus on what the public wants to know, for example, using randomized clinical trials to illustrate the scientific method and the difference between controlled experiments and observations.6 This is increasingly important in a world where early studies on new medical advances can receive substantial publicity in the popular press before randomized clinical trials are completed. A recent study compared conclusions presented in highly cited articles in three major general clinical journals to those of subsequent studies with larger sample size or better controlled design.15 Results showed that nearly one-third of highly cited studies were later contradicted and that this was most likely for non-randomized studies. Educational approaches which focus on the connection between biomedical science and human health may have additional benefits for women and underrepresented minority students; a number of investigators have shown that, for students from underrepresented groups, the study of science is made meaningful by connections to other fields.19

As new methods of teaching are developed, it is also necessary to develop effective methods to evaluate student gains in scientific literacy. In this work, we used small group interviews to assess student learning. Our results indicate that scientific literacy assessed in this manner was higher for students in BME 301 than for appropriate control groups of students who had not taken the course. A limitation of this approach is that it does not compare individual student literacy levels pre- and post-course. However, we did conduct interviews at three different time points during the course, and these results revealed that scientific literacy among students enrolled in BME 301 improved over the course of the semester (Fig. 1). We acknowledge that pre- and post-course assessment of all students would provide a better measure of changes in scientific literacy associated with the intervention. However, to accomplish this goal, it is necessary to develop an assessment tool that can be easily administered to a large group of students at multiple time points. The Force Concept Inventory,13 an instrument used to probe understanding of Newtonian concepts, provides an example of such an assessment that not only measures student understanding, but also identifies students’ misconceptions. A tool of similar design to assess biomedical literacy could prove valuable in assessing student gains in scientific literacy and further, to identify specific areas where instruction needs to be improved.

Introductory undergraduate science courses at research universities have drawn criticism for the lack of engagement and limited access to active researchers. The design of a course based upon the four perspectives of the learning environment and sound learning theory seeks to directly address this criticism. Results presented here demonstrate that a large course can include opportunities for active, project-based learning using data from a variety of situations to explore the need for objective health technology assessment. BME 301 shows that a data driven curriculum21 in combination with interactive cooperative learning sequences produces substantial gains in civic scientific literacy and facilitates a deep understanding of the nature of health care technology development and the limitations of health care in different parts of the world. The use of this approach to enhance curriculum in undergraduate courses of other disciplines has the potential to produce similar gains and merits further exploration.