Introduction

Biomedical engineering (BME) departments and programs began to emerge in significant numbers in the 1960s, focused on innovation at the intersection of engineering and medicine.1 The discipline grew from a base of other departments such as mechanical and electrical engineering, where faculty were intrigued with biomedical challenges.2 Graduate programs initially focused on training students in research at the engineering/medicine interface.2 These programs often involved collaborations with medical professionals to augment research efforts and help conceive or improve medical technologies. However, they were largely disconnected from the medical device and biotechnology industries, and BME graduates sometimes found themselves without clear job prospects upon completing their programs.2

Regardless of this challenge, student interest in biomedical technologies continued to grow—and not just among those studying engineering. At Stanford University, as the medical devices industry entered a remarkable period of growth in the mid- to late-1990s, increasing numbers of students from engineering, medicine, and business expressed a desire for coursework and training in the biomedical field. This movement corresponded to another push in academia to do more than equip students with first-rate technical skills but, in an increasingly competitive environment, provide them with a holistic, integrative education that was more practice-oriented and connected to the relevant industry.3. Around the same time, interdisciplinary problem solving was gaining increasing attention as a means of fostering innovation and addressing society’s most pressing problems.4

The formation of the Biodesign program at Stanford University in 2000–2001—with its focus on training the next generation of leaders in biomedical technology innovation—provided an impetus to take a fresh look at educational opportunities in this space. Our analysis made clear that course offerings to prepare students for careers in the local medical devices industry were inadequate. Across the schools of engineering, medicine, and business, there were no courses exclusively dedicated to understanding the dynamics of the medical devices field, developing medical technologies, or learning the variety of steps required to bring a medical invention into patient care. The lack of a graduate course with the objective of preparing students for roles in the field was particularly troubling given the favorable conditions on campus, which included engineering, medical, and business schools in close proximity of one another; excellent adult and children’s hospitals immediately on campus; and access to a deep and diverse network of biotechnology and medical devices industry experts in the Silicon Valley.

Accordingly, we designed a new graduate-level course intended to address this gap. The goal was to create an offering that was: (1) genuinely interdisciplinary, with a mix of masters and PhD students from BME and other engineering disciplines, medical students, residents, and fellows from medicine, and MBA students from business; (2) broad-based and integrative, exposing students to an end-to-end process for biomedical technology innovation; and (3) translational and applied in its focus to explicitly help prepare students to take on leadership and innovation-related roles in the medical devices industry. We would accomplish this using an experiential, project-based model that enabled students to identify important, real-world unmet clinical needs, invent new technologies to address them, and prepare to implement them into patient care.

From 2003 through 2019, graduate-level learners from the schools of engineering, medicine, and business participated in in the course in roughly equal numbers. The annual class size ranged from 24 to 48 students, who worked in blended teams of 4–6 students each. As shown in Fig. 1, the curriculum covered clinical topics (observations and problem identification, disease state fundamentals, treatment options, stakeholder analysis, clinical strategy, etc.), engineering subjects (prototyping, R&D strategy, quality and process management, etc.), and business matters (market analysis, intellectual property, regulatory and reimbursement, sales and marketing, funding, etc.).

Figure 1
figure 1

The biodesign innovation process. This diagram shows how the biodesign innovation process was initially codified (circa 2006). While the process was less formally structured when we launched the two-quarter graduate course, the curriculum covered the same basic topics.

Full size image

The hands-on nature of instruction enabled students to learn and then apply and practice these concepts as they advanced their projects through the two-quarter course sequence. To keep the experience practical (rather than a purely academic exercise), we leveraged seasoned medical device innovators and subject matter experts from the local industry as guest speakers and coaches/mentors on student projects. Over time, we took increasing advantage of our unique location in the heart of the Silicon Valley medtech ecosystem to expand the number of outside experts contributing to the course from approximately 20 per year to more than 70. Our ability to bring back course alumni as they gained professional experience in the medtech industry was helpful in increasing the number of external contributors.

There were other important changes that we made to the course over the years, with a few of the most notable referenced here. One was that, with each offering of the course, we increased the structure of our quarterly course expectations and deliverables. For example, in addition to requiring a mid-term presentation and final presentation/written report, we added four “progress deliverables” to the syllabus. These assignments serve the important function of “forcing” students to take a position on their project and helping prevent them from getting caught in a counterproductive spiral of research and analysis. Another technique that we started using to help the teams drive progress and keep their work grounded in the real world is the expectation that they will conduct at least two stakeholder interviews every week of the quarter. These “validation interviews,” as we call them, are used initially to gather information and test assumptions related to the team’s understanding of the unmet need. Later, they focus on gathering stakeholder input on the proposed solution.

We also recognized how positive team dynamics can dramatically enhance the student learning experience, while negative team dynamics can distract from it. To help address this, we gradually added relevant content to the curriculum, including a unit on team dynamics within the first two weeks of class, the use of team culture documents to help students think intentionally about the types of teams they want to build and sustain, and the addition of team dynamics coaches to meet with each team 2–3 times per quarter expressly about their team processes and group interactions. We also added a peer evaluation process that enables teammates to provide one another with feedback and alerts the teaching team to issues that may be brewing so that we can proactively intervene. In parallel, we modified our grading process, evolving from a purely team-based system to a combination of team and individual assignments (including student scores on the peer evaluation) to enable motivated students to differentiate themselves from their peers.

In terms of course outcomes, from the first year of the class, small numbers of students/teams expressed an interest in continuing to work on their projects afterwards, even though creating companies is not one of our explicit objectives. Initially, we provided resources and support to these teams informally. But, eventually, we created a “summer extension” funding program to make small grants to teams on a competitive basis to enable them to continue advancing and de-risking their projects for 3–4 months as a potential bridge to more substantial pre-seed funding.

Over the years, the number of companies that resulted from class projects was easy to count as one indirect metric of course success (see the results section for more information). However, aside from anecdotal feedback from students, we had no other metrics to help us determine whether the course was helping students in their post-graduate careers. Accordingly, we initiated a survey to gather student input more systematically. Consistent with the course design, we were particularly interested in determining the proportion of our students taking jobs in the healthcare field and, more specifically, the biomedical technology industry (called the health technology industry for the remainder of the paper since this is how we referred to it in the survey), as well as assessing how many were in leadership positions and roles related to innovation. We also wanted to know whether students perceived the course to be influential on their careers and if they found the integrative, need-driven process we teach to be useful in their post-graduate roles.

Methods

We designed and disseminated the survey under an IRB waiver to collect self-reported data from respondents. It included up to 33 questions, depending on how participants responded, and was administered using the Qualtrics platform (Qualtrics, Provo, UT).

The majority of students who had taken the course enrolled through the university registrar’s office, allowing us to pull their names and university-issued email addresses from official university enrollment records (n = 781). However, residents and fellows from the school of medicine were allowed to participate in the class without formally enrolling (and incurring tuition expenses). For these non-enrolled students, we pulled names and email addresses from historical class rosters for the years in which they were available (n = 85).

To request participation in the survey, we sent a direct email message to a total of 866 students in October 2019. However, this message reached only 128 students (14.8%, n = 97 enrolled; n = 31 non-enrolled), primarily because students lose access to their university-issued email addresses after graduation. The university alumni relations office agreed to send one message on our behalf through their database of more current contact information. Excluding those students who had already responded to the survey and those who could not be found in the alumni relations database, this message reached 621 enrolled students in January 2020.

Survey data from Qualtrics were exported into Microsoft Excel (Microsoft, Redmond, WA) for analysis using descriptive statistics. For questions calling for open-ended, qualitative responses, we began with open coding to organize the information and then used constant comparative method5 to identify the most dominant themes. Paired t-tests were used to determine differences between the students’ career commitment before/after the course.

Results

Through our direct email message and the message sent via alumni relations, the survey reached a total of 749 students (n = 718 enrolled; n = 31 non-enrolled). We received survey responses from a total of 186 students (n = 173 enrolled; n = 13 non-enrolled) for a combined response rate of 24.8%.

Respondents spanned all 17 years in which we offered the course as shown in Table 1, with 37.6% of participants (n = 70/186) having taken the course in the most recent 5-year period.

Table 1 Rspondents by academic year.
Full size table

Respondents also represented all three primary disciplines—engineering, medicine, and business—as shown in Table 2, with the greatest participation coming from engineering students (50%).

Table 2 Respondent affiliation during course.
Full size table

In terms of their commitment to a career in health technology, we asked respondents to rate their theoretical commitment level to pursuing a job in health technology before and after taking the course as shown in Fig. 2. Scoring is based on a Likert scale of 1 = not committed, 2 = just exploring it as an option, 3 = interested but not sure, 4 = pretty committed, and 5 = totally committed. Before the course, 62% (n = 115/186) expressed a strong commitment (i.e., a score of 4 or 5). After the course, this percentage increased to 84% (n = 156/186). Our analysis showed a mean score improvement from 3.8 pre-course to 4.3 post-course, which is statistically significant (p < 0.0001).

Figure 2
figure 2

Interest in a career in health technology before/after course.

Full size image

When asked about the first full-time job they accepted after leaving the university, 85% (n = 159/186) of respondents took a position in the healthcare field generally. Of those, 74% (n = 118/159) went to work in the health technology field. In this context, health technology was defined as medical devices, device-based diagnostics, digital health, and/or healthcare information technology. The most common reason cited for not accepting a role in healthcare was “found a more interesting/challenging opportunity in another field.” The most common reason cited for not accepting a role in health technology was “found a more interesting/challenging opportunity in another healthcare sector.”

When asked about their current position, 78% (n = 145/186) of respondents reported working in the healthcare field. Of those, 72% (n = 115/145) had positions in the health technology field. A full 82% (n = 146/179) reported being in a role related to innovation and 58% (n = 102/177) indicated that they held a management/leadership position. Examples of innovation-related roles included “lead an innovation center,” “specialize in design thinking, ideation, and rapid prototyping,” and “serial entrepreneur.” Examples of management/leadership positions included “co-founder and CEO,” “president,” “head of technology,” and “VP, product and engineering” .

Reflecting on their actual longer-term career trajectories, respondents generally reported that the course was influential on their choices, as shown in Fig. 3, even though many students enter the course with an established interest in health technology careers as noted above.

Figure 3
figure 3

Course influence on career trajectory.

Full size image

When we looked at this data by 5-year cohorts, results were generally consistent, with at least 62–69% of responding students rating the course as very or extremely influential on their career direction in every cohort except the first 5 years of the course. In that initial 5-year cohort, 73% (n = 40/55) of responding students rated it as very or extremely influential, perhaps because there was nothing else like it when the course was first offered.

Several themes emerged from the qualitative responses that students submitted regarding why the course was influential. The top three were that it “changed or deepened my understanding of the innovation process,” “changed my career interest,” or “solidified my career interest in health technology.”

Respondents also found the repeatable, need-driven biodesign innovation process to be useful in their careers, as shown in Fig. 4. This was true whether or not respondents were currently working in healthcare or the health technology field, which may indicate that they find utility in taking a need-driven approach to problem solving that is not domain specific.

Figure 4
figure 4

Usefulness of the biodesign innovation process.

Full size image

When we looked at this data by 5-year cohorts, results varied with 74–81% of responding students rated the process as very or extremely useful in every cohort except the 2013–2017 grouping. In that five-year cohort, 67% (n = 30/45) of responding students rated it as very or extremely influential. We are uncertain why the responses from this student subgroup would be lower.

From the qualitative responses that students submitted regarding why they found the process useful, the top two reasons were that they appreciated the “need-driven approach” and having a “structured process for innovation.”

When asked how effective the course was in meeting our defined learning objectives, the majority of respondent indicated that it was very or extremely effective (see Fig. 5).

Figure 5
figure 5

Success in achieving course learning objectives.

Full size image

One opportunity for improvement is in providing access to a durable network of mentors and other contacts that students can access for assistance beyond—46% of respondents said we were only moderately successful, slightly successful, or not successful at all in meeting this objective (n = 79/172). The other is in better preparing students for leadership roles in health technology companies—38% of respondents said we were only moderately successful, slightly successful, or not successful at all in meeting this objective (n = 65/172).

While it is not an explicit objective of the course to stimulate the formation of start-up health technology companies, this is one metric of innovation activity that is relatively easy to count. Among survey respondents, 8% (n = 13/157) reported launching companies based on their class projects, for a total of nine companies. Additionally, 19% individuals (n = 32/171) indicated that they formed a health technology start-up based on a need they identified after completing the course, for a total of 40 other companies.

Toward the end of the survey, we asked participants to share what, in hindsight, they appreciated most about the course and what they believe we should improve. From the qualitative responses, the things they liked most were learning a cross-functional, end-to-end process for innovation; the emphasis on need finding and need characterization as critical activities upstream of invention; the hands-on, project based learning experience; and working in multidisciplinary teams. The primary recommendations for content/curriculum improvements included going deeper on core engineering content and prototyping; incorporating more teaching related to important emerging technologies (e.g., digital health, artificial intelligence, machine learning, and data analytics); better addressing how technology innovation can reduce costs and/or meet the needs of traditionally underserved populations; and spending more time on commercialization planning and what advancing a technology into patient care is like in the real world. Unrelated to content/curriculum, a surprising number of respondents expressed interest in establishing an alumni network, reconnecting with fellows students, and contributing to the course.

Finally, when we asked for any “other comments,” students responding to this question disproportionately volunteered positive remarks, such as “Possibly the most influential course I have ever taken” and “Biodesign empowered me in a way no other course has…the material and speakers were much more connected to the real world.”

Discussion

In general, the survey results indicate that respondents generally placed high value on the interdisciplinary, integrative, translational approach to graduate-level health technology innovation education represented by this course. That said, the data have clear limitations. Given the 17-year history of the course, it would have been useful to hear from a greater number of students. Unfortunately, due to our reliance on the university alumni relations office for distributing the survey, we were not able to send follow-up emails to help boost the response rate. Further, self-selection bias is clearly embedded in the data, as students who had a positive experience in the course are more likely to have participated in the survey. Finally, self-reported data of the kind collected in this survey are inherently subjective and may be inaccurate when respondents must retrospectively recall their perceptions from many years in the past.

Despite these limitations, the data suggest that the course is effective at creating and/or sustaining student interest in health technology innovation, with 96% of responding students indicating that they were as or more committed to a career in the industry after taking the class. Similarly, respondents seemed to find the class experience persuasive in their career decision making, with 93% stating that it was influential to some extent—and 36% saying it was extremely influential—on their career direction.

As for the innovation process we teach, 100% of responding students indicated that it was useful in their careers, with 39% reporting that it was extremely useful. From this, we can infer that many students value being exposed to a systematic framework for innovation that combines elements from the different fields of business, engineering and medicine.

Interestingly, 82% of respondents reported holding innovation-related position, but only 58% said they currently held leadership roles. While there are many factors that could contribute to the difference, we noted that 38% of responding students felt the course was unsuccessful preparing them to take on a leadership position in health technology. This strikes us as an actionable opportunity for improvement in our current curriculum. Additionally, a full 46% of responding students reported that we were unsuccessful in helping them build a durable network of contacts and mentors that extended beyond class. Given our location in Silicon Valley, we believe we are underperforming against this learning objective and that we can be doing more (e.g., creating an alumni mailing list, inviting course alumni to our events, holding in-person and virtual networking events) to create mutually beneficial ongoing interactions with those who have taken the class.

In reflecting on the results of this survey, we believe we face several other important challenges in sustaining the relevance and value of the course. For one, objectively assessing student performance and assigning grades in this type of course is difficult. Our primary goal as instructors is to educate our students on the process of technology innovation, not to help them translate technologies into patient care or launch companies. Yet it is easy to confuse or conflate these objectives when providing feedback to students in project-based courses. Instructors naturally develop biases about what projects hold more promise than others, but it is important to distinguish between feedback aimed at guiding a team’s mastery of the process versus guidance intended to improve a project’s chances of success. When students experience a failure in their execution of the process, we should address and help rectify it to ensure a comprehensive learning experience. In contrast, project deficiencies—according to “commercial” standards and success factors—should evoke a more nuanced discussion around the relative value of the team’s choices and their downstream effects. And these commercial metrics should not work against student assessments or their grades. However, especially when outside experts from the industry participate in class, this delicate balance can be difficult to sustain, and it may feel to students like we are sending mixed messages. One way that we are trying to strike an effective balance is to explicitly lead with feedback focused on students’ execution and understanding of each step in the process. Then, as time permits, the discussion can shift to the commercial viability of the project.

Additionally, we continue to see strain placed on team dynamics in the course as students become more sophisticated and ambitious. While many continue to choose this elective course for the educational experience, others are increasingly interested in launching a company based on their course project. When students with different objectives end up on the same team, this can be a source of tension and frustration for all concerned. In the qualitative survey feedback, respondents cited issues such as “Take into account each member's goals for the course for a better match in team dynamics,” “More input and evaluation to determine which personalities and experiences may mesh well together,” and “Identify upfront which students have capacity/schedule flexibility to pursue the project beyond the course.” Over 17 years, we have experimented with many different team formation models, from allowing students to self-select (early years) to assigning students based on project preference (current model). We also have tried different selection approaches, including assessing student goals via an applications and interviews (both still used today). However, we have stopped short of forming teams based on who does/does not have entrepreneurial aspirations, fearing that this would interfere with our educational mission, place undue emphasis on company formation, and create unintended consequences in the course culture. Instead, we try to make the teaching team (and our team dynamics coaches) available to the students to mediate differences in goals and/or engagement levels throughout the course. We also give teams the opportunity to “recast” themselves (with some members continuing, others not continuing, and other students potentially joining) as part of applying for our summer extension program.

Another issue is that external factors relevant to health technology innovation are challenging educational programs such as ours. New technologies with promising healthcare applications are proliferating rapidly. Students are eager to understand and apply these capabilities to their projects but it is difficult for a small teaching team to keep pace with cutting-edge developments and identify and recruit outside subject matter experts to share relevant knowledge and experience with our teams. A related—and critically important—set of issues is rooted in health equity and access. Traditionally, biomedical technologies and have disproportionately benefitted those who are most privileged in terms of their socioeconomic status and may also be skewed with respect to gender, race, and age. As advances in healthcare accelerate through the application and invention of new technologies, social inequity and the “divisions between winners and losers” are at risk of widening.6 This creates an imperative to evaluate and adjust the process we teach to ensure it does not perpetuate the problem of health technologies being targeted primarily at the privileged.

Understanding and teaching how health technology innovators can help reduce rather than exacerbate disparities in health care is a complex challenge. To begin with, the basic structure of the healthcare system and the business mechanisms for delivering new technologies to patients tend to reinforce systemic health inequities. Even getting accurate data on health disparities can be tricky. And there are relatively few clear examples of how health technology innovation is being used to make sustainable improvements for traditionally underserved communities. That said, educators have an important and achievable goal to help students better understand the factors that lead to and perpetuate health inequities. As a first step toward this objective, we are trying to raise student awareness at multiple steps in the process we teach. For example, when choosing needs that will serve as the basis for student projects, we have started looking for problems that disproportionately affect minority populations. When assigning need research, we are challenging our teams to look beyond mainstream data sources, talk with stakeholders representing a diversity of backgrounds, and consider how factors such as access, availability, and trust affect the healthcare experience for different segments of the affected population. When benchmarking existing technologies or developing design requirements for a new solution, we are trying to prompt students to think about the unintended consequences of technology design choices.

None of the steps we are taking to address the challenges listed above provide a complete or definitive solution. However, they reflect the philosophy of continual improvement and active experimentation that has guided the evolution of our course from the start.