Student and participant populations
The SyBBURE Searle Program has had 282 undergraduate students participate in the program. Ninety-eight (34.8%) of these students were female, and 184 (65.2%) were male. The program’s overall ethnic diversity includes 141 (50%) Caucasian, 36 (12.7%) Asian, 20 (7.1%) African, 16 (5.7%) Latino/Hispanic, 15 (5.3%) East Asian, seven (2.4%) South/Southeast Asian, one (0.3%) Middle Eastern, one (0.3%) Jewish, two (0.7%) Caribbean, 13 (4.6%) of two more ethnicities, five (1.8%) other, and 25 (8.9%) unknown or unreported. These data are shown in the left panel of Fig. 1. These students represented a wide range of majors, as shown in the left panel of Fig. 2. Four students who participated in the program as advanced high school students did not have majors at the time and have not been included as part of the discipline diversity. This data is compounded with the high frequency of double majors in the program (51/282; 18%).
The VIX framework began to be developed in the summer of 2014. Since then, the SyBBURE Searle Program has had a total of 94 students who have participated in VIX or an earlier version for varying durations. The population of students since 2014 matches the overall demographics of the SyBBURE Searle Program with 61 (64.9%) male and 33 (35.1%) female students (Fig. 1, right panel). Ethnically, this population also matches the entire cohort of students with 44 (46.8%) Caucasian, 13 (13.8%) Asian, seven (7.4%) African, four (4.6%) East Asian, four (4.2%) Latino/Hispanic, three (3.2%) South Asian, one (1.0%) Caribbean, three (3.2%) students representing two or more ethnicities, two (2.1%) other, and eight (8.5%) unknown or unreported. Finally, this population also matches the overall discipline diversity of the full cohort of students (Fig. 2, right panel). However, since 2014, there has been an increase in the number of double majors (50/94; 53.2%).
There have been nearly 50 VIX teams, examples of which are listed in Table 2. Projects generally spanned a variety of areas, including fabrication, electronics, music, robotics, synthetic biology, chemistry, microfluidics, software, and app development, education, art, fashion, woodworking, sports medicine, 3D printing, and consumer product development. Projects were categorized into one of the following five areas, with the percentage of projects in that area included in parenthesis: Art/Design (11.4%), Food (14.3%), Science (20%), Education (5.7%), Technology (17.14%), Health (20%), and Community (11.4%). On average, there were 3.8 students per team.
Previous iterations building toward the vix framework
The VIX component of the SyBBURE Searle Program was implemented each term (summer, fall, and spring) beginning with the summer of 2014 up until the spring of 2018, when the program switched to an annualized model, as shown in Figs. 3 and 4. While the general intent of VIX over this time period was constant, the curriculum was iterated based on student feedback, both formative and summative, that was collected throughout the program. We describe and discuss the development and iteration of this design approach and environment in detail to give readers a sense for how they might implement and apply these principles to their own enterprises.
In the summer of 2014, out of a desire to improve the teamwork skills of our students, we launched our program’s design component by having students participate full-time in a 2-week innovation and entrepreneurship workshop. Through this workshop, we were able to jumpstart the students’ ability to think of their research project beyond the walls of the lab and to focus on design and problem-solving. Upon completion of this workshop, the students were introduced to the first version of our team-based design experience. We focused initially on encouraging the students to explore solutions to local or global problems in science, engineering, and medicine. They formed teams around a provided list of problems (based on grant calls) in which they were interested and worked to learn and devise strategies to solve these problems. Each week, they had the option to go deeper or to shift their problem focus. Teams presented updates weekly. The students were given the option to continue on their same project or switch to another in the following fall and spring. Outcomes were mixed, with no teams continuing, and one student pivoting their design ideas to their SyBBURE Searle research project. Students then spent the fall and spring largely exploring other problems, but making little progress toward the creation of a solution.
For the second iteration, starting in the summer of 2015, student teams were encouraged to identify their own problem, which required them to lean on their observational and assessment capacities. To make progress, they had to engage in the project, seek out information and experts for help, and develop strategies for successful teamwork. Presenting concise reports was a major portion of this activity, promoting strong oral communication. At the end of the iteration, the students were given the opportunity to assess the activity, which allowed regular improvements to be made. Additionally, during the summer of 2015, two teams of students were selected to engage in a pilot program to explore a series of topics, including developing a problem statement, assessing their idea, determining design requirements, brainstorming solutions, and project management. This pilot was used to develop materials and lessons that shaped the team-based design experience within the entirety of the SyBBURE Searle Program. Both pilot teams were able to produce and iterate a prototype, leading us to realize that a more defined framework is needed to enable students to make progress toward finding solutions to problems. During the fall and spring semesters, teams completed weekly updates (by way of a form) to help guide them through the design process.
Building on the lessons learned through the pilot and in the main design thread, during the spring of 2016, we began to develop a design guidebook with activities to lead teams of multidisciplinary students through the process of identifying their interests, researching those areas, determining problems in those areas, developing solutions to these problems, refining solutions, prototyping, and iteratively improving prototypes. We branded this design guide “Vigilante innovation” to encourage students not to wait for others to make a positive impact on society, but rather to take matters into their own hands. This guide and activity set was then used during the summer of 2016. We focused on a process of “think, create, and innovate” to lead students from generating ideas and finding problems, to building and getting feedback on a prototype. The guide provided students with a roadmap for invention that was particularly evident in the number of functioning prototypes produced over the summer. All student teams, for the first time in the history of our design framework, created prototypes. Many teams chose to continue into the fall and spring (Fig. 4).
Over the next year, we worked to enhance the guide visually and increase the number and type of activities. We also created a flexible track plan with suggested pathways through the guide based on the type of project being pursued (science, consumer product, education/community, health, and technology). We altered our phase-based approach to that of “understand, ideate, prototype, and validate,” and implemented the improved guide starting in the summer of 2017. We focused that summer on pushing students toward validation and realized that 10 weeks of roughly quarter-time effort was not sufficient to reach the validation phase. An additional factor contributing to student team success that arose during this summer was the creation of a student prototyping and testing space. For the first time in our program’s history, we were able to provide dedicated physical space in which students could work to develop and test their prototypes. By the end of the summer, all student teams again created prototypes, and many teams chose to continue their projects into the fall.
With the realization that time is a critical factor, we began restructuring the VIX process into a multi-semester approach. We also recognized that the physical space in which students could work on these projects had to be particularly accessible and, in many ways, driven by student needs. We spent months working closely with students to design and create ideal prototyping and validation space. Then we began a new cycle of VIX in the spring of 2018 under this new timeline and enhanced the physical environment. We formalized exploration and team formation as phases on the front end of the process. Teams were charged with completing the exploration, team formation, and understand phases during the spring semester, individually brainstorming in the break between the end of the semester and the start of the program, and completing the ideate, prototype, and validate phases during the 10-week summer program (still at ~ 25% effort). This change allowed student teams to progress further on projects and to engage more in the design process. By the end of the summer, all student teams had validated their prototypes to some extent. All but one team chose to continue their projects into the fall to continue validation. The remaining students began exploring potential new problems to work on beginning the following spring. The project cycle then continued.
The design project aspect of the SyBBURE Searle Program is now a staple of the program. Students now seek out the program as a way to engage not only in research but also in design. While this may skew the outcomes, our lessons learned remain. Although most of our results are anecdotal and gleaned from observation or focus group discussion with students, the implementation of these principles has allowed for successful integration of a design process and environment with the SyBBURE Searle Program: (1) students need a flexible structure when working on team-based design projects; (2) students are more motivated when working in areas of their own interest; (3) physical space with appropriate prototyping and validation tools removes hurdles and enables students to make more progress on their projects; (4) teams work better with a clear, self-selected leader; (5) deadlines with deliverables related to goals are critical; and (6) understand and ideate phases may be concurrent and iterative to more closely mimic real-life innovation and invention.
After sufficient testing, iteration, and learning, we landed on the flexible, phase-based design process shown in the bottom panel of Fig. 3 and top of Fig. 4 along with images representative of the example activities conducted during each phase. The first phase is exploration, during which the students considered their individual interests and related problems. This step has been accomplished through many mechanisms, including a group interest network diagram created with sticky notes, speed dating, and more hands-on approaches. Our group sticky note interest network diagram was particularly useful. In this activity, students write their name and a single interest or problem area on a sticky note, with no limit to the number of ideas they could propose. All of the notes are then compiled and sorted into a word cloud-type diagram, with common or related sticky notes close together in nodes and lines connecting lesser related nodes. Students can review the interest network diagram and self-sort around these common interests into groups. The second phase of the process is team formation. We didactically provide the context for the importance of team roles, cover basic content regarding these roles, and encourage students to take a simple online quiz to gain insights on the roles they typically play in teams. Once students form teams, they decide on an initial area of interest and discuss their team roles and the skills needed to complete the project. After the teams are formed, the students enter the understand phase in which they conduct background research, talk to experts/users, and describe a problem they wish to solve. The students then transition to the ideation phase in which they brainstorm solutions both individually and as a group and then select the features to include in their design. Students move into the prototype phase to create an initial prototype of their solution, and then into the validate phase in which they evaluate this prototype against their design features and constraints. The process ends with each team giving a final presentation and demo of their validated prototype.
VIX electronic survey
Twenty-two of 29 students responded to the survey in 2016, and 32 of 32 students responded in 2017. The results of the survey are shown in Tables 3 and 4, with data from 2016 and 2017 reported separately as the exact VIX program that was implemented differed. The main distinction between these two periods was related to the inclusion of exploration and team formation phases and the additional training and activities that accompanied these phases.
As shown in Table 3, data from both 2016 and 2017 indicate that students score learning how to collaborate with others as the most important component of VIX, with the mean rating from students increasing over the 2-year period. Data between the 2 years also indicates that students rate meeting with experts as the least important component. The order of importance for the remaining components is different for the two terms. Learning a problem-solving method saw the largest positive change in student-rated importance over the 2-year period with a difference in the mean rating of 0.39. Conversely, the largest negative change in mean rating (− 0.23) was observed for answering a scientific question which my team and I developed. It is unclear whether changes in the population of students or the VIX program itself contributed to differences in the importance of each component to the students during the 2-year reporting period. Additional years of data and pre/post-testing of the students could clarify whether we are enrolling students with more interest in certain components or providing training in such a way that they come to value different components.
When looking at how the experience addressed the key program components (Table 4), in 2016, the mean rating of all aspects falls between 3.59 and 3.91, indicating that components were nearly equally addressed, with learning a problem-solving method scoring the highest-rated and meeting with experts the lowest rated. In 2017, the mean rating range widened to 3.44 to 4.09, and the components that were particularly well addressed based on student response included solving a real-world problem and learning a problem-solving method. These two components had the highest mean rating change from 2016 to 2017 (solving a real-world problem, 0.20; learning a problem-solving method, 0.18). Meeting with experts again showed to be the least-included component as it was indicated by the student responses the previous year and also had the largest negative change in mean rating (− 0.15). While we have discussed the logistics of finding and meeting with experts, formulating questions to ask them, and taking thorough notes in several iterations of the VIX program, we have never required the consultation of experts as the timeline of the program does not often line up with the schedules of in-demand experts. This is perhaps a shortcoming of VIX as the survey results clearly show meeting with experts to be both the least-valued and least-addressed component. Given the iterative nature of VIX and the lack of control groups of students to compare these results, we take the results to indicate that we have included what we sought to include, but that there is always room to improve.
Student comments in an open-ended response around improving the program included making it less product-focused and making the structure of the VIX process more flexible. Students also commented on the fit of the activities in the guide to their actual projects, indicating a need for an approach sufficiently flexible to meet the needs of the variety of projects on which they wanted to work. Many students felt that the requirements were arbitrary and could have been achieved with more general guidelines, again suggesting a need for greater flexibility. The solution included adding different activities for different types of projects, along with recommended activity paths to help guide the variety of student projects. We also altered the guide by allowing students to set their own milestones and timeline; upon implementation, however, we learned that the instructor/facilitator should set major milestone dates, as students at this level need assistance with time management. Despite student desire for greater flexibility, the overall productivity during the time period for which the survey was focused (summer of 2016) was such that all groups achieved a viable prototype. One student commented, “I think that was due in large part to the amount of available time. During the school year, we won’t have near[ly] as much time to put into [VIX], and I think that could really slow progress and frustrate students. So, if we developed a slower version of [VIX], or put it on hold and did something like a journal club for the school year, that would be better in my opinion.” Students also commented on reducing the size of VIX teams, which was implemented in later iterations. Another student commented, “I believe it enhances the research experience by introducing new ideas and methods to use in my research. I think it can be improved to create more value for students by incorporating more in-depth problem solving and generation. [VIX] is a really awesome way to apply creativity to the scientific process and explore personal interests.” The majority of students were very positive about the VIX experience, and when asked “If there was one thing you would like the SyBBURE Searle Program to keep, what would it be and why?” responded with comments like “[VIX]. It really sets us apart from other lab experiences”; “[VIX]. It is a fun opportunity to collaborate with our peers”; and “I would like the program to keep [VIX] groups because of how it promotes inventive thinking as well as collaboration.” Overall, findings from this survey elucidated many alterations to the VIX structure to improve its framework and design. These alterations were implemented in subsequent versions of VIX beyond the summer of 2017.
As the electronic surveys were initially designed for quality improvement, their rigor and experimental design for the case study we present here is limited. Between 2016 and 2017 when the surveys were conducted, numerous variables were manipulated, including major changes to our design guide and the introduction of a physical space; thus, we are limited in the conclusions we are able to draw from these survey results. Additionally, while we have established specific goals for VIX that include student skill development and comfort with the design process, we have not conducted a pre/post-test to establish whether these goals were met. In order to provide some insights into the perspectives of the students participating during this 2016–2017 period, we provide the following interview results.
Interviews to explore student views on VIX and teamwork skill development
Students were asked to voluntarily participate in an in-person interview session to gain further insight into the impact of VIX. In the spring of 2018, 13 students were asked the questions listed in Table 1. For this case study, we utilized student responses to questions 1, 2, 3, and 4 to establish if the survey group was representative of the larger population. The disciplinary diversity of the students included majors in biomedical engineering (4), cell and molecular biology (1), medicine health and society (2), mathematics (3), computer science (1), neuroscience (1), chemical engineering (3), and pre-medicine (1), with 6/13 double majors, a subset of the major distribution of the full program. Questions 8, 9, 10, 11, 12, 13, 14, and 15 were used to evaluate and iterate the VIX framework. The interviewer coded student responses to these questions. Seventy-seven percent of the students reported creating a prototype, which occurs toward the end of the VIX process. When asked to indicate the most important aspect of team success, students responded most commonly with “diversity” (16%) and “organization or strategic approaches” (16%). The next most common answers were “having a clear leader” (11%) and “overall teamwork or team dynamics” (11%). Students also mentioned “creativity” (9%), “work ethic” (9%), “group size” (9%), and “communication” (9%) as critical components. Finally, a few students mentioned “passion for the project” (7%), “research” (2%), and “consistency” (2%). These results, including examples of response wordings, are shown in Fig. 5.