One of the focal activities of the INSuRE (Information Security Research and Education) project  is an applied research class. INSuRE is a consortium of 10 universities (Purdue University is the lead institution, and the other participants are Carnegie Mellon University, Dakota State University, Iowa State University, Mississippi State University, Northeastern University, Stevens Institute of Technology, the University of California Davis, the University of Maryland Baltimore County, and the University of Texas Dallas), plus the U. S. Department of Defense, Sandia National Laboratory, Pacific Northwest National Laboratory, Oak Ridge National Laboratory, the Indiana Office of Technology, and Hewlett-Packard. INSuRE aims to develop a partnership among sponsors that perform cybersecurity research and need the results to perform their missions, and cybersecurity researchers who conduct the research and produce results, including students and faculty at Centers of Academic Excellence in Information Assurance Research (CAE-R). INSuRE aims to become an agile, self-organizing, cooperative, multi-disciplinary, multi-institutional, and multi-level collaborative research project that can include both unclassified and classified research problems in cybersecurity. Currently, students from these 10 universities work on cybersecurity problems through coursework, directed independent study, and theses or dissertations.
The INSuRE applied research class provides an opportunity for students to work on problems provided by sponsors, as well as to be mentored by practitioners in the real world, rather than working solely on faculty-led research. More pressing and urgent problems are addressed, allowing the students to also benefit from the guidance of multiple and interdisciplinary research faculty from several institutions. The student-led research may in fact provide solutions for pressing national problems . To facilitate scientific discovery, learning, and collaboration we use an open source software platform called HUBzero®. HUBzero includes a powerful content management system built to support scientific activities. Users on a hub can write blog entries and participate in discussion groups, but it is possible to do so much more. They can work together on projects, publish datasets and computational tools with Digital Object Identifiers (DOIs), and make these publications available for others to use not as dusty downloads, but as live, interactive digital resources. Simulation/modeling tools published on a hub can be accessed with the click of a button. They run on cloud computing resources, campus clusters, and other national high-performance computing (HPC) facilities and serve up compelling visualizations.
Prior to the class, faculty solicit research proposals from external organizations in government and industry. These proposals are a paragraph or two in length, and describe a research problem in fairly general terms. For example, a proposal to examine biometric systems of authentication might be as follows:
Title: Security of Biometric Authentication
Biometric devices provide information about people that is often used to authenticate their identity. This information must be associated with other data that is used to match up the data from the device to the user. This raises two questions. First, how easily can the biometric device be fooled into reporting incorrect measurements? And second, can the user change the comparison data on the system? This project explores the second question by determining how to change the comparison data for a given biometric device.
This Spring, the list includes projects on forensics, using code variation as a defense, an analysis of the proposed TCPcrypt protocol, machine-assisted semantic understanding of code, profiling industrial control system nodes, and the impact of known vulnerabilities upon layered solutions. The list is compiled and made available to faculty immediately and to students on the first day of the semester/quarter.
Obtaining sponsor interest has thus far been very successful; indeed, typically there are more proposed projects than there are students. Faculty members have solicited projects from people they know and, in many cases, have worked with. Most projects have come from government groups, but industrial firms and organizations have also proposed several. Interestingly, the latter typically take longer to prepare and get approval for projects than do the government organizations. For example, at least two companies were hoping to propose projects for the Spring term, but were unable to obtain the necessary approvals in time. Whether students can propose their own projects is up to the faculty member teaching the class. Some faculty allow this if the projects are substantial enough and deal with a current topic, on the basis that the students are best motivated when they are working on a project that they feel strongly enough about to propose. Other faculty members prefer that students select from the sponsor projects. As the proposed projects are typically broad, the students and sponsors have had no trouble narrowing down the projects to be of interest (sometimes enthusiastically so) to the students.
Complicating project selection is that different universities have different rules about working with sensitive projects. For example, the University of California at Davis does not allow any classified work to be done on campus, because that would restrict the ability of the researchers to publish (among other reasons). But other universities do. Proprietary work for industry has similar but different constraints. Thus, all sponsors must agree that, should the results and the work merit publication, the research from any project they propose can be published. As of now, this has not been a barrier to obtaining interesting projects.
The students prepare bids on at least two projects. First, the sponsors make a brief presentation to clarify their research needs and goals. Then the students engage in exercises to identify the knowledge, skills, and competencies required to work on the projects they are interested in. Each bid has four key components: a personal statement of interest, a description of the research problem (the most substantive section), the expected outcomes, and a description of student’s skills, knowledge, and abilities relevant to the problem. Based on the students’ bids, the faculty and research sponsors move quickly to form research teams.
Critical to the success of the project, of course, is that the team members work well together. In some cases, faculty and sponsors select the students that make up each team, which requires judging how amenable the members are to one another. The rationale for pairing students in teams can be based on student interest, expertise, and/or work style. In other cases, the students organize their own teams; the faculty and sponsor must accept the membership. Having students organize their own teams provides them with an opportunity to consider the factors that will constitute a research team, which is a valuable lesson. This ensures that the teams are balanced. Sometimes team membership changes after the initial formation. For example, at the University of California at Davis, a team of three members was reduced to two because one of the students became more interested in a different project, and so moved to the other team. However, all the students knew one another to some degree, and there were no problems with the change. The project has also started forming cross-institutional teams for the first time in Spring 2015. There is a three-person team with students from Purdue University and Mississippi State University, and another two-person team with a student from Dakota State University and one from Purdue University.
These teams next prepare a proposal, the contents of which are similar to that which would go into a National Science Foundation proposal (but with much less detail). The key components of the proposal are the review and analysis of previous work, and the statement of the specific aims of the project. The proposal also contains a schedule of milestones that the students believe they can meet, a plan describing how the students will approach solving the problem, and a bibliography. It also requires a realistic schedule and budget, a list of deliverables, and a discussion of any foreseeable difficulties and anticipated plans to overcome them. When writing the proposals, the students interact iteratively with sponsors and faculty to define the scope of the problem and near-term action steps to be taken. This step is critical in helping students assume the research problem as theirs as opposed to a work for hire, where the sponsors have “dictated” the scope of work and the students are simply following directions.
Once the proposal is approved, the students begin their research. As a first step, the students conduct a thorough literature review. This augments the quick literature reviews done earlier. Those reviews are simply aimed to show that the project has not been done earlier, and that it is substantial enough to advance the state of cybersecurity in some way. This literature review is structured around an argument or arguments. Typically, these arguments point out critical gaps in the existing literature, or how the work in that literature might be extended. If the goal of the research is to validate or correct a published result, the argument would explain the context of the work to be validated or corrected, why it is important, and what would happen if the prior results were incorrect or not corrected. The literature review is of sufficient importance that it is treated as an assignment and is weighted as much as the proposal is weighted.
Following that, the students begin acting on the plan laid out in the proposal. However, the teams are not left on their own to simply execute a 10-week project plan. Instead, teams meet every week with faculty and sponsors to report progress, the challenges encountered, how they are dealing with those challenges, and the next weeks goals. The goals sometimes change based on the challenges encountered. The rapid, successive iterations permit sponsors to modify incremental research goals and apply results based on intermediate findings as the work progresses (the principle of incremental management within a semester or quarter), and allows students to experience first hand the truly iterative and fast-paced nature of cybersecurity research.
The class requires students to prepare a midterm progress report that is delivered as a formal presentation to all classes across the universities via teleconference, and a final project presentation that also includes a written report and poster. At the end of the semester, all students present the results of their research. For those on a semester system, this is a final presentation and report. For those on a quarter system, this occurs in the middle of the second quarter, and so is a penultimate presentation and report. Finally, those on the quarter system do a final project report and presentation at the end of the second quarter. The sponsor and the faculty member then evaluate the project.
One critical aspect of the final assignment is to document the progress made in a manner that allows the sponsor to iterate the next increment, and allows a team (at the same university or a different university) to pick the project up the next semester/quarter and continue the research, also in an agile, iterative manner. The specific manifestations of this differ based on the nature of the project. In some instances it includes a theoretical model that is sufficiently explained to allow a new team of student researchers to simulate and test the model. Another instance might be curating a dataset in a manner such that it is available for reuse and preservation. This type of documentation is essential to enabling the sponsor and faculty to incrementally manage research projects across semesters or quarters, and across institutions.
The sequence below summarizes the steps that the students follow :
Progress report and presentation
Final report and presentation for schools on semester system; penultimate report and presentation for schools on quarter system
Final report and presentation for schools on quarter system.
Given that the class uses a non-traditional model of research, specifically one with a much tighter time-line than traditional research, a new model is needed. Fortunately, such a model has been developed: the Agile Research process.