There have been increasing calls in recent years to make changes at the undergraduate level to better prepare students to apply mathematical modeling and critical thinking skills in general (Bordogna 1998; Radzi et al. 2009), and specifically in the field of hydrology and water resources engineering (e.g., Bourget 2006; CUAHSI 2010; Habib and Deshotel 2018; Howe 2008; Ledley et al. 2008; Merwade and Ruddell 2012; Wagener et al. 2010). Much of this is motivated by the perception that traditional approaches to teaching, focused on separate learning of distinct processes, do not provide a sufficiently holistic environmental and societal context for learning. To address these problems, we created an innovative online module, a unit of instruction organized around a central theme and hosted on the Internet. Our module develops mathematical modeling skills in addition to knowledge of the fundamentals of rainfall-runoff processes and engineering design. The HydroViz Dry Canyon module (HDC; https://hydroviz.org/Lessons/Index/UT/DryCanyonFFP), adopts the pedagogical strategy of a case-study in which an instructor uses “a written description of a problem or situation” to teach a concept situated in a real-world context (McDade 1995, p. 9). The problem of providing flood protection to the community living below the Dry Canyon watershed near Logan, Utah, United States, serves as a case study representative of many flooding problems throughout the world. The HDC module leads students through a series of data- and modeling-driven activities to design the dam and outlet structure for a flood detention basin used to hold flood water and reduce peak flow during times of flooding. In an effort to update the teaching approach for the required undergraduate hydrology course in the civil and environmental engineering degree programs at two United States universities, the HDC module was incorporated into the curriculum. HDC was designed with these courses in mind and developed for use in a blended learning setting, which includes face-to-face lectures, class time dedicated to a computer lab, and self-directed online learning. However, we also developed HDC with the intention that it could be a stand-alone learning tool.

The HDC module was designed for students to create a mathematical model based on the hydrological processes in Dry Canyon for the purposes of designing a detention basin to protect against flash flooding. The module assumes that students have completed prerequisite courses in mathematics typical in an engineering curriculum (e.g. calculus), but that context-based learning of these topics may have been limited (consistent with the findings of Faulkner et al. 2019, reviewed in the literature review). Additionally, in our experience, engineering students need support in applying the mathematics skills acquired through these prerequisite courses. Therefore, our first goal for the module is to elevate and reinforce basic mathematics knowledge through authentic problem based learning and mathematical modeling (consistent with recommendations of Crawley et al. 1994, reviewed in the literature review). The module also assumes that students have not yet been introduced to the hydrologic theory (e.g. precipitation, infiltration, runoff generation) needed to complete the module activities. Rather than following an approach of introducing the theory first, as is often done, the module introduces the content knowledge in the context of the authentic problem case study and then immediately requires students to problem solve. First it introduces how to delineate a watershed in the context of determining the watershed attributes needed for the case study and asks students to delineate the Dry Canyon watershed. This requires combining the geographic skill of watershed delineation with the mathematical skills of area calculation and area averaging. Next it examines precipitation measurements and the mathematical probability and statistics of precipitation data analysis, just before asking students to use historical data to design theoretical precipitation events based on established probabilities (e.g., a 100 year event with 1/100 probability of occurrence in any year). Then the module presents soil properties and the mathematics of infiltration and runoff generation just in time to model the runoff produced from the previously calculated precipitation. Then it introduces the mathematics of hydrograph theory to help students learn how watershed properties, as well as flood protection infrastructure (i.e. a detention basin and its outlet structure), modulate potentially damaging hydrograph peaks. Finally it introduces students to engineering software that encodes the mathematical and hydrologic process knowledge that they have learned, to enable them to experience the challenges of iterating on different protection design alternatives (detention basin and outlet flow control structure sizes) to learn about and evaluate the trade offs involved in engineering design.

The innovative module is premised on pedagogical studies indicating that student learning of mathematical and engineering concepts is enhanced through active interaction with the content (Prince 2004). Mathematical modeling facilitates these interactions. Thus, our second goal was for the HDC module to incorporate digital learning tools to engage students in mathematical modeling in the context of an authentic hydrologic design case study, and to develop students’ skills not just to compute or execute mathematical models, but to use them thoughtfully to make engineering decisions. It is our hope that this also serves as an example for other instructors to develop case studies in other areas where mathematical modeling is used in engineering.

The purpose of this study was to engage students in a teaching and learning innovation bridging mathematical modeling with hydrologic processes through the HDC module, and to examine their perceptions of their experience with the module. This paper examines student perceptions, the mental impressions they formed, expressed in their descriptions and opinions from doing this module, as reported in student surveys. Without a direct measure of student learning, these perceptions serve as descriptions of students’ experiences with the module rather than the impact of the module on their learning. The findings, based on perceptions, clearly have limitations, but do offer insights into students’ experiences with mathematical modeling embedded within the HDC learning module. Moreover, it is important to factor in student perceptions when considering curricular changes. The following research question guided this study: What are students’ perceptions of the use of mathematical modeling in an online module (i.e., HDC)?

The next section is a literature review that provides background on the key knowledge on technology and mathematical modeling in engineering education that underpins this paper. This is followed by the theoretical framework that draws upon Greefrath’s (2011) conception of mathematical modeling. The methods section then describes the iterative development of the HDC module and the data collected to understand student perceptions following their experience with the module. This is followed by sections giving findings, then discussion, limitations and implications.

Literature Review

As part of their review of the history of hydrology education, Ruddell and Wagener (2015) describe a recent shift in pedagogies to include internet-based data resources, modeling based activities, virtual trips, and gaming, among others. They write that one of the grand challenges of hydrology education in the twenty-first century is to “augment theoretical instruction with data and modeling driven cybereducation” (p. 5). Ruddell and Wagener argue that data and modeling driven cybereducation may be one of the best ways to teach complex systems and advocate for this type of instruction at the undergraduate as well as graduate levels. Integrating data and modeling driven cybereducation in hydrology courses requires the effective integration of technology, including cyberlearning resources, as well as mathematical modeling tasks. In the next sections we review research on each of these topics (i.e., technology and modeling) focusing on literature on innovative practices as well as on student perceptions, and then combine them in the theoretical framework.

Technology in Engineering Education

The United States’ National Science Foundation’s Task Force on Cyberlearning (2008) wrote that “widespread access to technology, increasingly sophisticated tools, and advances in understanding of how individuals learn combine to provide a stunning opportunity to transform education worldwide” (p. 7). In their report, they highlight how technological advances can bring new learning opportunities to students in science, technology, engineering, and mathematics (STEM) fields and call on researchers to leverage emerging technologies to enhance the learning opportunities of students in STEM. They note that cyberlearning could “allow interaction with scientific data, visualizations, remote and virtual laboratories, and human expertise” (p. 7). The Learning Enhanced Watershed Assessment System and the Online Watershed Learning System integrate these suggestions from the Task Force on Cyberlearning (Brogan et al. 2016). Specifically, these online learning platforms created a virtual laboratory for collecting and analyzing data while learning hydrologic content, and surveys of students using these platforms reported positive results. Other researchers have also found positive effects on student understanding of engineering concepts after working with virtual laboratories in their courses (Baher 1998; Habib et al. 2012; Kollöffel and de Jong 2013; Koretsky et al. 2011).

The National Science Foundation (2008) has called on STEM educators to integrate technology meaningfully to leverage student learning based on research indicating student learning gains for engineering students who participated in courses that integrated technology (Baher 1998; Brogan et al. 2016; Kollöffel and de Jong 2013; Koretsky et al. 2011). Researchers are still studying the best ways to do this. In 2012, Merwade and Ruddell surveyed hydrology instructors to better understand what the challenges and priorities should be for integrating data and modeling driven cyberlearning in university hydrology courses. One of the many suggestions by the faculty surveyed was to use publicly available datasets and generally accessible tools (e.g., spreadsheets software such as Excel or geographic information system software such as ArcGIS). They also cautioned against creating “black box” tools, meaning tools which allow the user to interact with the inputs and outputs but obscure the tool’s internal workings, making the concepts and modeling less apparent to the user. Habib and Deshotel (2018) conducted a set of 78 informal interviews with faculty members and practicing engineers in the field of hydrology and water resources. Their results emphasized the value of using emerging technologies in undergraduate classrooms, but also highlighted critical barriers such as time limitations for integration of technologies, the need to keep up with continuous updates to the modeling tools and datasets, and the steep learning curve of the technological tools by students and instructors. These findings suggest that, as instructors integrate technology into their teaching, they should be mindful of pitfalls, including technology as a “black box” and the potentially steep learning curve.

Student Perceptions of Content

There are three types of interactions that happen in distance education: learner-instructor, learner-learner, and learner-content (Moore 1993). Given that HDC was used as an independent learning module for students to complete individually, the interaction that is most important to consider is the learner-content interaction. Moore wrote that “it is the process of intellectually interacting with content that results in changes in the learner’s understanding, the learner’s perspective, or the cognitive structures of the learner’s mind” (p. 20). Gosmire et al. (2009) found that graduate students’ perceptions of their interactions with the content were statistically significantly positively correlated to their overall interactions in an online course. The importance of the interaction between the learner and the content in changing the learner’s perspectives highlights the need to examine learners’ perspectives on the content they engage with through cybereducation. Moreover, exploring student perceptions may give insight into what they found engaging or disengaging about the content, which is especially important as student engagement in mathematics is tied to student achievement (Middleton et al. 2017).

Mathematical Modeling in Engineering

The traditional structure of engineering curricula at most United States colleges and universities originated in the early 1950’s (Ruddell and Wagener 2015; Shea 1997). However, the modes and applications of the engineering concepts within industry are constantly changing and fundamental changes in engineering concepts will not occur nearly as quickly as their application (Moussavi 1998). Therefore, not only must new technology be brought into the classroom, but so must the modes and applications of the engineering concepts within industry. College level engineering students must graduate not only learning the fundamentals of engineering concepts, but prepared to apply current technology as an engineer in industry in today’s society (Crawley et al. 1994).

Over two decades ago, the need for “a revision of the engineering education and a change of attitude toward mathematics and modeling” (p. 966) was highlighted by Moussavi (1998) as a necessary essential to keep engineering students competitive in the global market. Since then, researchers on mathematical modeling in engineering have found that engineers often work with existing models to interpret situations and to predict and design solutions (Alpers 2011; Bissell and Dillon 2000). In a recent effort, Faulkner et al. (2019) interviewed 27 engineering faculty members within 11 disciplines from a large United States research-intensive institution about their experience teaching core engineering courses and, specifically, the mathematical abilities of their students. Every single interview participant made at least one statement which the authors coded as “a mathematically mature student uses and interprets mathematical models” (p. 111) and at least one statement coded as “computational tools reshape ‘what needs to be known’ to be mathematically mature” (p. 111). The interviews from this study indicate that faculty believe that mathematical modeling skills are the most important skills for engineering students. However, the students hold dysfunctional, underdeveloped epistemic beliefs about mathematics that undermine their ability to develop these skills; in particular, the students believe that “mathematics is unrelated to the real world and has little practical value” (p. 128). The primary recommendation made by Faulkner and colleagues is twofold: not only is there a need for more modeling in the engineering curricula, but also a need to do this in the context of authentic or real-world problems. Or, as Crawley et al. (1994) stated, mathematical modeling is “a natural requirement of almost any engineering course” (p. 55) and it is “more effective and meaningful when integrated into existing subjects” (p. 53). This sentiment has been echoed by others as well, suggesting that the engineering curricula should be considered dependent on modeling (e.g., Bissell and Dillon 2000; Moussavi 1998).

Although engineering faculty members expect their students to be adept at both mathematics and mathematical modeling, students are entering core engineering curricula lacking the necessary skills (Faulkner et al. 2019; Ferguson 2012; Gainsburg 2006). At least part of the problem may be attributed to the fact that, according to existing literature, mathematicians and engineers have different perspectives on what constitutes “mathematical modeling” (Alpers 2017; Bissell and Dillon 2000; COMAP and SIAM 2019; Faulkner et al. 2019;). Leaders in the mathematical community collaborated to produce the Guidelines for Assessment and Instruction in Mathematical Modeling Education (GAIMME) report (COMAP and SIAM 2019). The intention of the report was to help teachers incorporate the practice of mathematical modeling into primary and secondary classrooms and beyond. The definition of mathematical modeling used in the GAIMME report is used repeatedly throughout educational literature (e.g. English 2009; Maiorca and Stohlmann 2016; Suh and Seshaiyer 2017): “Mathematical modeling is a process that uses mathematics to represent, analyze, make predictions or otherwise provide insight into real-world phenomena” (p. 12). Mathematical modeling in education has two primary purposes: to solve a specific problem, and to learn the skills of mathematical modeling (Stillman et al. 2013). The GAIMME report specifies the following skills or components of the mathematical modeling process:

  1. 1.

    Identify the Problem: Identify something in the real world we want to know, do, or understand. The result is a question in the real world.

  2. 2.

    Make Assumptions and Identify Variables: Select ‘objects’ that seem important in the real world. Question and identify relations between them. Decide what will be kept and what will be ignored about the objects and their interrelations. The result is an idealized version of the original question.

  3. 3.

    Do the Mathematics: Translate the idealized version into mathematical terms and obtain a mathematical formulation of the idealized question. This formulation is the model. Do the mathematics to see what insights and results we get.

  4. 4.

    Analyze and Assess the Solution: Consider: Does it address the problem? Does it make sense when translated back into the real world? Are the results practical, the answers reasonable, the consequences acceptable?

  5. 5.

    Iterate: Iterate the process as needed to refine and extend our model.

  6. 6.

    Implement the Model: For real world, practical applications, report the results to others and implement the solution. (COMAP and SIAM 2019 p. 12)

However, the full definition of mathematical modeling in the GAIMME report differs from the aspects of modeling to which the engineering faculty members interviewed by Faulkner et al. (2019) referred. Bissell and Dillon (2000) point out that practicing engineers are much more likely to need to select (component 2), apply (component 4), and possibly adapt an existing model than to create a model (component 3).

There are many mathematical models and software tools used in engineering. In the context of this module, students used the United States Army Corps of Engineers Hydrologic Engineering Center Hydrologic Modeling System (HEC-HMS) (USACE n.d.). HEC-HMS is engineering software applied to the mathematical modeling of hydrologic processes throughout the world to solve hydrologic design problems. Engineers also extensively use Excel spreadsheets for calculations and mathematical modeling in their work.

Theoretical Framework

We used Greefrath’s (2011) conception of mathematical modeling using digital tools to frame the creation of the HDC module as well as to examine students’ perceptions of mathematical modeling using technology. Greefrath describes how modeling using digital tools requires two translations: one translation of the real world situation into the mathematical world (i.e., components 1 and 2: identify the problem, and make assumptions and identify variables) and a second translation from a mathematical model to a computer model (Fig. 1). One benefit to using digital tools to solve mathematical models is that they enable students to solve models that they might not be able to solve by hand, or that are more efficient to solve using the tools (Greefrath 2011). In the case of engineering students, the integration of digital tools in coursework allows students to engage with tools used by engineers in the field. Additionally, digital tools can support students in understanding the real problem (i.e., component 1: identify the problem) as students can examine data sets, graphs, and even maps to better understand the context of the problem they are looking to model and solve. Given these benefits, Greefrath suggested integrating technology throughout the mathematical modeling process.

Fig. 1
figure 1

Modeling cycle. Note. Modeling cycle (Blum and Leiß 2007) with added computer model (Greefrath 2011, p. 302)

Full size image

The HDC module was designed for students to: (a) solve a specific problem (i.e., to use the fundamentals of rainfall-runoff processes and the engineering design process to design a detention basin to mitigate flash flooding); (b) develop the skills of modeling; and (c) model using publicly available datasets and free-to-use engineering modeling and analysis software. Specifically, they were asked to create a mathematical model for the hydrological processes in Dry Canyon in order to design a detention basin to protect against flash flooding. The model encodes hydrologic process knowledge that students learn by doing hand calculations and using spreadsheets (i.e., Excel), as well as through using engineering software (i.e., HEC-HMS). In the HDC module, students were given the problem, variables, and fundamental equations, but had to gather data and formulate the assumptions associated with alternative design choices in the context of and constrained by real world data in order to design a detention basin. Although students were given the necessary formulas (e.g., probability density function) and fundamental equations (e.g., Darcy’s Law), they were asked to do the algebraic computations themselves. The module focused on having students analyze and assess the solution, iterate the model in both Excel and HEC-HMS using different variables and assumptions, and implement the model (i.e., report the results), including reconciling Excel results and HEC-HMS results (Fig. 2 and Table 1).

Fig. 2
figure 2

HDC module sections defined within the engineering design process

Full size image
Table 1 HDC module activities defined within the mathematical modeling process
Full size table

This approach followed Greefrath’s suggestions by integrating technological tools to support mathematical modeling throughout the module (Table 1). Specifically, in the HDC module, students use an interactive map to help them better understand the geographic location and spatial context for the problem. They are guided through activities where they collect data from different online sources that also help them better understand the variables that impact the problem. These online sources are the same data sources as would be used by engineers solving this problem, so the students learn important data skills and the challenges of working with real data in an applied context. After doing the mathematics in an Excel spreadsheet, the students verify the mathematics using the HEC-HMS hydrologic engineering software. They use the results from these two digital tools to analyze and assess their solutions, iterating as needed (iterating is facilitated by the use of the digital tools), before presenting their final solution. In addition to integrating Greefrath’s framework into the design of HDC, we also used it to examine students’ perceptions of modeling in HDC. That is to say, we looked at their perceptions of mathematical modeling as expressed in comments from their use of HDC.

Methods

This study employed a qualitative design to understand how students across 5 years at two universities perceived mathematical modeling as implemented through the HDC module.

Context

From 2014 to 2018, the HDC module was implemented in undergraduate hydrology courses for engineering students at two institutions from different regions across the United States. The first implementation was delivered as a document. Subsequent implementations were delivered via the online user interface, accessible at https://hydroviz.org/Lessons/Index/UT/DryCanyonFFP. Although the course instructors at each institution used the module in similar academic settings, the methods of implementation varied. Some instructors used the module as an independent assignment to be completed by the students out of class; meaning, no class time was dedicated to the module, it was not discussed in class, and no class time was devoted to working on the module in a computer lab. For some instructors, the module was discussed in class and some class time was devoted to working on the activities in an on-campus computer lab. Some instructors assigned the module in its entirety, while others assigned one or more sections at a time.

After each implementation, students were asked to complete a survey outside of class time regarding their perspectives of the module. For the purposes of this study, we analyzed responses to four open-ended questions included in the survey (see Appendix 1). Using open-ended questions allowed for students to share their perceptions of HDC. Additionally, because these questions did not specifically prompt students to discuss mathematical modeling, any mention of aspects of mathematical modeling by the students served as evidence of their understanding of modeling. Although the overall modeling tasks remained the same within HDC, revisions were made to the module each year based on feedback from these surveys. The cycle of implement-assess-revise was performed for four iterations between 2014 and 2018. In all, 247 students participated in these surveys. Demographic data were not collected for the surveys. After the 2018 implementation, four students from the two institutions participated in interviews regarding their experiences. Three of these students were male and one was female. We chose to interview students in 2018 in order to gather more in-depth descriptions of the students’ perceptions of the module. Although the surveys allowed us to capture data from many students, we found their responses to be rather thin. Engaging fewer students in semi-structured interviews allowed us to gather rich data by probing students for more detailed responses (Patton 2002).

Module Case Study: Dry Canyon

The HDC module is a case study that addresses the problem of potential flooding from Logan Dry Canyon. In the HDC module, students use the engineering design process along with mathematical modeling to design a detention basin for use in protecting an area of urban development from flooding. In doing so, students create a mathematical model to simulate the hydrologic processes involved in detention basin design.

This module was created and revised across the years with three main goals: (a) to create a module situated within a real-world context, requiring students to use mathematical modeling to solve an authentic hydrology problem; (b) to take advantage of digital tools used by engineers to support students’ development of the mathematical and engineering skills needed in the workforce; and (c) to use student feedback in revisions to improve their learning experiences. Below we describe the module with regard to these three goals.

An Authentic Hydrology Problem Requiring Students to Model Mathematically

In August of 1977, a rather unusual rain event occurred near the city of Logan, Utah, which is located on the western edge of the Bear River Range, a northern extension of the Wasatch Range in the Rocky Mountains of the United States. Approximately 4 in. (100 mm) of rain fell in this area in less than 12 h, a rainfall record that has yet to be broken. As a result of this rain event, homes were flooded when a flash flood occurred in Dry Canyon, a small and rather steep valley without a perennial stream. However, despite this past flood, increased urban growth in the 1990’s and 2000’s resulted in multiple homes being constructed near the mouth of Dry Canyon. To protect these homes, a detention basin was constructed at the mouth of Dry Canyon in 2008 (Fig. 3). A large stormwater drain was also constructed to carry outflow from the detention basin to Logan River. The HDC module was developed as a case study of the design of the detention basin and stormwater drain at the mouth of the Dry Canyon watershed. There are trade-offs between the size of the detention basin and size of stormwater drain that need to be examined during the design. Mathematical modeling can be used to evaluate these trade-offs by determining the hydrologic processes that result in peak flow rates.

Fig. 3
figure 3

Images of the dry canyon detention basin. Note. Looking west out of the mouth of the Dry Canyon detention basin (main). Aerial view of the Dry Canyon detention basin (insert)

Full size image

Integrating Digital Tools Used by Engineers

The sections of the HDC module are organized in conjunction with the steps of the design process for the detention basin (Fig. 2). Sections 1–4 are dedicated to data acquisition and analysis by hand and with Excel spreadsheets, while Section 5 includes computer modeling using HEC-HMS (Fig. 4) and the analysis of computer modeling outputs. Each section of the HDC module introduces topics in hydrology that pertain to the design of a detention basin and includes one or more activities that walk students through the mathematical modeling process (Table 1). The activities were designed to mimic, as closely as possible, the process an engineer would go through to produce a hydrologic model for the purpose of designing a detention basin as well as teach the needed material just in time and in the context of this problem. To achieve this and also produce realistic results, publicly available data were used whenever possible from agencies such as the United States Geological Survey, National Climatic Data Center, and National Oceanic and Atmospheric Administration. In addition, students design their detention basin using HEC-HMS, which is the same modeling software used by the consulting firm that designed the actual detention basin constructed in Dry Canyon. Each activity in the HDC module is dependent on information and knowledge from previous module activities. For example, students delineate the boundary of the watershed in the first activity using StreamStats, an online tool accessed through the United States Geological Survey website. In this activity, students click on the outlet location in an online map and the tool delineates the watershed draining to that location. This reinforces for the students what a watershed is and its relationship to its outlet, without them needing to tediously draw lines based on map contours. The second activity walks students through obtaining precipitation data from the National Climate Data Center database for the area defined in the first activity. The students then use Excel to perform the mathematical operations necessary to transform the data into a usable form. These mathematical tasks include averaging from the 15 min intervals of the original data to 30 min, 1 h, 2 h, and so on, and identifying the annual maxima of these new intervals for use later in the statistical analysis of annual maximum precipitation intensity, duration, and frequency.

Fig. 4
figure 4

Flow diagram of the sections and activities within HDC. Note. The sections are represented by blue rectangles and the activities are the white bubbles. Dashed lines indicate components of the modeling process that are performed both by hand or in Excel and modeled in HEC-HMS

Full size image

Using Student Feedback to Make Revisions

Student survey results and instructor feedback helped guide the development and evolution of the HDC module. Several iterative modifications were made to (a) make the case study more authentic and real-world, (b) help students narrow their results to feasible outcomes in order to keep them on track with their detention basin design, (c) help students better analyze the data and synthesize their final results, and (d) avoid the use of black box style tools.

Make the Case Study More Authentic

In an effort to make the case study more authentic, detailed background and supporting text was added to each section of the module. In order to expose the students to real-world resources, students were required to work with publicly available datasets and generally accessible tools used by practicing engineers. For example, students obtain raw data from publicly available datasets and preprocess that data within module activities rather than them being given clean data to use.

Help Students Narrow their Results

Quizzes were added at important milestone points for students to check their results and progress and to provide them intermediate feedback on estimates or assumptions, with the expectation that this would keep students from obtaining incorrect results while working through the module. This helped keep the students on track with their design goals by having them verify that their calculation results were within a realistic range.

Help Students Synthesize their Final Results

Based on student feedback that doing the assignment was ‘easy’ but understanding the ‘what’ and ‘why’ was difficult, summary questions were added after each activity to help the students synthesize important results in the form of a written synopsis. The synopses are meant to help the students integrate relevant information from previous activities in order to see the big picture of where they currently are in the design and modeling process and where they are going with the final design of their detention basin.

Avoid the Use of Black Box Tools

Originally, students were expected to create their own spreadsheets. This proved too difficult and time consuming, so preprogrammed Excel spreadsheet templates were added along with screencasts for support in completing and understanding the calculations within the templates. However, these templates also proved too difficult for students to follow (too black box) and students still had difficulty grasping the meaning behind the calculations, even with the help of the screencasts. Therefore, the templates were replaced with other spreadsheets that are less prescriptive and require students to build their own functions. In addition, initially the spreadsheet calculations and HEC-HMS results were not consistent and results between the two were thus not comparable. Student feedback suggested that they were not understanding the connection between the hand calculations and Excel results and the HEC-HMS results. Therefore, the module was restructured to have results from Excel and HEC-HMS be comparable if done correctly. This served to connect the separate learning of the mathematics describing individual processes and the modeling components involving hand calculations and Excel spreadsheets (sections 1–4) with the capstone section (5) which is dedicated to modeling all the hydrologic processes together in HEC-HMS. Activities in the last section were modified to produce HEC-HMS model outputs that mimic, for the purposes of comparison, the outputs completed by hand calculation and Excel spreadsheet in previous activities.

Data Collection

In order to better understand student perceptions of mathematical modeling in the HDC module between 2014 and 2018, we collected survey and interview data.

Student Surveys

Following each implementation of the HDC module, students were asked to answer survey questions about their experience using the module. The student surveys were administered online using Google Forms. Students were allowed approximately one week to respond to the survey outside of class time and after completion of the module. Approximately 64% of students enrolled in the courses completed the survey (total number of surveys, n = 247). Results from the survey were then used to evaluate module content, make necessary revisions, and inform the design and evolution of the online student user interface. For the purposes of this study, we used the open-ended questions from the survey which allowed students to broadly share their perceptions of what they found beneficial or not about HDC, thus not all comments generated were applicable to modeling (see Appendix 1).

Student Interviews

The second author conducted four interviews with students, representing both universities, in the spring of 2018 after the semester had ended. All the interviews lasted between 21 and 29 min. She used a semi-structured interview approach, in which she asked all four participants the same questions but also used a variety of probing questions to gather more information based on their responses. The semi-structured interview questions are included in Appendix 2. These interviews were transcribed verbatim.

Data Analysis

All survey and interview data were organized by year and uploaded to Dedoose (SocioCultural Research Consultants 2017). The first two authors reviewed the survey and interview response data. In the first read through, comments describing student perceptions of mathematical modeling were separated from the rest of the comments. We considered Greefrath’s (2011) framework and the components of mathematical modeling described in the GAIMME report (COMAP and SIAM 2019) and included any comments that addressed making assumptions, gathering data, determining values for variables, analyzing and assessing the solution, iterating, or implementing the model in reality, mathematics or technology. Next, two a priori codes (Saldaña 2013), “beneficial” or “non beneficial,” were applied to generally describe students’ perceptions of mathematical modeling in HDC. Then open codes (Saldaña 2013) were applied to these excerpts. Once all excerpts were coded, we read through the excerpts within an open code to look for congruity (i.e., that all excerpts within a code seemed to represent that code in a similar way) and repetitive open codes were condensed. Examples of repetitive open codes that were condensed at this point in the data analysis include “designing the detention basin”, “the last section”, “the design part at the end”, and “thinking about the design”. Lastly, open codes were categorized into broader themes to describe students’ perceptions of mathematical modeling in HDC. Statistics in the form of percentages were calculated for each of these themes using the total number of excerpts for a theme divided by the total number of surveys recorded (n = 247), not the total number of surveys that mentioned mathematical modeling. Therefore, the percentages reported indicate the percent of students who mentioned each theme out of all participants, not out of the subset of participants who mentioned mathematical modeling, and so the numbers will necessarily be smaller than if they were calculated from the smaller set. Given that the surveys did not specifically ask about mathematical modeling, any mention of some aspect of mathematical modeling by a student without prompting is noteworthy.

Findings

We found, subject to the limitation that these are perceptions from not all students, that students perceived the use of mathematical modeling in HDC in four primary and intersecting ways: (a) HDC allowed students to be a part of the modeling process, (b) using technology in HDC helped students to understand what they were doing in mathematical modeling, (c) using technology in HDC helped students to develop their skill set, and (d) difficulties with the technology and/or modeling decisions they had to make in the modeling activities were in some cases barriers that interfered with students’ ability to learn. To preserve student voices, all quotes presented in the findings are in the students’ words, without correction for grammar, spelling, or punctuation.

HDC Allowed Students to Be Part of the Modeling Process

Various technologies allowed students to participate in the entire design and modeling process. Approximately 20% (n = 50) of the student surveys were coded as “Modeling - Beneficial - Being part of the process.” This means that, without being asked specifically about the modeling process in HDC, 20% of the students who responded to the survey mentioned that some aspect of the HDC module allowed them to be part of the modeling process in a beneficial way. Positive survey feedback included comments about being involved in a “real-world” project, a “case study,” or “real engineering” where the students could engage in the design and modeling process, “the whole process ... from start to finish” (student survey 2014). “I think the most beneficial part of this assignment was helping me realize how real engineering is done. It’s not all canned equations” (student survey 2016). Another student responded that a benefit of HDC was “Seeing how to take the given Data and how to use it. Finally real life work” (student survey 2017). These comments reflect the students’ perspectives of the entire HDC module and do not necessarily pertain to specific sections or activities within HDC or the students’ ability to learn hydrology or the modeling process. However, many of these comments also expressed that the culminating activities using HEC-HMS software pulled together the hydrology concepts they had learned in class. “HEC-HMS was very beneficial, it ... helped understand everything coming together and how each part was important” (student survey 2016). Another student commented: “I really felt the HEC-HMS portion of the analysis really pulled together my knowledge of hydrology” (student survey 2014). The last section of HDC wraps up the mathematical modeling process, the students implement their model, and complete their detention basin design.

Some students also made comments describing “ah-ha” moments. For example, one student from 2014 discussed how modeling a step of the design process in Excel allowed them to better understand the mathematics behind the process. Another student commented in 2014:

I did learn that i suck at mathematical modeling. i also learned about what a catchment would actually be. i learned about hydrologic system management and floodplain analysis. i also learned that i wouldn’t know where to begin trying to actually perform either of those. Someone was asking me questions about a neighborhood being constructed nearby and i couldn’t actually answer a single question. it was pretty embarrassing.

Although this student’s overall perception was that they “suck at mathematical modeling,” this comment reveals an understanding of using mathematical modeling and knowledge of principles of hydrology to consider the needs in the design process.

Using Technology in HDC Helped Students Understand Mathematical Modeling

Of the students surveyed, 17.4% (n = 43) indicated that the use of technology helped them understand the design and modeling process, interpret their results, and learn the hydrology material. Some students’ comments revealed their interest in being involved in a project that brought together multiple technologies: “Being able to see how all of these programs work with on one project was very interesting” (student survey 2014). Using technology also enabled students to explore various parameters and observe the resulting outputs at different points in the design and modeling process. “Changing the time lag / Curve Number for the different soil classes - helps you understand more deeply why you get the results you were given based off characteristics” (student survey 2016). The activities in the final section especially allowed them to see how everything from the HDC module came together in the design of their detention basin. “The last section I found to be the most beneficial because I was using everything I learned in the class and applying it to a program and I was able to have a visual on what was happening” (student survey 2015). Students were also encouraged to compare results amongst themselves. “I also liked comparing my results to other classmates’ results. Seeing how different soil types affected everything was kind of neat” (student survey 2015). Some also compared their results to the actual design of the constructed detention basin. One student commented that it was beneficial: “Using HEC-HMS for constructing hydrological Model and comparing different studies with figuring out the reasons for different results” (student survey 2014). Some students also seemed to appreciate having to use their critical thinking skills and make independent decisions. They were not handed the inputs and not only had to find the appropriate data but use their judgment when data were missing. “The portion of the assignment that gave me the least amount of confidence but that taught me the most was the interpretation of the data and the necessity of using our engineering judgment to decide what kind of flow we should design for” (student survey 2014). Using the technology in HDC to understand how different inputs affected the model was one of our main goals for students.

Using Technology Helped Students Develop their Skill Set

Exposure to current technologies, real-world applications, and gaining practical skills rather than simply classroom knowledge was an opportunity that 30% (n = 73) of the students surveyed thought was beneficial and would be an advantage they might have over other graduates. Some students’ comments suggested that they found the real-world context of HDC and the opportunity to use tools used by practicing engineers engaging. “I think that it as a tool was kind of cool to see that this is a utility used in real engineering” (student interview 2018). In this interview the student directly connected the HEC-HMS tool to their future work as a practicing engineer. Overwhelmingly, the positive feedback on technology was related to the students’ interest and excitement about learning a tool used by engineers in industry. However, some also learned the limitations of technology: “I learned a lot from the excel assignments … and that you can’t take everything for face value” (student interview 2018). This comment suggests that, although this student may have had difficulty with the spreadsheets, the student questioned the process and therefore the spreadsheet calculations rather than blindly accepting them as model output.

Gathering and assembling information through the use of databases and websites included in HDC, which are not typically taught or used in class, was also a noted benefit by some students. For instance, one student commented that “The most beneficial part of the assignment for me was using databases and other resources as this may help in the future in case I would need to look for information while doing a job” (student survey 2016). Similarly, another student mentioned:

I found using the websites to find information on the soil, characteristics of the basin, and find precipitation data for design storms to be the most instructive. In the book problems we learn to solve problems given the parameters but this gives us a chance to learn how to find the parameters in the first place, which can be difficult to do well. (student interview 2014).

This student compared the process of working through book problems to the HDC activities. Like so many other student comments, this student appreciated the opportunity to learn a new part of the problem-solving process, gathering the information and determining whether it is necessary or correct as opposed to using what is simply given in a book problem. Not only did these students recognize that using the web and other resources to gather model inputs and parameters was a new skill, they appreciated having to do it.

Difficulties with Technology and Modeling Decisions Interfered with Students’ Ability to Learn

Although many students noted the positive experiences they had with mathematical modeling in HDC, 36% (n = 89) of students indicated that they were not able to fully understand the modeling because of (a) glitches in the technology, (b) technology as a black box, or (c) they had to make estimates or assumptions but they did not fully understand the implications of those estimates.

Glitches in Technology

Approximately 10% (n = 25) of students mentioned broken web links within the HDC activity instructions. “Some of the web links were out of data and required figuring out where the required data had moved” (student survey 2014). Other students described their struggles becoming familiar with new programs. One student mentioned the least beneficial aspect of HDC was “the HMS part because i felt i was fighting with the program more than learning the material” (student survey 2016). A struggle that researchers and engineers face, but that was new to many students using HDC, is that retrieving data can be frustrating and time consuming. “Waiting for days to receive the NOAA files needed to come via email was kind of ridiculous. There’s got to be a better way to make those available” (student survey 2017). Most of these comments highlight hurdles that are common in the real world but that become easier with more experience.

Technology as a Black Box

One idea that emerged from 21% (n = 52) of students was that technology acted as a black box, suggesting that the Excel spreadsheets, which calculated the necessary values with pre-programmed formulas, did not clarify the mathematical calculations for them. Some students focused more on inputting the right numbers than on considering the output of the model. “Sometimes hard to understand what we were doing… Since the spreadsheets did all calculations, it was hard to understand what actually just happened in the spreadsheet” (student survey 2016). Some students expressed that they did not understand what was happening in the spreadsheet, even though the spreadsheet directly aligned with the model calculations. “I can Google and YouTube what I need to do in Excel, it was just trying to understand what the question was and how it translated into Excel” (student interview 2018). Although there were some complaints about the activity instructions lacking enough detail, a surprising number of students suggested there was too much guidance in the instructions yet not enough clarity in helping them understand the results. “Overall, the directions were very clear. One could easily do the assignment, just understanding what happened seemed difficult...” (student survey 2016). “I’m fairly comfortable using Excel. I just felt like a lot of times it was follow a pretty well-defined instruction, procedure set, and punch data in. So it just felt like maybe the material was kind of taking a back seat to Excel’s functionality…” (student interview 2018). The digital tool that some students felt supported their understanding of the mathematical model the least were the Excel spreadsheets.

Estimates and Assumptions

About 15% (n = 38) of the students expressed frustration with their lack of experience with the processes and activities like those in HDC and therefore doubted their judgment and assumptions, which made their estimates feel like guesswork. “The only thing that was somewhat confusing was the estimated values that was allowed. I felt as if we had too much choice in what values we could use” (student survey 2014). “It was difficult not knowing the best answer. Many times I felt like I was just making assumptions…” (student survey 2016). Some students also found it frustrating when their results differed from other classmates’ or from the professional design of the basin constructed at the mouth of the canyon. “I had a hard time when I got drastically different results from the professional analyses. I know they have a lot more details, but it’s hard to be confident about your work when it’s so far off” (student survey 2014).

Discussion

This case study of the flood detention basin at the mouth of Dry Canyon provided an opportunity to both update the teaching approach in undergraduate engineering hydrology courses and integrate technology in the courses (Baher 1998; Brogan et al. 2016; Kollöffel and de Jong 2013; Koretsky et al. 2011; NSF 2008). It also proved to be an opportunity to put effort toward reform in hydrologic education, especially as is needed at the undergraduate level (CUAHSI 2010). The result is the HDC module, an online hydrology education module centered on case-based, data- and simulation-driven learning as described earlier. The HDC module is a tool that continues to be used at multiple institutions and has served as a foundation for the development of other similar learning modules. The findings from this study highlight more than just what worked or did not work with the HDC module. These findings can inform future teaching and learning innovations that seek to combine mathematical modeling and engineering content: (a) situating mathematical modeling problems within real-world contexts and presenting students with authentic engineering problems aids students in learning new content; (b) incorporating digital tools used by engineers can help students be a part of the mathematical modeling process; and (c) instructors should take student feedback into account in revisions to curriculum materials.

In developing the HDC module, we adopted an improvement-focused evaluation model (Posavac and Carey 2003) with an iterative design process that included multiple cycles of implement, assess, and revise. Once the original module was designed, it was tested in classes at two universities. The first implementation and each one thereafter were successful in the sense that the students were able to complete the module, gain knowledge in engineering hydrology, give feedback on their experience using the module, and we were then able to assess and revise the module so that a newer and more refined version could be used in the next implementation.

The instructors for each course in which the module was implemented used the module differently, which was helpful in testing the module’s flexibility but complicated the process of assessing year-to-year changes. Even when the same instructor used the module more than once, it was used differently each subsequent time. Although it would have been helpful to include a wider variety of students and courses in the implementations, the variety we encountered proved difficult enough to assess and then respond to in revisions. However, it was helpful to include students in the implementation from universities including those unfamiliar with Dry Canyon and the Logan area and who were also taught by instructors not associated with the design team for the module.

The responses to the open-ended questions were quite informative and therefore had a large impact on what changes were made to the module over time. The evolution of the module was essentially driven by the responses to these open-ended survey questions and various comments made by students, both in person and within the survey, along with the work turned in by the students and instructor feedback. In hindsight, it may have been more helpful to observe students while using the module (e.g., in the classroom, the computer lab, or while working in groups) and also interview them while they were working through it rather than after they completed it and moved on to the next course subject. It likely would have been most helpful to have a comparison group that did not use the module at all and compare what and how they were able to learn with the students who used the HDC module (Creswell and Guetterman 2019). However, that was not a feasible option for the courses taught at the institutions involved. Therefore, rating the effectiveness of the module is subjective and comes down to whether or not the instructor feels it worked for them: their curriculum, their teaching methods, and their students.

Our findings indicated that students who participated in HDC said that the digital tools allowed them to be a part of the mathematical modeling process, helped them understand what they were doing in mathematical modeling, and/or helped them develop their skill set. These types of positive interactions between the learner and the content are important in changing the students’ perspectives and possibly their beliefs about mathematical modeling (Gosmire et al. 2009). However, it is important to note that this study did not examine students’ engagement, beliefs, affect, or motivation. Rather, we asked students to describe their experiences with the HDC module. In many cases they were recounting their experiences more so than describing their engagement. Nevertheless, we viewed their perspectives as important insight into the ways in which they interacted with HDC which could possibly shed light on their beliefs. Many students also felt that the technology and/or modeling decisions they had to make in HDC interfered with their ability to learn. The first finding, that HDC allowed students to be a part of the modeling process, is in alignment with Greefrath’s (2011) description of modeling using digital tools, whereby digital tools can help students in every component of the mathematical modeling process. This finding highlights the importance of using digital tools not only to do the mathematics, but throughout the modeling process in engineering.

Incorporating digital tools and technology into engineering curricula not only addresses many of the desired changes in hydrology and water resources engineering education, but also aids students in understanding processes and learning new material. The HDC module includes authentic context and real-world, problem-based activities, as recommended by Faulkner et al. (2019). Student survey feedback supporting both our first and second findings suggest that the familiarity of the HDC context and the case-study perspective of the problem allowed students to be a part of the process and better understand the mathematical modeling involved. Some student comments further support that mathematical modeling is more effective and meaningful to the student when they are familiar with the context (Crawley et al. 1994) yet are pushed to use their critical thinking when solving problems within the mathematical modeling process. Moreover, these findings regarding student perspectives shed insight into their interactions with the module and informed revisions we made to the HDC module.

Both Crawley et al. (1994) and Moussavi (1998) pointed out that mathematical modeling is a necessary component for engineering students to remain competitive in a global market and, as graduates, these students should be prepared to apply current technology as an engineer in industry. In addition, engineers often work with existing models to interpret situations and to predict and design solutions (Alpers 2011; Bissell and Dillon 2000). The HDC module exposes students to real-world applications and allows them to blend practical skills along with classroom knowledge. Survey feedback shows that not only are these modeling skills valued by the students, but so is the exposure to additional digital tools, such as HEC-HMS, Excel, online databases, and other web-based tools.

However, some students also felt that difficulties with technology and/or modeling decisions they had to make, but where they did not feel prepared or had insufficient information, interfered with their ability to learn. Technical glitches can create problems when using digital tools and technology in learning (Hill 2002; Song et al. 2004). Student feedback included complaints of a variety of technical glitches with HDC, such as out of date web links, slow websites, as well as an inability to find the correct button or model input. Most technical glitches are avoidable but require constant maintenance, which can be a hurdle for the instructor. Another technical hazard highlighted by student survey feedback was that technology sometimes acted as a black box, that the HDC activities sometimes felt too plug-n-chug, or that students obtained answers that seemed correct but that they did not understand the process behind calculating the answer. Some of these complaints may be attributable to students’ experience, or lack thereof, with a tool or process, which influences the students’ perception of its usefulness and contribution to overall learning (Clarke III et al. 2001). However, as mentioned previously, Merwade and Ruddell (2012) cautioned against creating these types of “black box” tools which would make the concepts and modeling less apparent. Based on these findings and feedback from students, we revised HDC to replace the pre-programmed Excel templates with other sheets that are less hard-wired and allow students to build their own functions. We also added more checking-in quizzes to provide students with intermediate feedback on estimates or assumptions they make during the modeling process to assist the student in avoiding the accumulation of erroneous decisions or answers. Additionally, other real-world resources were integrated, including a watershed delineation web tool.

We do not have a direct assessment of how helpful the HDC module was in supporting student learning, as anonymous surveys could not be cross-referenced with student performance in the module. However, the general appreciation for working with authentic tools on a real world problem expressed by many of the students suggests that the innovations we made to this module were beneficial. The comments by students about their struggles with assumptions and their implications do need to be addressed though. It would be better for students to leave this module with greater confidence in their ability to execute this sort of mathematical modeling based design in the real world, than is evident in their responses. This could be addressed by providing a bit more guidance at places in the module where the students had trouble, to limit their confidence being undermined. Students’ general areas of difficulty can be identified when assessing their work and addressed during classroom lecture. Nevertheless, capturing student perceptions on this, through questions asking students to identify areas of difficulty may be a useful addition to future student surveys.

Limitations

The limitations of this study are attributable to data collection methods and also revisions within the HDC module. Two different types of data were collected over the 5 years of this study: (a) student survey data in 2014–2017 and (b) student interview data in 2018. Although the line of questioning was similar between these two formats, the data are not completely comparable. In addition, response rates for the student survey changed from year to year as participation requirements changed; some instructors required that students participate in the survey while other instructors allowed participation to be completely voluntary. Each year, the HDC module content and interface was revised based on feedback from students. In particular, additional Excel supports were added each year from 2014 to 2018. This was responsive teaching but also means that technology was integrated to support mathematical modeling differently across the years.

Implications

Our findings are in line with others’ (Baher 1998; Brogan et al. 2016; Greefrath 2011; Kollöffel and de Jong 2013; Koretsky et al. 2011), that technology has the potential to enhance students’ understanding of and skills with mathematical modeling. Engineering course instructors should consider integrating technology, especially tools used by practicing engineers, such as HEC-HMS and spreadsheet work, into mathematical modeling problems in their courses. The HDC module erred in some ways, creating complex spreadsheets that were perceived as black boxes, and which resulted in students processing the spreadsheets mechanically rather than being pushed to understand the calculations within the process. Instructors should strive to balance the detail provided in packaged models, like preconstructed spreadsheets, with having students construct their own spreadsheets and/or calculating their own solutions. Similarly, instructors should be sure to frame questions on the output that technology is producing to push students to think critically and interpret the outputs, not just report them and move on.

Our findings are also consistent with literature that real world authentic problems enhance student engagement that benefits learning (Freeman 2014; Habib and Deshotel 2018; Herrington and Herrington 2007). The student perception findings and positive feedback with respect to the real world case study aspect of HDC support this. The HDC module was framed to help students think critically about the links between how hydrologic processes work and real world ramifications. They learned and modeled hydrologic processes just in time, motivated by this real world context, and were stretched in their critical thinking skills to put their knowledge together in synthesizing a design. Some of the comments they made about challenges and uncertainties reflect that, on occasion, they perceived being stretched too far. It is our perception, as engineering educators, that this is good, but developing ways to evaluate this, perhaps using a survey of perceptions over time that quantified this, would be beneficial.

This study examined engineering students’ perceptions of mathematical modeling incorporated as part of an online module that embedded technological tools. Future research should examine how students interact with modeling by collecting data throughout such a module, including student work and interviews. Moreover, future research could assess students’ ability to engage in mathematical modeling for students who participated in such a module and for students who did not, to see if the skills learned transfer to a new situation.

Although engineers typically graduate with strong computational skills, mathematical modeling skills are still sometimes lacking. And yet these skills are equally as important, both within students’ engineering coursework and also their competitiveness once entering the job market. Multiple engineering education researchers have stated that mathematical modeling should be more central to engineering curricula (Crawley et al. 1994; Faulkner et al. 2019; Gainsburg 2006). Further supporting this, based on our findings, students’ perceptions of mathematical modeling are mostly positive. By including more opportunities for students to engage in mathematical modeling in their coursework, not only will students’ understanding of mathematical modeling change, they will become more familiar and comfortable with the modeling process and essential technology and gain the necessary associated critical thinking skills.

As engineering instructors consider course design, and especially as they consider creating online modules, the teaching and learning innovations we employed in the HDC module, and the findings from this study, can inform their work. Instructors should consider: (a) situating real-world problems in authentic contexts requiring students to model mathematically to find a solution, (b) integrating technologies and digital tools used by engineers to model mathematically and solve engineering problems, and (c) using an improvement-focused evaluation model to take student consideration into account when revising their module.