1 Introduction

With the ongoing rapid urbanisation and climate change, flood risk is expected to significantly increase in the twenty-first century and beyond (IPCC 2014). We will see nearly 40% of the global urban cities located in flood-prone zones by 2030 (Güneralp et al. 2015). Actually, flooding has already been reported to cause significantly increased societal and economic impact in the recent decades across the globe, especially in the urbanised areas with dense population and properties. In China, wide-spreading flash flooding has led to devastating consequences and affected sustainable development in most of the cities, which have experienced rapid expansion in the last few decades. For example, in July 2012, Beijing was hit by the most severe flood event in the last 61 years, killed 79 people, caused at least 10 billion Yuan RMB (~ USD 1.8 billion) of damages and destroyed at least 8200 homes. In September 2016, Typhoon Meranti and Typhoon Megi landed in Fujian Province and brought in record-breaking rainfall that caused severe flooding in Xiamen, Fuzhou and the surrounding cities and towns. The two sequential typhoons and the subsequent flood events have directly caused 18.7 billion Yuan RMB (USD 2.9 billion) of damage and affected over 2.47 million people, according to the local government statistics.

Numerical modelling provides an effective means to understand the dynamic process of flooding (Abderrezzak et al. 2009) and has become an indispensable tool to support flood risk assessment and management (Ernst et al. 2010; Dawson et al. 2011; Coles et al. 2017). In the last two decades, with the development of more efficient numerical methods and the availability of more powerful computing facilities, two-dimensional (2D) simplified or sophisticated hydrodynamic/hydraulic models have been increasingly applied in flood modelling, including the use of those hydrodynamic models developed by using mesh-free methods (e.g. Liang et al. 2015; Pu et al. 2013). When applying these models to practical urban flood simulations involving complex urban geometries, it has been widely recognised that the resolution and quality of the topographic data may have a significant influence on the simulation results (Zoppou and Roberts 1999; Gourbesville and Savioli 2002). Schubert et al. (2008) applied a hydrodynamic model to a 0.4 km2 localised urban domain in Glasgow, UK, using an unstructured mesh of 1–5 m resolution to demonstrate the importance of performing flood modelling at high spatial resolution. Gallegos et al. (2009) conducted dam-break flood simulations in a dense urban residential area in Southern California and recommended that 5 m or higher spatial resolution is necessary in 2D urban flood modelling in order to properly represent the small gaps between buildings and produce reliable simulation results. Hénonin et al. (2015) reproduced the severe July 21st 2012 flood in Beijing, China, using a multi-grid-based hydrodynamic model in an urban domain of ~ 900 km2 and concluded that low-resolution simulations could not properly represent the complex urban geometrics and topographic features to provide reliable flood predictions. Yin et al. (2016) integrated a storm surge model with a simplified inundation model to reproduce the urban flooding induced by Hurricane Sandy in part of the New York City. More recently, Leitão and de Sousa (2018) attempted to simulate flood dynamics at a spatial resolution of 0.5 m in a 0.9 km2 semi-urban catchment in Switzerland to confirm the importance of using accurate high-resolution terrain data in urban flood modelling. Generally, various researchers have concluded that high-resolution simulation is necessary to accurately interpret urban geometrics and topography to subsequently provide reliable urban flood predictions.

In order to perform high-resolution urban flood modelling under acceptable computational costs, a common practice is to restrict the simulation domain in a small area that only covers part of an urban catchment or city. However, the hydrological processes during a flood event are highly connected across the entire city due to the existence of complex drainage, street and river networks in the urban environments. Simulations on localised domains will inevitably introduce unnatural domain boundaries and cut off the dynamic links between the flood flows inside and outside of these boundaries, leading to unreliable predictions.

In the recent years, with the rapid development of new parallel computing technologies, it is now technologically feasible to perform high-resolution (e.g. ~ 1–5 m) urban flood modelling across an entire city at an affordable computational cost. Parallel computing techniques including Message Passing Interface (MPI), Open Multi-Processing (OpenMP) and graphics processing unit (GPU) high-performance computing have been reported to substantially accelerate simplified or full hydraulic/hydrodynamic models for large-scale flood modelling (e.g. Neal et al. 2010; Sanders et al. 2010; Castro et al. 2011; Smith and Liang 2013; Vacondio et al. 2016). For example, Sanders et al. (2010) presented the MPI parallelised Godunov-type 2D shallow water model ParBreZo for application in simulating large-scale urban dam-break flood and inundation caused by a hurricane-induced storm surge. Noh et al. (2018) developed and applied a hybrid parallel hydrodynamic model using OpenMP and MPI to simulate storm water flow in the Johnson Creek Catchment (40 km2) in Arlington, Texas, at 1 m resolution. The hybrid parallel code was reported to be 79 times more efficient than the same non-parallelised hydrodynamic model, depending on the hardware being used. Smith et al. (2015) applied the GPU-accelerated hydrodynamic model developed by Smith and Liang (2013) using the OpenCL programming framework to reproduce the January 2005 flood in Carlisle, UK, and compared the model performance on different hardware devices. They confirmed that a high-spec GPU could speed up the same simulation for more than 20 times in comparison with a multi-CPU server. Vacondio et al. (2016) developed a 2D shallow water model using the CUDA GPU programming framework to provide accurate and fast simulation of flooding induced by levee breaches in large domains. Attempts have also been reported to achieve superior computational speed using multi-GPU parallelisation (e.g. Sætra and Brodtkorb 2012). These GPU-accelerated hydrodynamic models have provided effective tools to support large-scale high-resolution flood modelling at an unprecedented computational speed (Liang et al. 2016). However, no attempt has been made to apply this latest hydrodynamic modelling technology to support flood simulation across an entire city and systematically investigate the effects of spatial resolution and scale on the simulation results.

In this paper, a multi-GPU high-performance hydrodynamic model is improved and tested for high-resolution simulation of a severe urban flash flood event caused by typhoon-induced intense rainfall in a 267.4 km2 urban catchment. The model is further used to systematically investigate and analyse the effects of spatial resolution and using localised simulations (scale) on flood predictions. The rest of the paper is organised as follows: Sect. 2 provides brief introduction of the model and relevant components; Sect. 3 introduces the case study site and data for setting up the model; Sect. 4 validates the model and analyses the effects of resolution and scale; and finally, brief conclusions are drawn in Sect. 5.

2 Model description

The current hydrodynamic model (i.e. High-Performance Integrated hydrodynamic Modelling System or HiPIMS in short) solves the full 2D shallow water equations (SWEs), taking into account other relevant hydrology processes such as infiltration and loss through drainage systems. The governing SWEs are given in a matrix form as

$$\frac{{\partial {\mathbf{q}}}}{\partial t} + \frac{{\partial {\mathbf{f}}}}{\partial x} + \frac{{\partial {\mathbf{g}}}}{\partial y} = {\mathbf{R}} + {\mathbf{S}}_{{\mathbf{b}}} + {\mathbf{S}}_{{\mathbf{f}}}$$
(1)

where t is time, x and y are the Cartesian coordinates, q is the vector containing the conserved flow variables, f and g are the flux vector terms in the two Cartesian directions, and R, Sb and Sf contain the source terms representing the effects of rainfall, bed slope and bed friction. The vector terms are, respectively, given by

$$\begin{aligned} {\mathbf{q}} & = \left[ {h,uh,vh} \right]^{\text{T}} ,\quad {\mathbf{f}} = \left[ {uh,u^{2} h + \frac{1}{2}gh^{2} ,uvh} \right]^{\text{T}} ,\quad {\mathbf{g}} = \left[ {vh,uvh,v^{2} h + \frac{1}{2}gh^{2} } \right]^{\text{T}} \\ {\mathbf{R}} & = \left[ {R - D,0,0} \right]^{\text{T}} ,\quad {\mathbf{S}}_{{\mathbf{b}}} = \left[ {0, - gh\frac{\partial b}{\partial x}, - gh\frac{\partial b}{\partial y}} \right]^{\text{T}} ,\quad {\mathbf{S}}_{{\mathbf{f}}} = \left[ {0, - \frac{{\tau_{bx} }}{\rho }, - \frac{{\tau_{by} }}{\rho }} \right]^{\text{T}} \\ \end{aligned}$$
(2)

where u and v are the two depth-averaged velocity components, h is the total water depth, b represents the bed elevation above datum, g is the acceleration due to gravity, R is the effective rainfall or runoff, D is the drainage loss, ρ is the water density, and τbx and τby are the friction stresses calculated using

$$\tau_{bx} = \rho C_{f} u\sqrt {u^{2} + v^{2} } \quad \tau_{by} = \rho C_{f} v\sqrt {u^{2} + v^{2} }$$
(3)

where Cf = gn2/h1/3 is the roughness coefficient with n being the Manning coefficient.

The above SWEs are solved using a finite volume Godunov-type scheme incorporated with an HLLC approximate Riemann solver to calculate interface fluxes, and the detailed model implementation may be found in Liang and Borthwick (2009). The finite volume time-marching scheme is given as

$${\mathbf{q}}^{n + 1} = {\mathbf{q}}^{n} - \frac{\Delta t}{{\varOmega_{i} }}\sum\limits_{k = 1}^{N} {{\mathbf{F}}_{k} } \left( {{\mathbf{q}}^{n} } \right)l_{k} + \Delta t\left( {{\mathbf{R}}_{i}^{n} + {\mathbf{S}}_{bi}^{n} + {\mathbf{S}}_{fi}^{n + 1} } \right)$$
(4)

where n is the time level, ∆t is the time step, i is the cell index, k is the index of the cell edges (N = 4 for Cartesian grids), lk is the length of cell edge k, Ωi is the cell area, and Fk(q) =  fk(q)·nx+ gx(q)·ny contains the fluxes normal to the cell boundary with n  = (nx+ ny) defining the outward normal direction. Herein, the surface reconstruction method (SRM) and implicit friction discretisation scheme reported in Xia et al. (2017) and Xia and Liang (2018) are also implemented to ensure stable and accurate simulation of overland flows with small water depth. The adopted numerical schemes strictly preserve the lake at rest condition (i.e. C-property) and ensure non-negative water depth for applications involving wetting and drying over irregular bed topography (see, Xia et al. 2017 and Xia and Liang 2018 for detailed implementation).

Urbanisation creates substantial impermeable area through construction of roads, buildings and other urban structures and subsequently increases runoff during a rainfall event. However, there still exist permeable areas such as gardens, grass fields and woods where infiltration effects cannot be negligible in most of the situations. In HiPIMS, the infiltration effect is taken into account using the Green–Ampt model (Green and Ampt 1911) which was derived based on Darcy’s law and the principle of mass conservation. The infiltration rate is calculated using the following formula

$$i = K_{s} \left( {\frac{{h_{0} + h_{f} + z_{f} }}{{z_{f} }}} \right)\frac{{\partial^{2} \varOmega }}{{\partial v^{2} }}$$
(5)

where i is the infiltration rate (m/s), Ks is hydraulic conductivity (m/s), h0 is the ponding water head on the ground surface, hf is the wetting front metric potential, and zf is wetting front depth. The effective parameters for the Green–Ampt infiltration model include soil properties and antecedent water-content conditions, which may be specified for different geological conditions with values suggested in the literatures (e.g. Brakensiek and Onstad 1977). The calculated infiltration rate will be deduced from the initial rainfall rate to obtain the effective rainfall or runoff in each cell.

In urban flood modelling, flow through the drainage networks may have strong influence on the simulation results and should be also considered. However, for large cities, detailed information about drainage network is normally not available or inaccurate. In this work, a simple approach is adopted to consider the drainage effect during a simulation. The drainage collection points (assumed to be manholes in this work) are distributed along roads and streets at certain intervals according to design regulations. Their locations are assumed according to design regulations and corrected and confirmed using GIS maps. The manholes and their locations can be also efficiently identified from street-view images (when available) using the Tencent Map API, which are then used to verify the manhole distribution scheme worked out using design regulations. The sewer loss through these storm water collection points is then decided according to the reference design capacity of the drainage systems, which becomes a sink term (D) in the current hydrodynamic model.

In order to substantially improve its computational speed for large-scale applications, HiPIMS is implemented for GPU high-performance computing using the NVIDIA CUDA programming framework (Amouzgar et al.2016; Xia and Liang 2016). Furthermore, parallel computation on multiple GPUs is achieved using a domain decomposition approach (Sætra and Brodtkorb 2012).

3 Case study

3.1 Study site

Fuzhou, the capital city of Fujian Province in the Southeast China, as shown in Fig. 1, is chosen to be the case study site for this work. Fuzhou is a coastal city with its eastern side opened to the sea and other three sides surrounded by mountains. With the Min River cutting through the heart of the City, Fuzhou is vulnerable to flooding from multiple sources, i.e. fluvial flooding, pluvial flooding and coastal flooding. In September 2016, two typhoons successively landed in Fujian Province, leading to severe floods in a number of cities including Fuzhou. The powerful Typhoon Megi (known as Typhoon Helen in the Philippines) directly brought heavy rainfall of over 250 mm in a few hours and led to severe urban flooding in Fuzhou. Figure 2 shows the rainfall history and accumulated rainfall recorded in a local gauge station. Figure 3 shows the tidal level at the same time period in which the high tide actually coincided with maximum rainfall. However, no overtopping or riverbank breaching was reported in the Min River during the event. Coastal flooding was therefore not relevant, and the urban flooding in Fuzhou was mainly due to the typhoon-induced intense rainfall. The inundation depth in certain areas in the old city centre (north bank of Min River) was reported to exceed 1 m. Many roads, homes, schools and commercial buildings were inundated and closed down, and the flooding also created chaos to the traffic systems. According to the local government statistics, Typhoon Megi and the subsequent floods have caused 1.6 billion Yuan RMB (USD 0.25 billion) of direct economic loss and affected over 0.67 million people in different parts of the city.

Fig. 1
figure 1

Location of the study site, Fuzhou, China (a), and the computational domain, i.e. the old town centre, reproduced from the adopted 2 m DEM (b). The rainfall gauge stations and locations with surveyed flood depth are marked by circles and ‘X’, respectively

Full size image
Fig. 2
figure 2

Time series of hourly and accumulated rainfall during Typhoon Megi

Full size image
Fig. 3
figure 3

Tidal level at the Min River during Typhoon Megi

Full size image

To reproduce the Typhoon Megi floods, the computational domain is chosen to be the most affected old town centre of Fuzhou. As shown in Fig. 1b, the 267.4 km2 computational domain is surrounded by mountains at the west, north and east and the Min River to the south, with the low-lying city centre in the middle. The computational domain is essentially an enclosed catchment with natural boundaries. Inside the domain, there are over 60 rivers of different size, leading to complicated river systems.

3.2 Data and model setup

The data available to support the proposed urban flood simulations include high-resolution digital elevation model (DEM), land-use map, rainfall measurements, tidal level and measured flood depth. The land-use map provides information for the selection of model parameters including Manning coefficients, infiltration parameters and drainage capacity. The rainfall measurements and tidal level define the boundary conditions to drive the simulations. The available measured flood depth and crowd-sourced data are used to validate the simulation results.

  • DEM

The original bare earth terrain data available for the current study have a spatial resolution of 2 m and vertical accuracy of 0.4 m, as shown in Fig. 1. In order to better represent the urban built environments that have great influence on the flood simulation results, buildings are reinserted onto bare earth terrain data to develop the final 2 m DEM. Since the building height does not have major effect on the simulation, all of the buildings are assumed to be 10 m in height in the improved DEM, as shown in Fig. 4. The high-resolution DEM is able to represent the sizable channels and complex river networks in the domain. The hydraulic correction method (HCM) (Chen et al. 2018) is applied to further improve the DEM to ensure flow connectivity in the channels.

Fig. 4
figure 4

The zoom-in DEM with buildings

Full size image

  • Land-use map

From the land-use map as displayed in Fig. 5, the computational domain may be classified into seven different land-use types, including buildings, roads, open built up spaces (e.g. plazas and car parks), green fields, bare grounds, mountain areas and water bodies. Based on the typical values as suggested in the literature (e.g. Arcement and Schneider 1989), the Manning coefficient is valued according to the land-used types: (1) 0.02 for roads, plazas and parking spaces; (2) 0.05 for buildings; (3) 0.08 for mountain areas and green fields; (4) 0.035 for bare grounds and water bodies. The parameters for estimating infiltration rate using the Green–Ampt model include the hydraulic conductivity, wetting front metric potential and wetting front depth. These parameters are obtained according to the main soil type extracted from land-use covers (e.g. green fields, bare grounds and mountain areas) (Brakensiek and Onstad 1977; Rawls et al. 1983). For the drainage loss, water is removed from the computational cells containing the drainage points detected using the approach introduced in Sect. 2, at a rate of 13.2 mm/h according to Code for Design of Outdoor Wastewater Engineering (GB 50014-2006) adopted in Fuzhou.

Fig. 5
figure 5

The land-use patterns in the simulation domain

Full size image

  • Rainfall

There are 23 rain gauges installed for collecting rainfall data in the domain, as indicated in both Fig. 1 and Fig. 6. More than half of these rain gauges are distributed in the urbanised area. Thiessen polygons are created from the gauge locations using a GIS tool. Figure 6 illustrates the generated Thiessen polygons marked with the maximum hourly rainfall measured at each of the individual gauges during Typhoon Megi. The time series of hourly rainfall during the event from all the gauge stations are plotted in Fig. 7. Intense rainfall mainly occurred from 01:00 to 05:00 on 28th September and the total duration of the event is 28 h (from 15:00 on 27th to 19:00 on 28th September 2016).

Fig. 6
figure 6

The spatial maximum hourly rainfall (mm/h) during Typhoon Megi, marked with the locations of the rain gauges

Full size image
Fig. 7
figure 7

Time series of hourly rainfall measured at different gauge stations during Typhoon Megi

Full size image

  • Tidal level

As shown in Fig. 1, a number of river channels inside the computational domain are connected to the Min River that is under influence of the tidal cycles of the East China Sea. During Typhoon Megi, high water levels were recorded in the Min River and the high tide coincided with the rainfall peak, as shown in Figs. 2 and 3. In the current simulations, the recorded tidal levels are used as boundary conditions in the hydrodynamic model. However, the tidal boundary conditions have minimal effects on the inundation as no breaching or overtopping of river banks was reported during the event.

  • Measured flood depth and crowd-sourced data

The local government agencies managed to record flood depth at 368 inundation points during the event (with the locations indicated by ‘X’ in Fig. 1). Most of the surveyed points are located at the low-lying urbanised areas where severe inundation occurred. The measured depths provided valuable information to validate the simulation results.

Additionally, complementary crowd-sourced data from online social media were also collected and used to further validate the modelling results. An imbedded social media searching engine system was used to search and collect messages and images (filtered through keywords) for the specific flood event. The collected information was then sorted and analysed for use according to time and geo-spatial information.

4 Results and discussion

In this section, HiPIMS is used to reproduce the severe urban flood event induced by Typhoon Megi in the 267.4 km2 domain of Fuzhou City. After being validated by comparing with measured data, the model is used to run further simulations to investigate the effects of spatial resolution and using a localised domain on flood prediction. All of the simulations are performed on a server computer equipped with Intel® Xeon® E5-2650 v3 processor (two Core, 2.3 GHz, 16 GB DDR4) and 8 × NVIDIA Tesla K80 GPU devices (note that each Tesla K80 GPU contains two Kepler GK210 GPUs).

Root-mean-squared error (RMSE) and Fit statistics (F) are utilised to compare and assess the simulation results. To compare the simulated water depth and field measurements at a point-by-point basis, the RMSE is defined as follows:

$$RMSE = \sqrt {\frac{1}{N}\sum\limits_{i = 1}^{N} {\left( {S_{i} - O_{i} } \right)^{2} } }$$
(6)

where Si and Oi, respectively, represent the simulated and observed water depth/level, and N is the number of observations available for comparison.

To quantify how well the simulated flood extent matches the measured extent or reference solution, the F-statistics is used and can be calculated as follows:

$$F\left( \% \right) = \frac{A - B}{A + B + C} \times 100$$
(7)

where A is the number of correctly predicted wet/flooded cells, B is the number of cells erroneously predicted as wet/flooded, and C is the number of cells erroneously predicted as dry (Horritt et al. 2010).

4.1 Baseline simulation and model validation

The baseline simulation to reproduce the Typhoon Megi floods in Fuzhou is carried out on the 2 m DEM for the whole 267.4 km2 domain, involving 66.92 million effective computational cells/nodes. Driven by the spatially varying rainfall measured at the gauge stations and tidal boundary conditions, the simulation is run for 28 h between 15:00 on 27th and 19:00 on 28th September. Figure 8 shows the resulting inundation maps at different output times. The maps only mark the locations with flood depth greater than 0.1 m below which is considered to have no obvious effect to pedestrians (Jian et al. 2007). At 18:00 27th September before the main rainfall peak arrived, water has just started to accumulate and inundate certain low-lying locations, as shown in Fig. 8a. Figure 8b, c presents the inundation maps at 23:00 on 27th and 04:00 on 28th when the rainfall intensity continued to increase and reached its peak. Accordingly, inundation extent and flood depth continued to increase and large part of the densely urbanised town centre has been affected. The flood water continued to accumulate after the rainfall peak passed and inundated a larger area with high depth in certain locations at 09:00 on 28th, as shown in Fig. 8d. After that, the flood water started to gradually recede, as shown by the inundation map at 14:00 28th, i.e. Fig. 8e.

Fig. 8
figure 8

Predicted inundation maps at different output times: a 18:00 on 27th September; b 23:00 on 27th September; c 04:00 on 28th September; d 09:00 on 28th September; e 14:00 on 28th September

Full size image

Figure 9 presents the maximum inundation map over the whole domain, with the three most severely affected zones marked by the ovals. The prediction is consistent with public observations during the event. In Zones 1 and 2, most of the roads and streets were submerged with an average depth of over 0.5 m, and 1 m or deeper water depth reported in many locations. Traffic was paralysed, causing chaos and damages. Some low-lying locations in Zone 3 were also reported to suffer from high inundation depth of more than 1 m, and most of these places were ongoing construction sites.

Fig. 9
figure 9

Maximum flood inundation predicted for Typhoon Megi on 2 m grid resolution

Full size image

Zone 1 is within the Central Business District (CBD) of Fuzhou City. It is a low-lying spot next to a local river and therefore frequently suffers from flooding during typhoon and intense rainfall events. The zoom-in maximum inundation map in Zone 1 is presented in Fig. 10. The Typhoon Megi-induced flood has attracted substantial public attention and people have uploaded and shared a lot of flood information by means of photographs, video clips and text messages on social media. The crowd-sourced flood information was collected from the social media (e.g. Weibo) during and after the event and sorted according to geo-locations and time. The collected text messages helped identify 157 inundation locations that impeded the movement of vehicles and interrupted pedestrians. Among these inundation points, 126 of them are captured by the current simulation with major inundation; 84 of these locations are predicted with water depth exceeding 0.3 m (accounting for 53.50%); and the other 42 locations have water depth in the range of 0.1–0.3 m. In addition, 137 photographs were collected and 72 of them were clear and geographically identifiable. Figure 11 compares the predicted flood depths with those estimated from the 72 photographs. As examples, five of the collected street-view photographs (as shown in Fig. 12) are used to verify the simulation result and their locations (A, B, C, D and E) are marked on the maximum inundation map in Fig. 10. According to the recorded time and the associated text descriptions captured from the social media, all of these photographs were taken at the time when the area was most seriously flooded (i.e. from 8:00 to 11:00 on 28th September). The flooded streets as shown on the five photographs are all correctly predicted to be seriously flooded, as evidenced in Fig. 10. The crowd-sourced photographs have provided extra evidence to effectively confirm the presented simulation results.

Fig. 10
figure 10

Maximum inundation depth at the Central Business District

Full size image
Fig. 11
figure 11

Comparing the predicted flood depths with those extracted from the crowd-sourced photographs

Full size image
Fig. 12
figure 12

Crowd-sourced photographs showing the street-view flooding during Typhoon Megi (Photo source: Photograph C from http://www.fjsen.com/ and others from Weibo)

Full size image

To further validate model, the maximum flood depth measured by the local government at 368 different locations is used to compare with the simulation results. Based on these measured depths, the RMSE is calculated using Eq. (6) and the final RMSE is returned to be 0.36 m. Considering the fact that the vertical accuracy of the DEM is 0.4 m and the inevitable uncertainty associated with the measured data, the resulting RMSE of 0.36 m is considered to be acceptable. Comparison is also made for the simulations set up with different drainage loss rates, i.e. a full rate of 13.2 mm/h, half of this rate and zero drainage loss to demonstrate the effectiveness of the proposed simplified drainage modelling strategy. The RMSEs against the measured depths are calculated to be 0.3815 m and 0.3969 m, respectively, for the simulations with half and zero drainage loss rates. The corresponding zoom-in maximum inundation maps around the CBD zone are shown in Fig. 13. It is evident that the current simplified drainage modelling approach effectively takes into account the effect of urban drainage systems on flooding and produces better simulation results.

Fig. 13
figure 13

Zoom-in maximum inundation maps obtained using different rates of drainage loss: a full drainage loss rate of 13.2 mm/h; b half of the rate; c zero drainage loss

Full size image

4.2 Sensitivity to spatial resolution

Further simulations are conducted on DEMs of different resolution to investigate the effect of spatial resolution on the simulation results, and five different DEMs (2 m, 5 m, 10 m, 20 m and 30 m) are considered. The four lower resolution DEMs are produced from the 2 m DEM using the bilinear resampling techniques available from a standard GIS tool, which has been recognised to provide the most accurate and physically intuitive results (Fewtrell et al. 2008). The number of effective computational cells associated with each of the DEMs is provided in Table 1.

Table 1 The number of computational cells associated with each of the adopted DEMs
Full size table

The inundation maps of maximum depth produced on the five DEMs are shown and compared in Fig. 14. The inundation maps on 2 m and 5 m grid resolutions are comparable and show similar inundation extent and depth. However, the simulation results obtained on DEMs of coarser resolutions (i.e. 10 m, 20 m and 30 m) are clearly deviated from the high-resolution predictions, especially in the Central Business District (CBD) of the city and the northern mountain area. Using the result on the 2 m DEM as the reference solution, the difference between the inundation maps is calculated, respectively, for the predictions on the 5 m, 10 m, 20 m, and 30 m DEMs and is presented in Fig. 15, which demonstrate variable spatial distributions. In general, the difference between the maximum depths becomes more prominent as the grid resolution turns coarser. Associated with complicated topographic features, the maximum depth difference tends to be positive near to the mountainous area located in the northeast of the city and appears to be negative in the low-lying area in the CBD with dense urban buildings.

Fig. 14
figure 14

Maximum inundation maps on different DEMs: a 2 m; b 5 m; c 10 m; d 20 m; e 30 m

Full size image
Fig. 15
figure 15

Maximum depth difference calculated against the prediction on the 2 m DEM: a 5 m; b 10 m; c 20 m; d 30 m

Full size image

Figures 16 and 17 present the zoom-in DEMs for the localised mountainous area and the densely urbanised CBD as marked in Fig. 14e. Evidently, small alleys and river channels become increasingly unresolved and neglected on the coarser resolution DEMs, especially in the densely urbanised area; certain flow paths in the upper mountainous areas also become blocked, leading to reduced flow connectivity and detained flood water in the upstream mountainous areas, as shown in Fig. 14d, e. This subsequently leads to positive depth difference, as shown in Fig. 15c, d. Consequently, the water depth in the downstream low-lying CBD is underestimated due to the accumulated flow volume at the upstream area. Furthermore, the representation of the buildings in the urbanised areas becomes unacceptably inaccurate and gives incorrect elevation and width of the alleys between buildings on the coarse-resolution DEMs (especially at the 20 m and 30 m resolutions), leading to mistaken prediction of flow depth and flood extent. In conclusion, poor representation of narrow river channels, small roads and building blocks in coarser grids inevitably induces blockage effects to reduce flow connectivity in urbanised area, which subsequently introduces major uncertainty and prediction errors to the results. The use of 5 m or higher resolution DEMs is necessary to produce flood inundation maps with acceptable accuracy in densely urbanised areas.

Fig. 16
figure 16

Zoom-in DEMs at the upper mountain area: a 2 m; b 5 m; c 10 m; d 20 m; e 30 m

Full size image
Fig. 17
figure 17

Zoom-in DEMs at the low-lying CBD: a 2 m; b 5 m; c 10 m; d 20 m; e 30 m

Full size image

To quantify the sensitivity of the simulation results to grid resolution, RMSE and F-statistics of the maximum depth inundations on the four coarser resolution DEMs (i.e., 5 m, 10 m, 20 m and 30 m) are further calculated against the result on the 2-m fine resolution DEM and listed in Table 2. The results clearly demonstrate that, when the resolution of the DEMs becomes coarser, the RMSE increases and F-statistics decreases. The F-statistics obtained for the result on 5 m DEM is about 71%, indicating that the predicted inundation map is largely consistent with the one produced on the 2 m DEM. However, the F-statistics associated with the 10 m DEM has substantially decreased to become less than 50%, which is clearly unacceptable. These results further confirm that the use of 5 m or higher resolution DEMs is essential for urban flood modelling.

Table 2 RMSE and F-statistics calculated for the predictions on different DEMs
Full size table

To reproduce the 28-hour Typhoon Megi flood event in Fuzhou, the current multi-GPU-accelerated urban flood model requires about 35 h of runtime on 8 x K80 GPUs for the simulation on the 2 m DEM, involving 66 million effective computational nodes. For the simulation on the 5 m DEM, the runtime is only about 3 h. This confirms that HiPIMS provides a new robust simulation tool for high-resolution urban flood modelling across an entire city scale.

4.3 Analysis of scale effect

In hydrodynamic urban flood modelling, the use of smaller localised computational domains, rather than an entire urban catchment, represents a common practice in order to reduce the associated high computational cost. However, urban domain is intriguingly interconnected through complicated street, river and drainage networks during a flood event and the use of localised domains may introduce unacceptable simulation errors to the results. To investigate the scale effect of using localised domains (i.e. part of an entire urban catchment) on the simulation results, further simulations are carried out on a small urbanised area as marked by the red polygon in Fig. 18.

Fig. 18
figure 18

The location of the localised computational domain for investigating scale effect

Full size image

The 0.82 km2 localised catchment is chosen using GIS-enabled hydrology analysis and it is a densely urbanised area located near to the northern mountain area with steep slopes. To simplify the boundary conditions, the area is specifically chosen such that there is no river channel flowing through. It should be noted that river boundaries will increase the inter-connectivity with the outer domain and hence may introduce further simulation errors. All of the simulations are carried out on 2 m grid resolutions and the results are compared with the full-scale prediction over the whole 267.4 km2 domain. Two simulations are performed on this localised domain: 1) in simulation I, open boundary conditions are assumed along the physical boundary of the domain; 2) in simulation II, inflow/outflow boundary conditions are imposed at those major street inlets/outlets connecting to the outer domain and the flow hydrographs resulting from the full-scale simulation are used as the required boundary conditions. The locations of the inflow and outflow boundaries are marked by the red rectangles in Fig. 19a.

Fig. 19
figure 19

Analysis of scale effect: a locations of the inflow and outflow boundaries; b inundation map produced by simulation I; c inundation map produced by simulation II; d inundation map extracted from the full-scale simulation

Full size image

Figure 19b, c presents the maximum inundation maps produced by the two localised simulations, in comparison with the prediction from the full-scale simulation in Fig. 19d. As evident from the results, simulation I significantly underestimates the flood hazard with much smaller flood depth and extent. With the boundary conditions providing certain level of interaction with the outer flood domain, simulation II produces slightly better prediction, but the overall flood hazard is still largely under predicted. The RMSE and F-statistics calculated for the two sets of simulation results (against the reference solution obtained from the full-scale simulation) are, respectively, 0.49 m and 76.17% for simulation I, and 0.39 m and 87.28% for simulation II. In this major flood event, most of the localised domain is flooded and therefore the values for F-statistics are high for both simulations. However, the inundation depths predicted by the localised simulations are assessed to be seriously underestimated. This indicates that localised urban flood simulations may introduce great uncertainty to the numerical predictions and provide mistaken information for flood risk assessment.

5 Conclusions

This paper presents a High-Performance Integrated hydrodynamic Modelling System (HiPIMS) for application in reproducing a severe flood event induced by typhoon rainfall in a 267.4 km2 urbanised domain in the city of Fuzhou, China. Accelerated by multiple GPUs, the model is able to support computationally efficient city-scale flood simulations at a very high solution, e.g. close to real time for the simulation at 2 m spatial resolution involving over 66 millions computational cells. The simulation results are consistent with field measurements provided by local governments and extracted from crowd-sourced data.

Further simulations have been carried out to investigate the effects of grid resolution and domain scale on the numerical predictions. After systematically comparing the inundation maps created on five DEMs of different resolutions, it is concluded that the simulation results are sensitive to grid resolution. Coarse-resolution DEMs (> 5 m) may not represent well the complex topographical features (e.g. river channels, small roads and building blocks) and subsequently reduce flow connectivity in the densely urbanised area, which will lead to the prediction of unacceptable flood simulation results. Therefore, DEMs of 5 m or higher resolution are recommended for urban flood modelling.

To analyse the scale effect, further simulations in a localised computational domain have been performed with and without considering flow interaction with the outer domain using boundary conditions. The simulation results are compared with that obtained from a full-scale simulation. Both simulations predict significantly underestimated water depth and smaller inundation extent. As a common practice in urban flood modelling to cope with the high computational cost, localised simulation may introduce large simulation errors and producing unacceptable numerical predictions.