Dynamic Agent Based Simulation of an Urban Disaster Using Synthetic Big Data

Abstract

This paper illustrates how synthetic big data can be generated from standard administrative small data. Small areal statistical units are decomposed into households and individuals using a GIS buildings data layer. Households and individuals are then profiled with socio-economic attributes and combined with an agent based simulation model in order to create dynamics. The resultant data is ‘big’ in terms of volume, variety and versatility. It allows for different layers of spatial information to be populated and embellished with synthetic attributes. The data decomposition process involves moving from a database describing only hundreds or thousands of spatial units to one containing records of millions of buildings and individuals over time. The method is illustrated in the context of a hypothetical earthquake in downtown Jerusalem. Agents interact with each other and their built environment. Buildings are characterized in terms of land-use, floor-space and value. Agents are characterized in terms of income and socio-demographic attributes and are allocated to buildings. Simple behavioral rules and a dynamic house pricing system inform residential location preferences and land use change, yielding a detailed account of urban spatial and temporal dynamics. These techniques allow for the bottom-up formulation of the behavior of an entire urban system. Outputs relate to land use change, change in capital stock and socio-economic vulnerability.

Appendices

Appendix 1: Data Disaggregation Method

This appendix describes the disaggregation procedure of spatially aggregated alpha-numeric real-estate values and populations and their socio-economic attributes into discrete spatial units at the building level. It then proceeds to describe the validation of this allocation method.

The number of floors in residential buildings (F _R ), is calculated by dividing building height by average floor height of 5 m:

$$ {F}_R = \frac{H_B}{5} $$

In the case of non-residential buildings, the number of floors (F _N ) is estimated as the building height divided by average floor height of 5 m:

$$ {F}_N = \frac{H_B}{7} $$

Floor space for each building (S _B ) is then calculated by multiplying the number of floors in each building by its polygon area representing roof space:

$$ {S}_B={S}_R\times F $$

where:

• S _R  = Building polygon footprint

• F = Building number of floors

The GIS buildings layer and building type serve as the basis for the calculation of residential building value, non-residential building and equipment value. To create estimates of residential building value we use average house prices per m² 2008–2013 (in real 2009 prices). In cases where no transactions exist in a specific SA over that period we use a regional estimate for residential property prices.

• 1. Value of residential buildings (P _BR ) is calculated as follows:

  $$ {P}_{BR}={P}_{SR}\times {S}_{BR} $$

  where:

  P _SR  = Average SA price per m²

  S _BR  = Residential building floor space.

• 2. Value of Non residential buildings is calculated as follows:

  Non residential value per m² by region (P _RN ):

  $$ {P}_{RN} = \frac{V_{RN}}{S_{RN}} $$

  where:

  S _RN  = Total regional non residential floor space.

  V _RN  = Total regional non residential building stock

  Non residential building value per m² for each region is multiplied by the floor space of each non-residential building to produce non-residential building values (P _BN ):

  $$ {P}_{BN}={P}_{RN}\times {S}_{BN} $$

  where:

  S _BN  = non residential building floor space.

  Regional non-residential stock estimates have been calculated for nine aggregate regions elsewhere (Beenstock et al. 2011).

• 3. Value of Equipment and Machinery (P _RE ) is calculated as follows:

  $$ {P}_{RE} = \frac{V_{RE}}{S_{RN}} $$

  where:

  V _RE  = Total regional non residential equipment stock

  The equipment stock per m² for each region is multiplied by the floor space of each non-residential building to produce equipment stock totals by building (P _BE ):

  $$ {P}_{BE}={P}_{RE}\times {S}_{BN} $$

  where:

  S _BN  = non residential building floor space.

  The source for regional estimates of regional equipment and machinery stock is as above (Beenstock et al. 2011).

  The buildings layer also allows for the spatial allocation of aggregated households, and population counts (see Table 1) into building level households and inhabitant totals. Given the existence of these spatial estimates, the distribution of aggregate average monthly earnings, participation in the work force, employment sector, disabilities and age (see Table 1) into a building level distribution is implemented.

• 4. Household density by SA (households per m ²) of residential floor space in a statistical area (H _SR ) is calculated as follows:

  $$ {H}_{SR} = \frac{H_S}{S_{SR}} $$

  where:

  H _S  = Total population per statistical area (IV in Table 1).

  S _SR  = Total statistical area residential floor space.

  The number of households per building (H _B ) is calculated as follows:

  $$ {H}_B={H}_{SR}\times {S}_{BR} $$

• 5. Average number of inhabitants per m ² of residential floor space in a statistical area (I _SR ) is calculated as follows:

  $$ {I}_{SR} = \frac{I_S}{S_{SR}} $$

  where:

  S _SR  = Total statistical area residential floor space.

  I _S  = Total population per statistical area.

  Population counts per building (I _B ) are then calculated as follows:

  $$ {I}_B={I}_{SR}\times {S}_{BR} $$

• 6. Total earnings per building (M _B ) is calculated as follows:

  $$ {M}_B={M}_{SI}\times {H}_B $$

  where:

  M _SI  = Average monthly earnings per household by SA.

  H _B  = Total number of households in a building.

• 7. Number of inhabitants in each building participating in the labor force 2008 (I _W ) is calculated by multiplying the number of inhabitants in a building by the labor participation rate in the corresponding SA.

  $$ {I}_W={W}_S\times {I}_B $$

  where:

  W _S  = % of inhabitants participating in the labor force in an SA

  I _B  = Population count per building

• 8. Number of inhabitants per building by employment sector (I _O ) is calculated by multiplying the percentage of inhabitants employed by sector (commercial, governmental, industrial or home-based) per statistical area by the number of inhabitants in each building.

  $$ {I}_O={O}_S\times {I}_B $$

  where:

  O _S  = % of inhabitants employed in an employment category.

  I _B  = Population counts per building

• 9. Number of disabled inhabitants in each building (I _D ) is calculated by multiplying the number of inhabitants in a building by the percentage of disabled in the corresponding SA.

  $$ {I}_D={D}_S\times {I}_B $$

  where:

  D _S  = % of disabled inhabitants in an SA

  I _B  = Population count per building

• 10. Number of inhabitants in each age category (I _A ) is calculated by multiplying the number of inhabitants in a building by the percentage of inhabitants in each age category in the corresponding SA.

  $$ {I}_A={A}_S\times {I}_B $$

  where:

  A _S  = % of inhabitants in each age group category in an SA

  I _B  = Population count per building.

1.1 Validation of Data Disaggregation

To validate the data disaggregation procedure, we compare population counts and demographic attributes of our synthetic distribution with the Household Travel Survey (HTS) conducted in 2010 by the Jerusalem Transport Masterplan Team (JTMT; Oliveira et al. 2011). The idea behind the validation procedure is to re-aggregate the discrete counts yielded by our distribution to the spatial units used by the survey. The survey uses Aggregated Transportation Area Zones (ATAZs) which have a different spatial configuration to those of the SAs on which our distribution is predicated.

The HTS is based on representative samples of the population in the Jerusalem metropolitan area. It uses an expansion factor by which the population and attributes of each household are multiplied to replicate their representation in the ATAZs.

ATAZs covered by the HTS but not fully covered by our distribution (for example, ATAZs in East Jerusalem) are excluded from the analysis. This is because the HTS data are aggregates and we cannot account for their distribution within the ATAZ, as we can with our discrete data. Note also that some ATAZs are fully sampled in the HTS and others contain much lower sampling coverage. This can cause misrepresentation of populations and demographics in some ATAZs. This is taken into account in the validation procedure. Validation is performed for the following attributes: number of households, number of persons, persons ages 0–18, 19–64 and number of cars.

Table 2 presents the validation results. MAPE statistics are calculated for two subsets of the original ATAZ dataset. Results are compared between our synthetic distribution with those ATAZs with complete coverage and those with a sampling coverage of 70+%. Mean errors across the two sources are within the range of 18–25 %.

Table 2 Validation results for key variables

Appendix 2: Behavioral Rules for the ABM

• 1. Residential Location Choice is derived as follows:

  $$ {h}_h={b}_j\Rightarrow \left[\frac{I_h}{3}>H{P}_j\right]\ast \left[{k}_h>S\left({b}_j\right)\right]=1 $$

  where:

  h _h is the new residential location for household h,

  b _j is the building considered,

  $\left[\right]$ is a binary expression with value of 1 if true and 0 otherwise,

  I _h is household h’s monthly income,

  HP _j is monthly housing cost of an average apartment in building j,

  k _h is a random number between [0,1] indicating tolerance to change in residential environment incurred by relocation,

  S(b _j ) is a similarity score for building j in relation to current place of residence, calculated as follows:

  $$ S\left({b}_j\right)=\frac{\Phi \left(\frac{{\overline{I}}_j-{\overline{I}}_h}{I_{\sigma_h}}\right)+\Phi \left(\frac{{\overline{A}}_j-{\overline{A}}_h}{A_{\sigma_h}}\right)}{2} $$

  where:

  Φ is the standard normal cumulative probability function,

  Ī _j , Ā _j are the average household income and average age of individuals in building j, respectively

  Ī _h , Ā _h are average household income and average age of individuals in residential buildings within 100 m of current residential location of household h,

  ${I}_{\sigma_h}$, ${A}_{\sigma_h}$ are standard deviations of household income and of resident age in residential buildings within 100 m from current home location of household h, respectively.

• 2. Choice of sequence of activities: occurs in two stages. First, the number of activities is fixed and then activities are allocated to locations:

  $$ \begin{array}{ll}\hfill & \#A{c}_i=\left\Vert a*\left(\frac{k_i}{0.5}\right)*\left(1+ca{r}_h*0.33\right)*\left(1-di{s}_i*0.33\right)*\left(1+\left[ ag{e}_i=2\right]*0.33\right)\right.\\ {}& *\left.\left(1-\left[ ag{e}_i\ne 2\right]*0.33\right)\right\Vert + employe{d}_i* her{e}_i\hfill \end{array} $$

  where:

  # Ac _i is the number of activities for resident i,

  k _i is a randomly drawn number between [0, 1] reflecting preferences regarding number of activities,

  car _h is a binary variable equal to 1 if the household h owns a car and 0 otherwise,

  dis _i is a binary variable equal to 1 if individual i is disabled and 0 otherwise,

  age _i is the age group of individual i,

  employed _i is a binary variable equal to 1 if i is employed and 0 otherwise,

  here _i is a binary variable equal to 1 when i’s workplace is located within the study area and 0 otherwise,

  ‖x‖ indicates the nearest integer number to x,

  a is the average number of activities based on employment status; equals 2.5 for employed residents and 3 for non-employed.

  $$ {a}_{t+1,i}={b}_j\Rightarrow \left\lfloor {b}_j\ne {a}_{t,i}\right\rfloor *\left\lfloor {k}_i\ge Att\left({b}_j\right)\right\rfloor =1 $$

  where:

  a _t,i is the current location of individual i,

  ${a}_{t+1,i}$ is the next location of activity of individual i,

  k _i is a randomly drawn number between [0, 1] reflecting activity location preferences,

  Att(b _j ) is the attractiveness score for building j, calculated as follows:

  $$ \begin{array}{l}Att\left({b}_j\right)\\ {}=\frac{1-\raisebox{1ex}{$\Sigma {E}_j$}\!\left/ \!\raisebox{-1ex}{$\Sigma {B}_j$}\right.+1-\raisebox{1ex}{${D}_{ij}$}\!\left/ \!\raisebox{-1ex}{$ \max {D}_i$}\right.*\left(1+0.33*\left(-ca{r}_h+di{s}_i+\left[ ag{e}_i=3\right]\right)\right)+\left[L{U}_j=non\mathrm{R}\mathrm{e}s\right]*\raisebox{1ex}{$F{S}_j$}\!\left/ \!\raisebox{-1ex}{$ \max FS$}\right.}{2+\left[L{U}_j=non\mathrm{R}\mathrm{e}s\right]}\end{array} $$

  where:

  ΣE _j is the number of non occupied buildings within a 100 m buffer of building j,

  ΣB _j is the number of all buildings within a 100 m buffer of building j,

  D _ij is the distance of building j from the current location of individual i,

  max D _i is the distance of the building within the study area furthest away from the current location of individual i,

  LU _j is the land-use of building j,

  nonRes is non-residential use,

  FS _j is the floor-space volume of building j,

  maxFS is the floor-space volume of the largest non-residential building within the study area.

• 3. Choice of workplace location is calculated similarly to the choice of activity location:

  $$ W{P}_i={b}_j\Rightarrow \left[L{U}_j= EL{U}_i\right]*\left[{k}_i>\frac{\raisebox{1ex}{${D}_{ij}$}\!\left/ \!\raisebox{-1ex}{$ \max {D}_i$}\right.+1-\raisebox{1ex}{$F{S}_j$}\!\left/ \!\raisebox{-1ex}{$ \max FS$}\right.}{2}\right]=1 $$

  where:

  WP _i is the workplace location of individual i,

  ELU _i is the employment-sector-related land-use for individual i,

  k _i is a randomly drawn number between [0, 1] representing workplace location preferences,

  D _ij is the distance between building j and individual i’s place of residence,

  max D _i is the distance of the building within the study area furthest away from individual i’s place of residence.

• 4. Building values and the monthly cost of living in a dwelling unit are derived in a 3-stage process. First, daily change in average house price per SA is calculated. Then, values of individual buildings are derived and finally the price of the single, average dwelling unit is calculated. For non-residential buildings, the calculation of individual building values is similar.

  $$ \begin{array}{l}AH{P}_{z,t+1}=AH{P}_{z,t}*\left(1+ \log \left(\frac{\raisebox{1ex}{$po{p}_{z,t+1}$}\!\left/ \!\raisebox{-1ex}{$po{p}_{z,t}$}\right.+\raisebox{1ex}{$re{s}_{z,t}$}\!\left/ \!\raisebox{-1ex}{$re{s}_{z,t+1}$}\right.+\raisebox{1ex}{$n\mathrm{R}\mathrm{e}{s}_{z,t+1}$}\!\left/ \!\raisebox{-1ex}{$n\mathrm{R}\mathrm{e}{s}_{z,t}$}\right.}{3}\right)\right)\hfill \\ {}ANR{V}_{z,t+1}=ANR{V}_{z,t}*\left(1+ \log \left(\frac{n\mathrm{R}\mathrm{e}{s}_{z,t}}{n\mathrm{R}\mathrm{e}{s}_{z,t+1}}\right)\right)\hfill \end{array} $$

  where:

  AHP _z,t is average housing price per meter in SA z at time t,

  pop _z,t is population in SA z at time t,

  res _z,t is the number of residential buildings in SA z at time t,

  nRes _z,t is the number of non-residential buildings in SA z at time t,

  ANRV _z,t is the average non-residential value per meter in SA z at time t,

  $$ \begin{array}{c}H{P}_{j,t}=AH{P}_{z,t}*F{S}_j*\raisebox{1ex}{$S{L}_{j,t}$}\!\left/ \!\raisebox{-1ex}{$S{L}_{z,t}$}\right.\\ {}{V}_{j,t}=ANR{V}_{z,t}*F{S}_j\end{array} $$

  where:

  HP _j,t is the house price of a dwelling unit in building j at time t,

  SL _s,t is the service level within area s at time t—the ratio of non-residential buildings to residential buildings in this perimeter,

  V _j,t is the non-residential value of building j.

  $$ {P}_{du,t}=\frac{{\overline{I}}_t*\left(1+\frac{\raisebox{1ex}{$H{P}_{j,t}$}\!\left/ \!\raisebox{-1ex}{$\Sigma A{p}_j$}\right.-\raisebox{1ex}{${\displaystyle \sum_{l=1}^{L_t}H{P}_{l,t}}$}\!\left/ \!\raisebox{-1ex}{${\displaystyle \sum_{l=1}^{L_t}\Sigma A{p}_l}$}\right.}{P_{\sigma_t}}\right)}{c} $$

  where:

  P _du,t is the monthly cost of living in dwelling unit du at time t,

  Ī _t is the average household income in the study area at time t,

  ΣAp is the number of dwelling units within a building. If the building is initially of residential use, this is equal to its initial population size, otherwise it is the floor-space volume of the building divided by 90 (assumed to be average dwelling unit size in meters),

  L _t is the number of residential buildings in the study area at time t,

  ${P}_{\sigma_t}$ is the standard deviation of dwelling unit prices within the study area at time t,

  c is a constant.

• 5. Land-use changes, from residential to commercial and from commercial to unoccupied are based on the congruence between the building floor-space volume and the average intensity of traffic on roads within a 100 m radius over the preceding 30 days. Both these values are compared with the (assumed) exponential distribution of all values in the study area. This is done by computing the logistic probability of the relative difference in their locations in the distribution:

  $$ {P}_{j,t}\left(\Delta {x}_{j,t}\right)=\frac{e^{-\Delta {x}_{j,t}}}{1+{e}^{-\Delta {x}_{j,t}}} $$
  $$ \Delta {x}_{j,t}=\frac{z_{T{R}_{j,t}}-{z}_{F{S}_{t,t}}}{\left|{z}_{FS}\right|} $$
  $$ {z}_{y_{t,j}}=\frac{e^{-\raisebox{1ex}{${y}_{j,t}$}\!\left/ \!\raisebox{-1ex}{${\overline{y}}_t$}\right.}-{e}^{-\raisebox{1ex}{${y}_{me{d}_t}$}\!\left/ \!\raisebox{-1ex}{${\overline{y}}_t$}\right.}}{{\overline{y}}_t} $$

  where:

  P _j,t is the probability of land-use change for building j at time t,

  Δx _j,t is the relative difference in position of traffic load and floor-space for building j at time t,

  z _yj,t is the position of value y in the exponential distribution, relative to the median for building j at time t,

  $\frac{e^{-\raisebox{1ex}{${y}_{j,t}$}\!\left/ \!\raisebox{-1ex}{${\overline{y}}_t$}\right.}}{{\overline{y}}_t}$ is the exponential probability density value for $y\left(\frac{1}{{\overline{y}}_t}={\widehat{\lambda}}_t\right)$ for building j at time t,

  y _{med t} is the median of y at time t.

  If P > 0.99 for residential use, it changes to commercial. If the value is in the range [P(1) − 0.01, P(1)] for commercial uses, the building becomes unoccupied. This functional form and criteria values reduce the sensitivity of large commercial uses and small residential uses to traffic volume. Consequently, the process of traffic-related land-use change is not biased by a tendency to inflate initial land uses.

• 6. Earthquake impact is calculated as follows:

  $$ {\mathrm{Im}}_j=\frac{c*{10}^{mag}}{D_j*\left| \log \left({D}_j\right)\right|*{F}_j} $$

  where:

  Im _j is the impact building j suffers, c is a constant, mag is the earthquake magnitude (similar to Richter scale), D _j is distance of building j from the earthquake epicenter, F _j is number of floors in building j.

Appendix 3: Sensitivity Analysis

The hard-to-predict nature of model dynamics is a substantive issue in agent-based simulation (Ngo and See 2012). To test for parameter stability, we test two key model parameters:

• 1. Land-use sensitivity (‘lu’)—in this scenario, the sensitivity of actual and potential commercial land-use to traffic volumes is set in relation the nearest road and not all roads within 100 m as in the baseline case.

• 2. Housing budget distribution (‘hb’)—instead of defining a rigid parameter share for housing service costs out of total monthly income (0.33), variation within the willingness to pay for housing parameter is allowed. A unique value is drawn for each household from a normal distribution centered on the average expenditure on housing in Israel (23.4 % of total income), see http://www1.cbs.gov.il/reader/newhodaot/hodaa_template_eng.html?hodaa=201415290.

Each scenario is simulated 25 times with extreme results discarded (two cases for the lu scenario and one for the hb scenario). All other parameters remain unchanged. To obtain high resolution results we compute the most frequent land-use and the average residential value, non-residential value and vulnerability index for each building at discrete time points. For land-use we compute the share of times a different land-use is registered in relation to the baseline scenario. For other variables we compute the Median Absolute Percentage Error (MAPE). The results are presented in Fig. 14.

Fig. 14

Sensitivity tests for (a) lu scenario and (b) hb scenario.

The results illustrate morphological stability with the same land-use over scenario registered for almost 90 % of the buildings at all time points. The other variables also exhibit parameter stability (which is greater in the lu scenario). The only sensitive variable is average non-residential value. This is due to two changes exerting influence on the spatial distribution of commercial functions. The lu scenario constrains agglomeration tendencies and the hb scenario affects residential, and hence traffic, patterns. As commercial stock is small relative to residential stock these changes do not exhibit strong morphological influence but register greater and more fluctuating differences than in the other variables. While general patterns indicate parameter stability, micro-level differences are still observed.