1 Introduction

Air pollution is a global concern due to its detrimental impact on human health. Numerous studies have identified links between air pollution and increases in mortality and hospital admissions due to respiratory and cardiovascular disease (Brunekreef & Holgate, 2002). Predicting air pollution levels can help public health officials make recommendations to limit outdoor air exposure and plan for increased strain on the health care system during periods of higher air pollution.

The Air Quality Health Index (AQHI) was developed by Health Canada and Environment Canada to communicate the health impact of short-term exposure to current air pollutant levels to Canadians (Environment and Climate Change Canada, 2019). The AQHI is an aggregate indicator of the average trailing 3-h concentrations of three pollutants: ozone (O3), nitrogen dioxide (NO2), and fine particulate matter (PM2.5). The AQHI is reported using 11 values ranging from 1 to 10+. A value of 1 represents the lowest health risk, while a value of 10+ represents a very high health risk. The AQHI values are further categorised according to their respective health risk; Low Risk (1–3), Moderate Risk (4–6), High Risk (7–10), and Very High Risk (10+). Each risk category has a recommended action for both the general population and the at-risk population (Environment and Climate Change Canada, 2015). For example, the Low Risk category recommends continuing all outdoor activities, while the Very High Risk category recommends avoiding all strenuous outdoor activities.

Ontario uses a modified form of the AQHI, where hourly concentrations of additional pollutants are also considered when assigning risk categories. In Ontario, ozone, nitrogen dioxide, sulphur dioxide, carbon monoxide, and total reduced sulphur compounds are the pollutants that may change the AQHI categories. For instance, if hourly air concentrations of one or more of the contaminants mentioned above exceed Ontario’s Ambient Air Quality Criteria (AAQC), a desirable level of a contaminant in air, and the AQHI value is currently considered Low Risk or Moderate Risk, then the AQHI risk category is adjusted to a High or Very High Risk category (Ontario Ministry of the Environment, Conservation and Parks, 2019a).

The discrete-time Markov chain is a probabilistic model used to analyse stochastic processes. It has been employed in a wide range of applications such as modelling precipitation (da Silva et al., 2019; Schoof & Pryor, 2008), infrastructure deterioration (Baik et al., 2006), and wind speed (Sahin & Sen, 2001). Discrete-time Markov chains have also been employed to model air pollution (e.g., Asadollahfardi et al., 2016; Caraka et al., 2019; Mohamad et al., 2018; Romanof, 1982). Nebenzal and Fishbain (2018) found that, for forecasting long-term NO2 pollution, Markov chain models reduce the total error compared to other forecasting methods (e.g., multiple linear regression, moving average, exponential smoothing, Holt, and persistence methods).

Most air pollution studies that used Markov models focused on modelling specific contaminant concentrations such as particulate matter (Asadollahfardi et al., 2016; Caraka et al., 2019; Mohamad et al., 2018), nitrogen dioxide (Nebenzal & Fishbain, 2018), ozone (Rodrigues et al., 2019), and sulphur dioxide (Romanof, 1982), while few studies have focused on directly modelling an air pollution index similar to the AQHI (Alyousifi et al., 2018; Alyousifi et al., 2019; Zakaria et al., 2019). From a public health standpoint, directly modelling the AQHI is more informative since the AQHI categories are directly related to health risks and outdoor air quality advisories.

This study aims to provide further insight into the stochastic behaviour of AQHI risk categories measured in Ontario, which has different climate conditions than those in the other studies (i.e., Alyousifi et al., 2018; Alyousifi et al., 2019; Zakaria et al., 2019). A discrete-time Markov chain was employed using three of the AQHI risk categories (Low Risk, Moderate Risk, High Risk) as states to determine (1) the transition probabilities between these states, (2) the long-run proportion of time spent in each state, and (3) the mean persistence time of each state. These results were then used to analyse spatial trends in the AQHI risk categories that could help public health officials understand the characteristics of air quality advisories, including their likelihood and duration.

2 Methodology

2.1 Study Area

Ontario is a province located in Central Canada. It is Canada’s most populous province and second-largest by total land area. There are thirty-nine air monitoring stations in Ontario operated by the Ontario Ministry of the Environment, Conservation and Parks (MECP). Most of the thirty-nine air monitoring stations are located in populated areas in Southern Ontario, with the highest number in the Golden Horseshoe area around Toronto. Five of the thirty-nine monitoring stations were excluded from the study due to insufficient data. The locations of the selected air stations for this study are shown in Fig. 1. The characteristics of these stations are illustrated in Table 1.

Fig. 1
figure 1

Locations of air monitoring stations in Ontario, Canada

Table 1 Ontario AQHI air monitoring station details

2.2 Data

Historical Ontario AQHI data is available in hourly increments for a period of 24 h from the Ontario Ministry of the Environment, Conservation & Parks (MECP) website (Ontario Ministry of the Environment, Conservation and Parks, 2019b). Most air monitoring stations had AQHI data available for the period from January 1, 2015 to the current date, despite Ontario replacing their former Air Quality Index with the AQHI on June 24, 2015.

The AQHI is an aggregate indicator of the average trailing 3-h concentrations of three pollutants: ozone (O3), nitrogen dioxide (NO2), and fine particulate matter (PM2.5). The AQHI is calculated according to the following formula: \( \mathrm{AQHI}=\frac{1000}{10.4}\times \left({e}^{0.000537\times {O}_3}-1+{e}^{0.000871\times {NO}_2}-1+{e}^{0.000487\times {PM}_{2.5}}-1\right) \) (Szyszkowicz, 2019). Ozone and nitrogen dioxide concentrations are inputted as parts per billion, and fine particulate matter as μg/m3. The AQHI is then rounded to the nearest integer. Additionally, if hourly air concentrations of ozone, nitrogen dioxide, sulphur dioxide, carbon monoxide, or total reduced sulphur compounds exceed Ontario’s Ambient Air Quality Criteria (AAQC), and the AQHI value is currently considered Low Risk or Moderate Risk, then the AQHI value is adjusted to a High or Very High Risk category (Ontario Ministry of the Environment, Conservation and Parks, 2019a).

A Python script was developed to compile the data from the MECP website by automating the process of downloading the hourly data for each 24 h. This study used the available data for each air monitoring station from January 1, 2015 to December 31, 2019. Five of the thirty-nine air monitoring stations were excluded from the study due to insufficient data (missing more than a year’s worth of data). The total number of observations in the historical dataset for the air monitoring stations during the study period was 1,463,652 h. This amount represents an overall data coverage of 98.3% (i.e. 1.7% of the data is missing). The data coverage for each air monitoring station is included in Table 1.

2.3 Discrete-time Markov Chain Model

Markov chains are probabilistic models used to analyse stochastic processes. The choice of discrete-time or continuous-time Markov chain depends on the analysed time series. In this application, the discrete-time Markov chain is used since the time series consists of discrete hourly increments. Discrete-time Markov chains are characterised by the number of states in the state space and the number of previous states the transition probability is dependent on.

Consider a stochastic process {Xn,  n = 1, 2, 3, …} that takes on a finite or countable number of possible values defined as the state space, S. If Xn = i, then the process is said to be in state i at time n. The stochastic process is a Markov chain if the conditional distribution of any future state is independent of the past states and depends only on the present state. This condition is known as the Markov or memoryless property and can be expressed as P{Xn + 1 = j| Xn = i, Xn − 1 = in − 1, …, X2 = i2, X1 = i1} = P{Xn + 1 = j| Xn = i,} = pij for all states i1, i2, …, in − 1, i, j and all n ≥ 0. The value pij is the one-step transition probability and represents the probability of transitioning from state i to state j. \( {p}_{\mathrm{ij}}^n \) is used to represent the elements of the nth power of the transition probability matrix Pn. The transition probability matrix, P, for a Markov chain with k states is composed of k × k one-step transition probabilities, pij, where 0 ≤ pij ≤ 1 and the sum of the probabilities in each row is equal to 1 (\( {\sum}_{j=1}^k{p}_{\mathrm{ij}}=1 \)).

Each element, pij, of the transition probability matrix was calculated for a given state i, where nij is the observed frequency of one-step transitions in the historical data from state i to state j, using:

$$ {p}_{\mathrm{ij}}=\frac{n_{\mathrm{ij}}}{\sum_{j=1}^k{n}_{\mathrm{ij}}} $$
(1)

The validity of fitting a Markov chain k states to observed data can be investigated using the Chi-square (χ2) test statistic

$$ {\chi^2}_{\mathrm{calc}}=\sum \limits_{i=1}^k\sum \limits_{j=1}^k\frac{{\left({n}_{\mathrm{ij}}-{e}_{\mathrm{ij}}\right)}^2}{e_{\mathrm{ij}}} $$
(2)

where nij represents the observed transition frequency, and eij represents the expected transition frequency (Wilks, 2011). The null hypothesis is that the observed data is serially independent. The alternative hypothesis is that the observed data was generated by a Markov chain. Under the null hypothesis of independence, eij is calculated as follows:

$$ {e}_{\mathrm{ij}}=\frac{\left({\sum}_{i=1}^k{n}_{\mathrm{ij}}\right)\left({\sum}_{j=1}^k{n}_{\mathrm{ij}}\right)}{\left({\sum}_{i=1}^k{\sum}_{j=1}^k{n}_{\mathrm{ij}}\right)} $$
(3)

For a Markov chain with k states, the test statistic follows the χ2distribution with (k − 1)2 degrees of freedom.

2.3.1 Stationary Distribution

The stationary distribution shows the long-term proportion of time each air monitoring station spends in a specific AQHI risk category. The estimated time can be used to evaluate the air quality at each air monitoring station by comparing the time spent in the different AQHI risk categories. For example, a longer portion of time spent in the Low Risk category would indicate better air quality.

The stationary distribution of a Markov chain refers to the long-run probability distribution that remains unchanged as time progresses. A finite-state Markov chain that is ergodic will have each row of the limiting distribution (\( \underset{n\to \infty }{\lim }{\boldsymbol{P}}^n \)) converge to the stationary distribution. Let πj represent the long-run proportion of time spent in a state j. If the finite-state Markov chain is ergodic, the stationary distribution is unique, and πj can be calculated with the following equations (Ross, 2014):

$$ {\pi}_j=\sum \limits_{i=1}^k{\pi}_i{p}_{\mathrm{ij}} $$
(4)
$$ \sum \limits_{j=1}^k{\pi}_j=1 $$
(5)

A finite-state Markov chain is said to be ergodic if it is aperiodic and irreducible. A finite-state Markov chain is irreducible if all states communicate with each other. State i and state j are said to communicate if they are accessible from each other. State j is said to be accessible from state i if \( {p}_{\mathrm{ij}}^n>0 \) for some n time-steps. A Markov chain is said to be aperiodic if all of its states are aperiodic. A state is aperiodic if it is not periodic. A state is said to be periodic if the chain can return to the state only at multiples of some specific integer larger than 1.

2.3.2 Mean Persistence Time

The expected amount of time that, after the Markov chain enters an AQHI risk category, it remains in the same AQHI risk category before exiting can be referred to as the mean persistence time. Developing a model to estimate the mean persistence time of a specific AQHI risk category can be useful for public health officials to predict the length of air quality advisories. When public health officials communicate the health message associated with each AQHI risk category, additional information about the expected duration could be included as well. This information would allow the concerned population to better plan their activities to comply with the health message. For example, if, after entering the AQHI High Risk category, the mean persistence time is 8 h, then an air quality advisory to reduce or reschedule strenuous outdoor activities for at least 8 h can be issued before requiring a reassessment.

The transition probability matrix can be used to calculate the expected time it takes to enter any set of absorbing states from a transient state. A transient state is a non-recurrent state. It means that there is a non-zero probability that a Markovian process starting in a transient state will never return to that state. An absorbing state is defined as a state that cannot be transitioned out of after it is entered. A state i can be transformed into an absorbing state by setting pii = 1 and pij = 0 for all j ≠ i. For an ergodic Markov chain with k states, after transforming b states into absorbing states, a new matrix with t = k − b states can be defined as

$$ {\boldsymbol{P}}_T=\left[\begin{array}{cccc}{p}_{11}& {p}_{12}& \cdots & {p}_{1t}\\ {}{p}_{21}& {p}_{22}& \cdots & {p}_{2t}\\ {}\vdots & \vdots & \ddots & \vdots \\ {}{p}_{t1}& {p}_{t2}& \cdots & {p}_{\mathrm{tt}}\end{array}\right] $$
(6)

where the elements of PT are the reordered and renumbered one-step transition probabilities pij for all non-absorbing states (Ross, 2014). Let sij be the expected amount of time before absorption that a Markov chain spends in state j, given it started in state i. A t × t matrix S composed of elements sij can be calculated using S = (I − PT)1 where I is the t × t identity matrix (Ross, 2014).

By making all states, except for state i, an absorbing state, it is possible to calculate the mean persistence time of state i (the expected amount of time that the Markov chain will remain in state i before transitioning out of state i). For this case, sii can be calculated using:

$$ {s}_{\mathrm{ii}}=\frac{1}{1-{p}_{\mathrm{ii}}} $$
(7)

3 Results and Discussion

3.1 Discrete-Time Markov Chain Model

We used a discrete-time Markov chain to model the occurrence of AQHI risk categories at each air monitoring station as a stochastic process. We defined the state space as S = {1, 2, 3}, where states 1, 2, and 3 represent the Low Risk, Moderate Risk, and High Risk AQHI categories, respectively. The Very High Risk AQHI category was excluded from the study since only one occurrence was observed during the study period across all air monitoring stations (i.e. 1 out of 1,463,652 h). Additionally, the High Risk AQHI state was removed from the state space of the Markov chain of specific air monitoring stations if there were no observed occurrences during the study period.

A transition probability matrix was determined for each air monitoring station. The observed frequency of transitions is contained in Table 2. The transition probabilities were estimated using Eq. (1). Each element of the transition probability matrices is included in Table 3, and the spatial distribution is shown in Fig. 2. An example of the average transition probability matrix observed at air monitoring stations in Ontario is:

$$ \boldsymbol{P}=\left[\begin{array}{ccc}0.990& 0.010& 0.000\\ {}0.134& 0.865& 0.002\\ {}0.003& 0.443& 0.554\end{array}\right] $$
Table 2 Markov chain transition observation frequency
Table 3 Markov chain transition probabilities
Fig. 2
figure 2

Transition probabilities. a p11. b p12. c p13. d p21. e p22. f p23. g p31. h p32. i p33

The validity of fitting a Markov chain to the observed data for each individual air monitoring station was investigated using Eq. (2). The Chi-square test statistic, χ2calc, was calculated for each air monitoring station, and compared to the χ2distribution with (k − 1)2 degrees of freedom. For this analysis, the significance level was chosen to be 0.01. The calculated p value and degrees of freedom for each air monitoring station are contained in Table 2. The p value for the observed data from each air monitoring station was found to be less than 0.001. At the 1% significance level, there is sufficient evidence to reject the null hypothesis that the observed data is serially independent.

Overall, the transition probability matrix for each air monitoring station is similar. The transition probabilities for the transition from the Low Risk AQHI state to the Low Risk AQHI state was the highest occurring probability in the transition probability matrix for each of the air monitoring stations (mean of 0.990, ranging from 0.978 in Toronto West to 0.999 in Petawawa). This implies that the Low Risk AQHI state is generally stable since once the Low Risk AQHI state is entered, it is expected to remain for a while. Transition probabilities for moving directly from the Low Risk AQHI state to a High Risk state were generally non-existent for every air monitoring station. This means that the sudden onset of a High Risk AQHI state from the Low Risk AQHI state is extremely unlikely.

The transition probabilities for the transition out of the Moderate Risk AQHI state were highest for the transition to the Moderate Risk AQHI state (mean of 0.865; ranging from 0.782 in Thunder Bay to 0.890 in Toronto Downtown). These results imply that the Moderate Risk AQHI state is generally stable since once the Moderate Risk AQHI state is entered, it is likely to remain in that state. The next likeliest transition was to the Low Risk AQHI state (mean of 0.134, ranging from 0.110 in Toronto Downtown to 0.209 in Sault Ste Marie). The least probable transition was to the High Risk AQHI state (mean of 0.002, ranging from 0.000 at multiple air monitoring stations to 0.012 in Grand Bend).

Due to the low number of observed occurrences of High Risk AQHI category during the study period, the transition probabilities from the High Risk AQHI state to other states are less consistent across the air monitoring stations. Generally, the most probable transition from the High Risk AQHI state was to the High Risk AQHI state (mean of 0.554, ranging from 0.000 in Cornwall to 0.714 in Kitchener). The next likeliest transition was to the Moderate Risk AQHI state (mean of 0.443, ranging from 0.000 in Thunder Bay to 1.000 at multiple air monitoring stations). Finally, the least probable transition was to the Low Risk AQHI state (mean of 0.003; ranging from 0.000 at all air monitoring stations except Thunder Bay to 1.000 in Thunder Bay).

Generally, transitions out of an AQHI state were almost always to an AQHI state with a risk category one level above or below it. For example, only three transitions from the Low Risk AQHI state directly to the High Risk AQHI state were observed out of 1,365,195 transitions from the Low Risk AQHI state, and 1,463,652 transitions in the study period. Only one transition from the High Risk AQHI state directly to the Low Risk AQHI was observed out of the 386 h spent in total in the High Risk AQHI state and 1,463,652 h in the study period. This indicates that the AQHI risk states usually transition gradually, and sudden, large changes are not expected. This fact means that public health officials will likely not need to issue an air quality advisory for the High Risk AQHI category without an ongoing air quality advisory for the Moderate Risk AQHI category. Additionally, this observation indicates that if any air quality mitigation measures were instituted during the study period, they were not effective enough to cause an immediate reduction from the High Risk AQHI category to the Low Risk AQHI category.

3.2 Stationary Distribution

The stationary distribution of the Markov chain for each air monitoring station was calculated. Each Markov chain was shown to be irreducible and ergodic, which meant that the stationary distribution could be calculated and that it was unique. It is worth mentioning that the primary source of air pollution in Ontario is population growth. In the long term, this factor may increase the probability of air pollution over time; however, the study period (2015–2019) is relatively short. Therefore, the change in air-pollution probability over time is considered insignificant that means the Markov chain adopted in this study is assumed to be homogeneous in time. The stationary distribution of each Markov chain is contained in Table 4. An example of the average stationary distribution for all of the air monitoring stations is [π1, π2, π3] = [0.933, 0.067, 0.000].

Table 4 Stationary distribution

The highest proportion of time for each monitoring station was spent in the Low Risk AQHI category (mean of 0.933, ranging from 0.838 in Hamilton Downtown to 0.994 in Petawawa). The second highest proportion of time was spent in the Moderate Risk AQHI category (mean of 0.067, ranging from 0.006 in Petawawa to 0.161 in Hamilton Downtown). Very little time was spent in the High Risk AQHI category (mean of < 0.000, ranging from 0.000 at most air monitoring stations to 0.001 at eight air monitoring stations (Table 4).

Generally, the air monitoring stations with the lowest expected proportion of time spent in the Low Risk AQHI category are near densely populated areas in urban environments. A summary of the spatial analysis of the stationary distributions is shown in Fig. 3.

Fig. 3
figure 3

Long-run proportion of time spent in a AQHI Low Risk category, b AQHI Moderate Risk category, and c AQHI High Risk category

3.3 Mean Persistence Time of an AQHI Category

The mean persistence time of each state of the Markov chains was calculated for each air monitoring station; the results are reported in Table 5, and the spatial distribution is shown in Fig. 4. The average mean persistence time in hours for all of the air monitoring stations in Ontario is [s11, s22, s33] = [101.7, 6.7, 1.5].

Table 5 Mean persistence time of an AQHI category
Fig. 4
figure 4

Mean persistence time of a AQHI Low Risk category, b AQHI Moderate Risk Category, and c AQHI High Risk category

The mean persistence time was the highest for the Low Risk AQHI category (mean of 101.7 h, ranging from 46.6 h in Hamilton Downtown to 955.2 h in Petawawa). The second highest mean persistence time was for the Moderate Risk AQHI category (mean of 6.7 h, ranging from 4.6 h in Thunder Bay to 9.1 h in Toronto Downtown). The lowest mean persistence time was for the High Risk AQHI category (mean of 1.5 h, ranging from 1.0 h at five air monitoring stations to 3.5 h in Kitchener).

The ideal distribution for air quality health impacts would have a very high mean persistence time for the Low Risk AQHI category, and very low mean persistence times for the Moderate Risk, and High Risk AQHI categories. Generally, this pattern was observed at air monitoring stations in less populated, rural environments. At the air monitoring stations in more populated, urban environments, this pattern was also observed to a lesser extent. Based on the results of this analysis, public health officials can, on average, expect to issue air quality advisories in Ontario with health messages lasting for 6.7 h for the Moderate Risk AQHI category and 1.5 h for the High Risk AQHI category.

4 Conclusions

In this study, we used the Markov chain model to investigate the pattern of AQHI risk categories in Ontario for a period of 5 years, from 2015 to 2019. We estimated the transition probability matrix for each air monitoring station. We identified a general trend in the transition probability patterns for AQHI risk categories. We found that each AQHI risk category generally tends to be stable across each air monitoring station. And transitions between the AQHI risk categories occur gradually; sudden, large transitions between risk categories two levels away are not expected to occur.

The transition probability matrix facilitates the calculation of the stationary distribution showing the long-term proportion of time that each air monitoring station spends in a specific AQHI risk category. We calculated the mean persistence time for each AQHI risk category and identified the average duration of air quality advisories in Ontario for the Moderate Risk and High Risk AQHI categories. Overall, we found that air monitoring stations in less populated, rural environments had better air quality and spent less time in Moderate Risk and High Risk AQHI categories than air monitoring stations in more populated, urban environments.

The discrete-time Markov chain analysis done in this study can be extended to include other air monitoring stations outside of Ontario to broaden the study area and determine if the trends identified in this study can be generalised to other areas. The analysis could also be split into multiple periods to identify temporal trends in the observed AQHI risk category data. In the future, if there is enough data, a similar analysis of AQHI data recorded during the period of the COVID-19 pandemic would provide more insights about the impact of reduced human outdoor activities on air quality.