A comparative study on predicting influenza outbreaks using different feature spaces: application of influenza-like illness data from Early Warning Alert and Response System in Syria
An accurate forecasting of outbreaks of influenza-like illness (ILI) could support public health officials to suggest public health actions earlier. We investigated the performance of three different feature spaces in different models to forecast the weekly ILI rate in Syria using EWARS data from World Health Organization (WHO). Time series feature space was first used and we applied the seven models which are Naïve, Average, Seasonal naïve, drift, dynamic harmonic regression (Dhr), seasonal and trend decomposition using loess (STL) and TBATS. The Second feature space is like some state-of-the-art, which we named \(53-weeks-before\_52-first-order-difference\) feature space. The third one, we proposed and named \(n-years-before\_m-weeks-around\) (YnWm) feature space. Machine learning (ML) and deep learning (DL) model were applied to the second and third feature spaces (generalized linear model (GLM), support vector regression (SVR), gradient boosting (GB), random forest (RF) and long short term memory (LSTM)).
It was indicated that the LSTM model of four layers with \(1-year-before\_4-weeks-around\) feature space gave more accurate results than other models and reached the lowest MAPE of \(3.52\%\) and the lowest RMSE of 0.01662. I hope that this modelling methodology can be applied in other countries and therefore help prevent and control influenza worldwide.
KeywordsInfluenza-like illness (ILI) Feature space Time series analysis Long short term memory (LSTM)
Early Warning Alert and Response System
dynamic harmonic regression
seasonal and trend decomposition using Loess
\(n-years-before\_m-weeks-around\) feature space
generalized linear model
support vector regression
long short term memory
mean absolute percentage error
root mean square error
Influenza epidemic results in three to five million cases of severe illnesses and approximately 290,000 to 650,000 deaths worldwide each year . WHO’s Early Warning, Alert and Response System (EWARS) is designed to improve disease outbreak detection in emergency settings. The system has been built in SYRIA since 2012 to collect and detect near real-time information on several outbreaks including influenza . The Existing researches on modeling influenza epidemic falls into two categories: Mechanistic and Statistical models. They are summarized in literature reviews [3, 4, 5, 6] and in the CDC comparisons [7, 8]. Researches under statistical category vary according to the different Features and methods used. Some researchers used the number of patients in the past as features [9, 10, 11, 12, 13], while others integrated other data sources to predict the number of patients in the future. Examples of these sources are climatological data [14, 15], search engine queries [16, 17, 18, 19], public comments on social media like Twitter [20, 21], online information-seeking behavior on websites like Wikipedia [22, 23] and a combination of multiple data streams [15, 24, 25, 26]. Different methods on these features were applied. Some researches treated the problem as an instance of more general time series forecasting using time series methods (ARIMA, ARIMA-STL, GARMA) [9, 10, 17, 27], while others used ML methods including Stacked linear regression [24, 26], AdaBoost regression with decision trees , GB , SVR [26, 28], elastic net  RF [11, 12, 28], Artificial Neural Network (ANN) [12, 20]. Recently, a DL method Called LSTM has attracted much interest in ILI prediction and gave excellent results, which are more accurate than those of other methods [12, 13, 15, 29]. In addition to investigating the performance of the three different feature spaces with multiple time-series, ML and DL based methods to predict the weekly ILI rate in Syria; we proposed novel future spaces \(n-years-before\_m-weeks-around\) that integrate into state-of-the-art ML and DL methods. There are two important contributions of this paper. First, the use of \(n-years-before\_m-weeks-around\) future spaces to predict ILI rate. Second, analyzing multiple models performance over the EWARS data from WHO in Syria.
Materials and methods
Figure 1a illustrates the raw data. The Y-axis represents the weekly ILI rate, and the X-axis represents the time series. The seasonality is obvious, as shown in Fig 1b. We split the data into two parts: the first 90% was the training set and the last 10% was the testing set.
In this study, we reviewed three types of feature space. We named them as (time series feature space, \(53-weeks-before\_52-first-order-difference\) feature space and \(n-years-before\_m-weeks-around\) feature space). Tables S1 and S2 illustrate the data set, response, predictors and the pretreatment of the source data. An additional pdf file shows this in more detail (see Additional file 1).
Time series feature space (fs1) We treated the ILI weekly data as time series with seasonal value 52.
\(53- weeks-before\_52-first-order-differences\)feature space (fs2) Some previous studies found that using the ILI rate of the past 53 weeks and the 52 first-order differences helped improve the results of the prediction models for influenza data [11, 12, 13].
\(n-years-before\_m-weeks-around\)(YnWm) feature spaces (fs3) We reviewed a maximum of 3 years before and 5 weeks around. To predict the ILI rate of week WX of year YX, We used the ILI rate of the past m weeks before WX in YX, the ILI rate of past n seasonal weeks and the ILI rate of the m weeks around seasonal weeks. In the case of \(1-year-before\_4-weeks-around\), the response y is the ILI rate of week 30 of year 2017 then the predictors are the ILI rates of weeks 29, 28, 27, 26 from year 2017 and the ILI rates of weeks 26, 27, 28, 29, 30, 31, 32, 33, 34 from year 2016. The number of predictors and rows changed depending on the value of n and m and we had to drop some data rows so that we get the same training data length for any value of n and m. Figure 2a illustrates the feature spaces with examples of \(3-years-before\_5-weeks-around\) and \(1-year-before\_4-weeks-around\) feature spaces.
The models, programming languages, and libraries, which were used in this study were illustrated in Additional file 1. We trained all models in R Programming Language (version 3.4.4). For time series models, we used the “forecast” package (version 8.4). For ML models, we applied the caret package (Version 6.0-8). For DL models, we used the Keras package (Version 2.2.4) based on Tensorflow (Version 1.10). A personal computer with Intel I7-8550U processor, 8 GB of RAM and an NVIDIA 130 MX GPU was used for the experiments. Each experiment takes approximately 1 to 30 min to train the model. The prediction takes less than 5 s on the same hardware.
fs1 with time series models
The MAPEs and RMSEs of the testing set for all methods applied on fs1: time series and fs2: \(53- weeks-before\_52-first-order-differences\) and fs3: \(n-years-before\_m-weeks-around\) feature space
(a) The MAPEs%
(b) The RMSEs
fs2 with ML and DL models
We applied four ML methods (GLM, SVR, GB and RF) and two DL methods (LSTM with 3 layers and LSTM with 4 layers). Table 1(a), (b) shows that none of the models achieved better results than the baseline model in MAPE metric but in RMSE metric, the LSTM model of 3 layers (RMSE = 0.02294) outperformed the baseline model and all otehr models with fs1 and fs2 .
fs3 with ML AND DL models
According to \(0<=n<=3\ and\ 0<=m<=5\) we trained 23 feature spaces in four ML methods (GLM, SVR, GB and RF) and two DL methods (LSTM with 3 layers and LSTM with 4 layers). Totally, we made 138 experiments with results shown in Table 1(a), (b). In accordance with MAPE metric, We found that 22 experiments show a decrease in metrics less than that of the baseline model (1 experiment: by using GB model with Y2W2 feature space, 10 experiments: by using LSTM model of 3 layers with “Y1Wm, Y2Wm: excluding m=0 and m=4” feature spaces, 11 experiments: by using LSTM model of 4 layers with “Y1Wm: excluding m=1,Y2Wm: excluding m=5,Y3w1” feature spaces) but In accordance with RMSE metric, We found that 44 experiments show a decrease in metrics less than that of the baseline model (1 experiment: by using GB model with Y2W2 feature space, 20 experiments: by using LSTM model of 3 layers with “YnWm: excluding n=3 with m=3,4,5” feature spaces, 23 experiments: by using LSTM model of 4 layers with “YnWm” feature spaces). we achieved the best result (MAPE = 3.52%, RMSE = 0.01662) by using LSTM model of 4 layers with Y1W4 feature space. in comparison to models with fs2, the results show that for any model used, there are at least one value to n and m in fs3 that achieved better result than the same model with fs3. Our explanation of the success of this feature space is that models can learn the seasonality by n years before and can recognize the trend of the data by m weeks around.
The temporal variation of the real weekly ILI rate and the predicted values obtained from the three models (the best result in each feature space) for the test period were plotted in Fig 2b. As can be seen, the predicted values of weekly ILI rate were in a good agreement with their related observed values and the used models could be used to model the weekly ILI rate. Moreover, LSTM model of 4 layers with fs3 resulted in better predicted values and trend between them for the observed values of ILI rate than the other models especially when there is a change in trend between observations.
The accurate forecast of an outbreak of ILI could support public health officials in taking public health actions, such as allocating or temporarily readjusting medical resources for hospitals and medical centers. The ILI rate varies from year to year depending on multiple factors. Despite this difference, the ILI rate plot in each year takes a semi-constant form with an increase or decrease compared to previous years and could be predicted by using specific observations of past years with statistical models. Performance of statistical models is data dependent and there is no model that performs well in all situations. Therefore, evaluating the performance of different models is of great importance as they provide useful and important information regarding strengths and weaknesses of the models and gives an insight to use better models for forecasting purposes. Some sate of the arts [11, 12] utilize 53 weeks before with 52 first order difference with different statistical models and found that both the recent observations and the later observations with the difference were interesting and had significant influence on the predication. In this study, we proposed novel future spaces, namely \(n-years-before\_m-weeks-around\), and compared to some existing future spaces that utilize historical observations in different ways by integrating it into state-of-the-art ML and DL models. Our results revealed the success of our future space for some values of n and m and its failure for other values to outperform other future spaces in prediction ILI rate in Syria. This fact suggests that combining carefully selected number of Nearby Historical Observations with Carefully selected number of seasonal Historical Observations is advantageous over simply choosing all Historical Observations. While the results presented here are for ILI within Syria, our novel future spaces shows promise to be easily extended to accurately track not only influenza in other countries but also other infectious diseases Through careful tuning of the values of n and m. We believe that it is up to policymakers in organizations concerned with EWARS to decide whether these forecasts are ready for use in decision support at the current level of accuracy.
In conclusion, we performed Naïve, Average, Seasonal naïve, Drift, DHR, STL, TBATS, GLM, SVR, GB, RF and LSTM methods with different feature spaces to predict the weekly ILI rate using EWARS data from WHO in Syria. We found that the TBATS method with time series feature space gave better results than those resulted from all methods in \(53-weeks-before\_52-first-order-differences\) feature space. We also found that the GLM, SVR, GB, RF and LSTM methods with a good choice to n and m in \(n-years-before\_m-weeks-around\) feature space gave better results than those resulted in \(53-weeks-before\_52-first-order-differences\) feature space. In all the models, the LSTM model of 4 layers reached the lowest MAPE (3.52%) and the lowest RMSE (0.01662).
Climatological data and Pharmaceutical Sales could be used to achieve better performance of the used models. We would like to investigate the impact of these parameters in future work.
This work is part of master degree research about “health map prediction in Syria” in Higher Institute for Applied Science and Technology.
AD conceived the research topic, explored that idea, provided the data, performed the statistical analysis and drafted the manuscript. YR and AJ contributed as supervisor to work and participated in the conception of the research topic, exploration the idea. YR participated in data analysis and writing. AJ participated in performing the statistical analysis and drafted the manuscript. All authors read and approved the final manuscript.
Higher Institute for Applied Sciences and Technology provided technical support for the present study, by providing the personal computer used for the experiments. The funder had no role in the design of the study and collection, analysis, and interpretation of data and in writing the manuscript and decision to publish.
Ethics approval and consent to participate
Consent for publication
The authors declare that they have no competing interests.
- 1.World Health Organization. Influenza (Seasonal) Fact Sheet. http://www.emro.who.int/syr/publications-other/ewars-weekly-bulletin-2014.html. Accessed 16 July 2019.
- 2.World Health Organization Regional Office for the Eastern Mediterranean. Syrian Arab Republic—EWARS: The Early Warning Alert and Response System. http://www.emro.who.int/syr/publications-other/ewars-weekly-bulletin.html. Accessed 16 July 2019.
- 8.Biggerstaff M, Johansson M, Alper D, Brooks LC, Chakraborty P, Farrow DC, Hyun S, Kandula S, McGowan C, Ramakrishnan N, et al. Results from the second year of a collaborative effort to forecast influenza seasons in the united states. Epidemics. 2018;24:26–33.PubMedPubMedCentralCrossRefGoogle Scholar
- 9.Gao H, Wong KK, Zheteyeva Y, Shi J, Uzicanin A, Rainey JJ. Comparing observed with predicted weekly influenza-like illness rates during the winter holiday break, united states, 2004–2013. PLoS ONE. 2015;10(12):0143791.Google Scholar
- 20.Lee K, Agrawal A, Choudhary A. Forecasting influenza levels using real-time social media streams. In: 2017 IEEE international conference on healthcare informatics (ICHI). New York: IEEE; 2017. p. 409–14.Google Scholar
- 21.Paul MJ, Dredze M, Broniatowski D. Twitter improves influenza forecasting. PLoS Curr. 2014. https://doi.org/10.1371/currents.outbreaks.90b9ed0f59bae4ccaa683a39865d9117.PubMedPubMedCentralCrossRefGoogle Scholar
- 29.Burdakov A, Ukharov A, Myalkin M, Terekhov V. Forecasting of influenza-like illness incidence in amur region with neural networks. In: International conference on neuroinformatics. Berlin: Springer; 2018. p. 307–14.Google Scholar
- 30.World Health Organization Regional Office for the Eastern Mediterranean. Syrian Arab Republic—EWARS Weekly Bulletin. http://www.emro.who.int/syr/publications-other/ewars-weekly-bulletin-2014.html, http://www.emro.who.int/syr/publications-other/ewars-weekly-bulletin-2015.html, http://www.emro.who.int/syr/information-resources/2016-ewars-bulletins.html, http://www.emro.who.int/syr/ewars-workshops/ewars-bulletins-2017.html, http://www.emro.who.int/syr/information-resources/ewars-weekly-bulletins-2018.html. Accessed 16 July 2019.
Open AccessThis article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.