Introduction

Big Data, characterized by large volumes, high processing speed, and wide variety of datasets [1,2,3], have been shown to be very valuable in health care research, with Health Informatics being the field in which big data analytics have been extensively applied [4]. A popular way of addressing the challenge of Big Data is the analysis of online search traffic data [5, 6], mainly with data from Google Trends [7]. Over the past decade, this field of research, i.e., analyzing online search traffic data, has been widely used and is growing in popularity for assessing various topics, though it has mostly focused on the fields of Health and Medicine [8].

Many studies on the subject have empirically shown that Google Trends’ data are related to public health data. Topics that have been explored up to this point include the analysis, assessment, and prediction of epidemics and outbreaks, as, for example, Ebola [9, 10], Measles [11], the Bed-Bug epidemic [12], and Tuberculosis [13]. A much studied topic is that of influenza like illness (the flu), which is a seasonal disease and has shown well performing results in the past [14,15,16,17].

Recently, more topics on relating Google data with official health data have been visited, as in the case of suicide rates, where it has been show that Google queries can be used to monitor the risk of suicide [18, 19]. On a different direction, there has been shown that correlations exist between Google Trends data and prescription drugs issuing [20, 21] and revenues [22]. Apart from prescription drugs, focus has been given to illegal drugs as well, with notable examples including the tracking of dabbing in the US [23], Krokodil in Russia [24], and Methamphetamines in Central Europe [25].

According to Infodemiology [26], data available on the Internet can be used to inform public health and policy by monitoring the public’s behavior towards diseases, selecting the relevant available information, as well as monitoring how the public reacts to health marketing campaigns. Though it is widely supported and evident that official health data and online search traffic data are correlated, the most important step towards health assessment using Google Trends is that of finding methods of predicting and nowcasting diseases’ occurrence and outbreaks, as well as forecasting seasonal diseases’ prevalence.

Though seasonality has been assessed in various cases, such as, for information on tobacco and lung cancer [27], the restless legs syndrome [28], and in sleep-disordered breathing [29], studies developing methods towards the direction of forecasting and nowcasting exhibit significantly lower numbers. Despite that, recent research has exhibited promising results in the forecasting of various diseases and outbreaks, as, for example, Tuberculosis [13], influenza like illness [17], pertussis [30], suicide risk [18], and dementia [31].

As Infodemiology data can be retrieved in real time and thus allow the nowcasting of human behavior based on Internet data, the detection, monitoring, and prediction of epidemics and outbreaks can be much assisted by the analysis of Google queries. A topic that is of high significance and interest is that of AIDS (Acquired Immune Deficiency Syndrome) and HIV (Human Immunodeficiency Virus). HIV is a virus that is mainly transmitted via sexual intercourse and needle/syringe use [32]. The treatment for HIV consists of the antiretroviral therapy, which controls the HIV virus. If the HIV remains without treatment, it affects the immune system, which worsens as time passes. The HIV infection consists of 3 stages: (1) acute HIV infection, (2) clinical latency, and (3) AIDS; the latter being the most severe stage of the HIV infection [33], which leads to an increased number of ‘opportunistic infections’ [32].

People would more easily search for information online than consult a doctor in general. In the case of AIDS, as it is a sensitive subject, the anonymity provided by the Internet allows people to search for information online. Thus the monitoring of Internet data is essential in the overall assessment of AIDS prevalence in regions where Internet penetration is high, as in the case of the United States. Novel methods of assessment are needed, as data on ‘AIDS Prevalence’, ‘AIDS Diagnoses’, and ‘AIDS Deaths’ provided by the Centers for Disease Control and Prevention (CDC) are not available in real time, as gathering, analyzing and making these data available is a long process that takes over a year.

AIDS is categorized as an epidemic [34], and as such it needs constant assessment. The aim of this paper is to analyze the online interest in AIDS related terms and estimate forecasting models for AIDS prevalence in the US using data from Google Trends. The rest of this paper is structured as follows: the “Research methodology” section consists of the procedure of the data collection and methodology followed to analyze and forecast AIDS prevalence, in the “Results” section the results of the analysis are presented, the “Discussion” section consists of the discussion of the analysis, while the “Conclusions” section consists of the overall conclusions and future research suggestions.

Research methodology

Data

Data from Google Trends are downloaded online in ‘.csv’ format and are normalized over the selected time-frame as follows: “Search results are proportionate to the time and location of a query: Each data point is divided by the total searches of the geography and time range it represents, to compare relative popularity. Otherwise places with the most search volume would always be ranked highest. The resulting numbers are then scaled on a range of 0–100 based on a topic’s proportion to all searches on all topics. Different regions that show the same number of searches for a term will not always have the same total search volumes.” [35]. Google Trends is not case-sensitive, though takes into account spelling errors and accents. In this study, this effect is minimized, as the examined term, i.e. AIDS, is universal, not translated, and difficult to misspell. Note that data may slightly vary when retrieved at different time points.

Methods

The choice of terms is crucial for the robustness of the results when using online data [36]. In Google Trends, the four options below are available when retrieving data for the examined disease. The term’s online interest can be retrieved in the ‘Search Term’ form, i.e. include all queries that had the respective term, thereafter referred to as ‘AIDS (Search Term)’. In addition, Google Trends groups related queries under other search terms as well, which in this case are ‘AIDS (Illness)’. Finally, Google Trends also gives the option of including terms related to the topics of ‘Management of AIDS/HIV (Topic)’, and ‘Diagnosis of HIV/AIDS (Topic)’.

Analysis stages

At first, an overall assessment of all four available terms and topics’ variations in online interest is provided, so as to identify the option that would increase the validity of further analysis on the subject. The next step towards examining the possibility of forecasting AIDS prevalence and incidence, is to identify any existing correlations between Google data on related terms and topics and official health data for AIDS. In this study, data on 'AIDS Prevalence' (2004–2015) are retrieved by the CDC website [37]. Depending on the significance of the calculated Pearson correlations, the possibility of forecasting AIDS prevalence in the US will be assessed. Finally, forecasting models of AIDS prevalence based on Google Trends’ data for the US as well as for each 50 States plus DC are estimated.

Results

At first, an overall assessment of the online interest towards AIDS in the US is performed, followed by the exploring of the correlations between AIDS prevalence and Google Trends data in the US and each US State individually. Finally, forecasting models for AIDS prevalence in the US are estimated, at both national and State level, so as to elaborate on the usefulness of the tool in health assessment in the US.

AIDS online interest in the US

Figure 1 consists of the changes in the online interest in ‘AIDS (Search Terms)’ and ‘AIDS (Illness)’ from January 2004 to December 2015, while Fig. 2 depicts the monthly normalized online interest in the ‘Google Trends’ topics of ‘Diagnosis of HIV/AIDS’ and ‘Management of HIV/AIDS’ from January 2004 to December 2015.

Fig. 1
figure 1

Monthly Normalized Google Trends’ Data for ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015

Full size image
Fig. 2
figure 2

Monthly normalized Google Trends’ Data for ‘Diagnosis of HIV/AIDS (Topic)’ and ‘Management of HIV/AIDS (Topic)’ from January 2004 to December 2015

Full size image

The top related queries for ‘AIDS (Search Term)’ include ‘aids hiv’ (100), ‘hearing aids’ (99), ‘hiv’ (97), ‘aids symptoms’ (33), ‘aids and hiv’ (25), ‘aids day’ (24), ‘africa aids’ (22), ‘aids cure’ (16), ‘aids test’ (11), ‘aids statistics’ (11), and ‘aids virus’ (10). For ‘AIDS (Illness)’, the top related queries include ‘aids’ (100), ‘hiv’ (26), ‘aids hiv’ (14), ‘hiv/aids’ (6), ‘aids symptoms’ (5), ‘africa’ (4), ‘aids day’ (4), ‘hiv symptoms’ (3), ‘aids cure’ (2), ‘hiv infection’ (2), ‘hiv transmission’ (2), and ‘aids statistics’ (2).

For the topic of ‘Diagnosis of HIV/AIDS’, the top related queries include ‘hiv’ (100), ‘hiv test’ (53), ‘hiv testing’ (50), ‘free hiv testing’ (13), ‘test for hiv’ (11), ‘hiv symptoms’ (9), ‘hiv home test’ (7), ‘aids’ (6), ‘hiv aids’ (6), ‘hiv rapid test’ (4), ‘free hiv test’ (4), ‘hiv positive’ (4), ‘hiv test results’ (4), ‘positive hiv test’ (3), ‘rapid hiv testing’ (3), ‘hiv test kit’ (3), and ‘oraquick hiv test’ (2). For the topic ‘Management of HIV/AIDS’, the top related queries include ‘antiretroviral’ (100), ‘hiv’ (86), ‘aids’ (59), ‘antiretroviral therapy’ (58), ‘aids drugs’ (38), ‘antiretrovirals’ (28), ‘hiv treatment’ (23), ‘antiretroviral treatment’ (22), ‘hiv aids’ (20), ‘antiretroviral drugs’ (16), ‘hiv management’ (12), ‘highly active antiretroviral therapy’ (7), and ‘hiv medications’ (4).

Figure 3 consists of the heat maps of the online interest by US State from January 2004 to December 2015 for ‘AIDS (Search Term)’, ‘AIDS (Illness)’, ‘Diagnosis of HIV/AIDS (Topic)’, and ‘Management of HIV/AIDS (Topic)’.

Fig. 3
figure 3

Online Interest by State for ‘AIDS (Search Term)’, ‘AIDS (Illness)’, ‘Diagnosis of HIV/AIDS (Topic)’, and ‘Management of HIV/AIDS (Topic)’ from January 2004 to December 2015

Full size image

It is evident that the terms related to AIDS exhibit high and constant interest from 2004 to 2015. The topics of ‘Diagnosis of HIV/AIDS (Topic)’ and ‘Management of HIV/AIDS (Topic)’ cover a narrow range of AIDS related terms and will thus not be included in further analysis.

AIDS prevalence vs. Google Trends

In order to examine the possibility of forecasting AIDS prevalence in the US, the relationships between online search traffic data from Google and official health data on AIDS prevalence are at first examined, by calculating the respective correlations at both national and State level. Depending on the significance of the correlations, the possibility of forecasting AIDS prevalence in the US will be examined. For the analysis of AIDS related queries, both Google Trends categories, i.e. ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, are analyzed. Data for the categories ‘AIDS Deaths’, ‘AIDS Diagnoses’, and ‘AIDS Prevalence’ are available for 12 years, i.e. from January 2004 to December 2015.

Statistically significant correlations are observed between ‘AIDS Prevalence’ with both ‘AIDS (Search Term)’ (r = − 0.9508, p < 0.01) and with ‘AIDS (Illness)’ (r = − 0.9615, p < 0.01) in the US. For ‘AIDS (Search Term)’, statistically significant correlations are observed with ‘AIDS Diagnoses’ (r = 0.8743, p < 0.01), and with ‘AIDS Deaths’ (r = 0.9343, p < 0.01). Significant correlations are also identified for ‘AIDS Diagnoses’ with ‘AIDS (Illness)’ (r = 0.8945, p < 0.01), and for ‘AIDS Deaths’ with ‘AIDS (Illness)’ (r = 0.9423, p < 0.01). Therefore, we proceed to the next step of identifying correlations between online and health data in each US State.

Table 1 consists of the Pearson correlation coefficients (r) between ‘AIDS Prevalence’ and (a) ‘AIDS (Search Term)’ and (b) ‘AIDS (Illness)’ from January 2004 to December 2015, while Table 2 consists of the Pearson correlation coefficients (r) between ‘AIDS Diagnoses’ and a) ‘AIDS (Search Term)’ and b) ‘AIDS (Illness)’ from January 2004 to December 2015. Table 3 consists of the Pearson correlation coefficients (r) between ‘AIDS Deaths’, and a) ‘AIDS (Search Term)’ and b) ‘AIDS (Illness)’ from January 2004 to December 2015.

Table 1 Pearson correlation coefficients between ‘AIDS Prevalence’ and ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015
Full size table
Table 2 Pearson correlation coefficients between ‘AIDS Diagnoses’ and (a) AIDS (Search Term)’ and (b) ‘AIDS (Illness)’ from January 2004 to December 2015
Full size table
Table 3 Pearson correlation coefficients between ‘AIDS Deaths’ and (a) AIDS (Search Term)’ and (b) ‘AIDS (Illness)’ from January 2004 to December 2015
Full size table

For ‘AIDS Prevalence’, all correlations are statistically significant. Therefore it is evident that the online behavior towards AIDS follows that of ‘AIDS Prevalence’. Thus the States that exhibit statistically significant correlations are further selected for the forecasting of AIDS in the US.

For ‘AIDS Diagnoses’, the States with significance of correlation of p < 0.01 in both examined terms are Arkansas, California, Connecticut, Delaware, DC, Florida, Illinois, Indiana, Maine, Maryland, Massachusetts, Michigan, Minnesota, Missouri, Nevada, New Hampshire, New Jersey, New York, Oklahoma, Oregon, Pennsylvania, Rhode Island, South Carolina, Washington, and West Virginia. For ‘AIDS Deaths’, the respective States are Arizona, California, Connecticut, Delaware, DC, Florida, Georgia, Illinois, Louisiana, Maryland, Massachusetts, Michigan, Mississippi, Missouri, New Jersey, New York, Pennsylvania, Tennessee, Texas, Utah, and Washington.

Forecasting AIDS prevalence in USA

As ‘AIDS Prevalence’ data are highly correlated with both ‘AIDS (Search Term)’ and with ‘AIDS (Illness)’ in all 50 States (plus DC), the next step is to examine the relationships between Google data and AIDS data and estimate the forecasting models. The relationship is logarithmic and of the form \(y\, = \,\alpha { \ln }\left( x \right)\, + \,\beta\), where y (y-axis-dependent variable) denotes the ‘AIDS Prevalence’, x (x-axis-independent variable) denotes the respective Google Trends’ data, namely ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, and α and β are constants. To elaborate on the robustness of the estimated models, the R2 is selected, as it is the statistical measure by which the variable variation is explained. R2 takes values between 0 and 1 (i.e. 0% to 100%), and the higher the percentage, the better the fit.

Table 4 consists of the coefficients for the estimated logarithmic models for ‘AIDS Prevalence’ for both the examined Google Trends’ terms, i.e. ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, while Figs. 4, 5, 6 and 7 depict the respective relationships in the US and in each individual State.

Table 4 Regression coefficients and R2 for the estimated forecasting models for ‘AIDS Prevalence’
Full size table
Fig. 4
figure 4

‘AIDS Prevalence’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015 (USA; Alabama–Idaho)

Full size image
Fig. 5
figure 5

‘AIDS Prevalence’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015 (Illinois–Montana)

Full size image
Fig. 6
figure 6

‘AIDS Prevalence’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015 (Nebraska–South Carolina)

Full size image
Fig. 7
figure 7

‘AIDS Prevalence’ vs. Google Trends ‘AIDS (Search Term)’ and ‘AIDS (Illness)’ from January 2004 to December 2015 (South Dakota–Wyoming)

Full size image

In the US, the estimated models for ‘AIDS Prevalence’ based on the two examined terms have an R2 of 0.9695 and 0.9844, which shows that the relationship between AIDS prevalence and Google Trends data is well described using the estimated equations and that AIDS prevalence can be predicted based on online search traffic data from Google. Furthermore, most States’ models exhibit high R2 in at least one Google Trends’ category, which is indicative of the significance of the estimated forecasting models of AIDS prevalence in the US States.

Though in several States the R2 is higher for the respective linear or polynomial forecasting model, the relationship is overall logarithmic as clearly shown in the case of the US. Therefore, all estimated models for all categories and all individual States are calculated based on a logarithmic relationship independent of the value of R2, as this will be more evident when more years’ data are available.

The categories ‘AIDS Diagnoses’ and ‘AIDS Deaths’, though significant correlations with Google data are identified, are not included in further analysis, as the results are not significant for all States, though the respective analyses on said categories can be found in Appendix 1 and Appendix 2.

Discussion

The AIDS epidemic is a serious health issue and needs immediate and constant attention. In the Internet age, new methods for the monitoring and assessment of AIDS are required, so as to decrease the numbers of AIDS prevalence and deaths around the globe, and especially in developing countries. In this study, we provide a novel approach of monitoring online search traffic data retrieved from Google Trends in order to develop forecasting models for AIDS prevalence in the US.

Both examined Google terms, i.e. ‘AIDS (Search Term)’ and ‘AIDS (Illness)’, exhibited significant correlations with official data on ‘AIDS Prevalence’, ‘AIDS Diagnoses’, and ‘AIDS Deaths’, especially in the States where the AIDS rates are higher. Despite previous concerns on the reliability of Google Trends data [38], our results support research over the last decade showing that empirical relationships widely exist between Google Trends’ data and public health data records [5, 6, 9, 11, 20,21,22, 26, 39,40,41,42]. Therefore, the forecasting of AIDS prevalence is possible, as the estimated models for several States are robust despite the limitation of data being available for only 12 years. For ‘HIV (Search Term)’ and ‘HIV (Illness)’, though search volumes are high throughout the examined period, the correlations with official HIV data were not as statistically significant as in the case of AIDS, and were identified in fewer US States, which is an interesting topic to be examined in future research on the subject.

Table 5 consists of the coefficients and the R2 for the estimated forecasting logarithmic forecasting models of the form \(y\, = \,\alpha { \ln }\left( x \right)\, + \,\beta\) for States that exhibit high significance in all three categories, i.e. ‘AIDS Prevalence’, ‘AIDS Diagnoses’, and ‘AIDS Deaths’.

Table 5 Estimated Logarithmic forecasting models for USA and selected states
Full size table

This study has some limitations. The estimated forecasting models are based on only 12 years’ data, thus the robustness of the models will increase when more years or smaller interval data are made officially available. In addition, we do not argue that each hit on the AIDS related keywords corresponds to an AIDS case and vise versa, as hits can also be attributed to general or academic interest, or increased interest due to an event, incident, or public figure that announces something related to the disease. Overall, the online interest towards AIDS increases according to the rates of AIDS prevalence (Appendix 3), thus it is expected for the forecasting models to be robust in the States for which the rates—and the online interest—are increased. Therefore, when more data are available, the significance will most probably increase.

Overall, this study highlights the importance of the analysis of online queries in order to better and more timely assess various issues in the US Health Care System. The estimated forecasting models on AIDS prevalence have very good performance, indicating that Google data can be of value in dealing with this sensitive subject, as we can this way have access to data that would not easily or at all been accessed with conventional methods.

Conclusions

This study aimed at introducing a novel approach in forecasting AIDS prevalence in the US using data from Google Trends on related terms. The results, exhibiting significant correlations between Google Trends’ data and official health data on AIDS (2004–2015) and high significance of the estimated forecasting models in several US States, support previous work on the subject suggesting that Google Trends’ data have been shown to be empirically related to health data and that they can assist with the analysis, monitoring, and forecasting of several health topics. This study, however, also addresses a more important issue; that of anonymity. A Google Trends important advantage is that it uses the revealed and not the stated data [37] in general, but in the case of AIDS the latter is even more important. As HIV and AIDS testing, diagnosis, and treatment is a sensitive subject, people may less easily go to the hospital or consult a doctor, health official, especially before testing and diagnosis.

Therefore, the monitoring of the interest towards States with increased rates of AIDS prevalence is essential, so that health officials can a) make relative information available on the Internet at time point e.g. with advertisements, b) take preventive measures, e.g. organizing event etc., and c) prepare the Health Care System accordingly, e.g. organize free testing outside of the hospitals. AIDS and HIV are terms that are not translated, not easily misspelled, and do not include accents or special characters. Thus, future research can include the application of this method in other countries and regions, as well as taking into consideration data retrieved by other online sources.