Introduction

Economists and policy-makers have traditionally considered gross domestic product (GDP) as a good indicator of well-being in society, mainly because it is strongly linked with the standard of living indicators [1]. However, GDP has been criticized as a weak indicator of well-being and, therefore, a misleading tool for public policies [2]. The Stiglitz Commission [3] in 2009 observed that other statistical tools should be used, complementary to GDP, for the measurement of well-being. Therefore, considering that well-being is difficult to be captured only with GDP, researchers with various backgrounds, from economists to psychologists, suggested two main approaches to measuring the overall well-being; objective well-being and subjective well-being.

Defining objective well-being has always been considered a challenging task, and therefore researchers have focused on exploring its dimensions rather than its definition [4, 5]. It is due to its objective nature that one could claim that objective well-being could be measured in terms of GDP. However, it must reflect both people’s material living conditions and the quality of their lives. In fact, the Organisation for Economic Co-operation and Development (OECD) [6], the United Nations Development Programme (UNDP) [7] and the Italian Statistics Bureau (ISTAT) [8] have identified six major objective and observable dimensions for its measurement: health, job opportunities, socioeconomic development, environment, safety, and politics. All these dimensions together represent the objective well-being, which is assessed through the extent to which these “needs” are satisfied. The objective approach investigates the objective dimensions of a good life, whereas the subjective approach examines people’s subjective evaluations of their own lives. In 2013, the OECD [9] recognized the importance of taking into consideration people’s perceived well-being, labeled as subjective well-being when investigating the overall well-being. Subjective well-being, also called happiness, has been defined by Veenhoven [10], as the degree to which an individual assesses the overall quality of her life-as-a-whole favorably. This might as well be different as compared to GDP, which cannot be representative of societal happiness. Indeed, GDP explains only a small proportion of its variations on humans [11], and it might be different from people’s perceptions of their well-being [12]. Therefore, subjective well-being has been traditionally captured through studies based on data collected by self-reports. These studies highlight five main dimensions of subjective well-being: the role of human genes, which seem to be fairly heritable [13,14,15,16,17,18,19,20,21], universal needs, meaning basic and psychological needs [22,23,24], social environment, such as education and health [25,26,27,28,29], economic environment, including a lot of research on income [30,31,32,33,34], and political environment, such as democracy and political freedom [35, 36].

Traditionally, both objective and subjective well-being are measured through surveys of household income and consumption [37]. Although these surveys have been considered accurate and valid, they bring some considerable disadvantages. For example, they cannot provide constant updates of well-being to policy-makers, and they have high costs to be conducted, making it difficult for many developing countries to estimate well-being frequently. The last few years have witnessed a drastic change in the approaches used to measure well-being. Researchers of different disciplines propose several innovative data sources and methods, which could potentially overcome the limitations of the traditional methods for the individual and collective well-being measurement, both objective and subjective.

To support research in this direction, the European project SoBigData [38] has created a virtual environment within a research infrastructure that provides theoretical knowledge, data, and innovative methods to scholars that want to address challenging questions involving both objective and subjective well-being.

Therefore, in line with the purposes mentioned above and the support of SoBigData, the aim of this paper is to provide the theoretical background on objective and subjective well-being, including their relevant dimensions. Additionally, the article seeks to present to researchers the new data sources used for capturing well-being, as well as discuss indicative existing studies.

We believe that this study offers great value to the scientific community and especially to researchers interested in “Data Science for Social Good” (DS4SG) or similarly “Artificial Intelligence for Social Good” (AI4SG) [39], since it could work as a reference point for adequate measurement of well-being with the use of innovative data sources and tools. In particular, at this critical moment that the global society is under financial and political crisis and instability, policy-makers need frequent updates of well-being. This could facilitate them to react on time on applying the right policies to prevent detrimental societal effects and contribute effectively to societal progress.

The remainder of this paper is organized as follows: It is divided into two main sections, as suggested from the literature, i.e. objective and subjective well-being. In particular, Sect. 2 is dedicated to objective well-being and Sect. 3 is dedicated to subjective well-being. For both sections, we provide a theoretical background on objective and subjective well-being and their dimensions respectively. We then provide the data sources used for monitoring well-being. Besides, we present essential studies on well-being; to present them in an organized flow, we categorize the presentation of the studies by matching each well-being dimension separately with each data source. Finally, in Sect. 4, we provide a discussion on the study, highlighting the opportunities for future research on well-being.

Measuring objective well-being

Suggesting a single definition of objective well-being is a substantial challenge, mainly due to its multi-dimensionality. Therefore, researchers have focused on carefully specifying its objectively measurable dimensions [4, 5]. Objective well-being is traditionally captured through surveys, such as household income and consumption surveys [37]. However, usually, such surveys are very costly and time-consuming [40], making it difficult for many countries and global institutes to update their estimates frequently. Therefore, the last few years have witnessed a change in the way of measuring objective well-being. In particular, researchers of various disciplines propose several methodologies to measure individual and collective objective well-being, based on a combination of new data sources and traditional surveys [41,42,43,44]. The United Nations also stimulate this change of studying well-being in two recent reports, where the usage of new, mostly big, data sources, is encouraged for the investigation of patterns of phenomena related to people’s health and well-being [45, 46].

Fig. 1
figure 1

The figure relates the sources of data (left) with the dimensions of the objective well-being (right)

The dimensions of objective well-being

During the last years, public institutions and non-governmental companies have worked on identifying dimensions that are considered essential for the improvement of the societal well-being and its comparison between countries and years. For example, the Organisation for Economic Co-operation and Development (OECD) has identified 11 essential topics labeled as OECD well-being framework [6]; the United Nations Development Programme (UNDP) has identified 17 sustainable development goals, labeled as SDGs [7]; and the Italian Statistics Bureau (ISTAT) has created an ambitious project named “Benessere Equo e Sostenibile” (BES) that stands for “Fair and Sustainable Well-being” [8]. From the initiatives mentioned above, it is evident that for different institutions, well-being dimensions might be different, sometimes vague, and statistically hard to be captured. Therefore, based on the aforementioned official authorities, we suggest the following concrete and measurable dimensions of well-being (Fig. 1).

Health

Health status represents an essential factor for people’s well-being, as shown by the WHO Commission on Macroeconomics and Health in 2001 at global level [47], and by the Lisbon Strategy for Growth and Jobs in 2000 [48]. Health brings together many other benefits, from job opportunities to social relationships, from reduced health care costs to an increased life expectancy. Indeed, there have been remarkable gains in life expectancy over the past 50 years in OECD countries [49], due to the health care spending growth, lifestyle, educational, and environmental changes. Chronic (non-communicable) diseases, such as cancer, diabetes, and chronic respiratory conditions, are nowadays the primary disability and mortality factors in OECD countries. Fortunately, some indicators can help prevent the diseases mentioned above. For example, the number of people who are driving carefully, who are non-smokers or who do not drink a large amount of alcohol, are risk-indicators, which, if taken into consideration, could contribute to an improvement in the health status of a territory.

Job opportunities

This is a crucial dimension of well-being since it has obvious economic and societal benefits, contributing to people’s health and societal, political, and economic stability. The job opportunities dimension is composed of three main determinants: employment rate, quality of work, and work–life balance. The employment rate is a crucial aspect since individuals in countries with a high level of employment, are well connected in society. In particular, it is a proxy used by policy-makers to avoid poverty and social exclusion. The second determinant is the quality of work, in terms of objective working stability, retribution, skills, and safety at work, which might show some differences between different working environments. Moreover, work-life balance is the determinant that mainly aims to capture the balance between work and life. In the OECD countries, a full-time worker devotes 62% of the day on average (15 hours) on personal care (e.g., eating, sleeping) and leisure (e.g., socializing with friends and family, hobbies) [50]. This determinant is mainly created to capture women’s work-life balance. Indeed, the quality of a country’s employment is measured by the balance women have between family care and paid work.

Socioeconomic development

While socioeconomic indicators alone do not suffice to represent societal well-being, it cannot be doubted that they positively influence it. The variables that contribute to its measurement are income, wealth, consumption expenditure, housing conditions, and possession of consumer durables, and it can implicitly influence access to university, health care, and more. In particular, the Organization for Economic Co-operation and Development (OECD) [6] and the Italian Statistics Bureau (ISTAT) [8] suggest two main determinants that constitute the overall economic well-being: available income and wealth, and consumption expenditure.

In a market economy, income measures the purchasing capacity of individuals, and it is thus an essential predictor of economic well-being. Wealth, on the other hand, takes into account savings, monetary gold, stocks, securities, and loans [51]. Therefore, wealth could be considered an essential source of revenue, which could make people less vulnerable to difficult economic situations that might affect their life.

Additionally, consumption expenditure is a direct estimate of the goods and services that contribute to determining the living conditions of individuals. Unlike income, consumption expenditure can contribute to making interpersonal comparisons, since it captures whether each individual can acquire her desired goods and services.

Environment

A healthy natural environment is essential for all individuals’ well-being in society. Clean water, clear air, and uncontaminated food are examples of goods that can only be possible in an environmental context where humans’ productive and social activities are made with respect to the environment and its natural resources. For the reasons mentioned above and due to the recent environmental crisis, the United Nations set sustainable environmental goals [7], such as Clean Water and Sanitation, Climate Action, and more. Similarly, ISTAT [8] suggests five determinants for describing the interactions between society and the environment that are connected. These determinants are quality of the water, quality of the air, quality of the soil and the land, biodiversity, and matter, energy, and climate change. Finally, the “OECD Environmental Outlook to 2050” projects the number of premature deaths associated with exposure to PM10 and PM2.5 to increase from just over 1 million worldwide in 2000 to about 3.5 million in 2050 [52]. Therefore, the more these determinants are taken into consideration by policy-makers and by citizens’ activities, the more the citizens can contribute to radical changes for the protection of societal well-being.

Safety

It includes the risk of people being physically assaulted, falling victims, and suffering from other crimes, such as economic loss, physical damage, and psychological post-traumas stress. Reducing violent crime, sex trafficking, forced labor, and child abuse are clear global goals, as suggested by the United Nations [7]. Besides, the Italian BES project [8] suggests that safety is characterized by two determinants: criminality and violence.

Criminality is one of the most common security threats in developed and emerging countries, and it has both a direct and indirect impact on people. It directly influences individuals’ health (physical and mental) and economic situation. According to the latest OECD data, the average homicide rate in the OECD is 3.6 murders per 100,000 inhabitants [53]. Indirectly, criminality has an impact on non-victims’ well-being when being on victims’ social network or by news spread on (social) media.

Another determinant is violence suffered inside and outside the family and it has both a direct and indirect impact on people. In particular, victims suffer from the direct effects, which can last for long periods, if not for the whole life, depending on individuals’ ability to manage their daily life, medical expenses, dependence on others, and capacity to achieve happiness. Indirectly, it causes insecurity and anxiety, which brings difficulties in their daily activities [54].

Politics

This dimension is also essential for objective well-being. Today, due to the economic crisis, more than ever, citizens demand greater transparency from their Governments and the Public Institutions. Fair civic and political participation, as well as transparency, do not only contribute directly to well-being but also indirectly since they allow greater efficiency of public policies, a lower cost of transactions, and the minimization of the risk of fraud. Therefore, two determinants fall under this category, which are associated with the Public Sphere as a driver of the individuals’ well-being, on either local or national level: civic and political engagement, and trust and social cohesion. Voter turnout is the best existing means of measuring civic and political engagement, and is measured as the percentage of the registered population that voted during elections. According to OECD data, voter turnout, is averaged 69% in OECD countries, which shows that not everyone exercises the voting right [55]. Regarding trust and social cohesion, OECD suggests public engagement (e.g., stakeholder engagement) for developing regulations [55]. If citizens have the possibility to participate in the development of laws and regulations, it is more likely that they will trust the government institutions and they will comply with the societal rules.

Table 1 Pros and cons for each data source used for the measurement of objective well-being

Data sources for monitoring the dimensions of objective well-being

Figure 1 describes the new data sources (left) that have been used to estimate one or more dimensions of objective well-being (right). The presence of a link in Fig. 1 between a data source and a dimension indicates that there are papers in the literature on monitoring that dimension with that data source. In this section, we describe, for each data source, its features (e.g., the process of data collection, its biases and limitations) and the main works in the literature that use it to measure several dimensions of objective well-being. Table 1 provides a summary of the data sources used, highlighting the pros and cons of each one. We refer to a link between a data source and a dimension using a letter-number notation. For example, B3 indicates the link between GPS data (B) and socioeconomic development (3).

CDRs

Many works in the literature are based on the analysis of mobile phone data, the so-called CDRs (Call Detail Records) of calling and texting activity of users, because they guarantee the repeatability of experiments in different countries and on different scales given the worldwide diffusion of mobile phones [56].

CDRs collect geographical, temporal, and interaction information on mobile phone use [57,58,59,60,61,62], hence providing a comprehensive picture of human behavior at a societal scale. Each time an individual makes a call, the mobile phone operator registers the connection between the caller and the callee, the duration of the call, and the coordinates of the phone tower communicating with the served phone. Table 2 illustrates an example of the structure of CDRs.

Table 2 Example of Call Detail Records (CDRs). Every time a user makes a call, a record is created with timestamp, the phone tower serving the call, the caller identifier and the callee identifier (a). For each tower, the latitude and longitude coordinates are available to map the tower on the territory (b)

Note that CDRs suffer from different types of bias [63, 64]. For example, the position of a user is known at the granularity level of phone towers, and only when they make a phone call. Moreover, phone calls are sparse in time, i.e., the time between consecutive calls follows a heavy tail distribution [65, 66]. In other words, since users are inactive most of their time, CDRs allow reconstructing only a subset of a user’s behavior.

CDRs are used to monitor several dimensions of well-being, notably health (A1), job opportunities (A2), socioeconomic development (A3), environment (A4), and safety (A5).

CDRs provide one of today’s most exciting opportunities to study human mobility and its influence on disease dynamics (A1). Many researchers use mobile phone data for public health, as the analysis of individual and population mobility patterns is more objective and with finer spatiotemporal resolution in comparison to traditional methods. Furthermore, mobile network data can also provide insights into human behavior that can support the assessment and monitoring of the health of specific communities at risk, thus paving the way toward improved health promotion and prevention [67]. Taking into consideration that the spatiotemporal evolution of human mobility and the related fluctuations of population density are essential drivers of disease outbreaks, Finger et al. [68] use CDRs to track the cholera outbreak in 2005 in Senegal. Findings show that a mass gathering taking place during the initial phase of the outbreak has an essential impact on the course of the disease. Besides, Kafsi et al. [69] contribute to the fight against epidemics of infectious diseases using CDRs provided by France Telecom-Orange. They use 2.5 billion calls made by 5 million users in the Ivory Coast, recorded over 5 months, from December 2011 to April 2012, to study and model behavioral patterns of the affected population and propose several strategies for personalized behavioral recommendations to reduce the infections. Lima et al. [70] use the same data set to build a model that describes how diseases circulate the country as people move between regions, and they enhance the model with a concurrent process of relevant information spreading. This process corresponds to people disseminating disease prevention information, e.g., hygiene practices, vaccination campaign notices, and others, within their social network. Finally, Madan et al. [71] use CDRs and mobile phone-based co-location sensing to measure characteristic behavior changes in symptomatic individuals. These behavior changes are reflected in their total communication, interactions with respect to time of day, diversity, and entropy of face-to-face interactions and movement. Using these extracted mobile features, they manage to predict the health status of an individual, without having actual health measurements from the subject.

Besides, researchers use CDRs to study job opportunities (A2). Pappalardo et al. [72] use CDRs to study the link between human mobility and the employment rate of French cities, finding a strong correlation between measures of mobility entropy and the unemployment rate in urban environments. Toole et al. [73] show that changes in the calling behaviors of individuals, aggregated at regional level, can improve forecasts of macro unemployment rates. Sunds et al. [74], use CDRs to create a model which predicts unemployment with a 70.4% of accuracy. They also provide promising support to the collection of data for populations in developing countries, which are often under-represented in official surveys.

Most of researchers use CDRs to investigate socioeconomic development (A3). A seminal work analyzes landline calls and a nationwide mobile phone data set to show that, in the UK, regional communication diversity is positively associated with a socioeconomic ranking [75]. Other works address the issue of mapping poverty [76] and other socioeconomic determinants [77] with mobile phone communication data, combined with airtime credit purchases data in the Ivory Coast [78]. Blumenstock et al. [79, 80] show preliminary evidence of a relationship between individual wealth and the history of mobile phone transactions. Frias-Martinez et al. [81,82,83,84] analyze the relationship between human mobility and the socioeconomic status of urban zones, presenting which mobility indicators correlate best with socioeconomic levels and building a model to predict the socioeconomic level from mobile phone traces. Pappalardo et al. [85] analyze mobile phone data and extract meaningful mobility measures for cities, discovering an interesting correlation between human mobility aspects and socioeconomic determinants. Lotero et al. [86] analyze the architecture of urban mobility networks in two Latin-American cities from the multiplex perspective. They discover that the socioeconomic characteristics of the population have an extraordinary impact on the layer organization of these multiplex systems. In a successive work, Lotero et al. [86] analyze urban mobility in Colombia representing cities by mobility networks. They encode the origin-destination trips performed by a subset of the population corresponding to a particular socioeconomic status and they show that spatial and temporal patterns vary across these socioeconomic groups. Amini et al. [87] use mobile phone data to compare the human mobility patterns of a developing country (the Ivory Coast) and a developed country (Portugal). They show that cultural diversity in developing regions can present challenges to mobility models defined in less culturally diverse regions. Smith-Clarke et al. [88] analyze the aggregated mobile phone data of two developing countries and extract features that are strongly correlated with poverty indexes derived from official statistics census data.

Moreover, researchers use CDRs to monitor the quality of the environment and its impact on people’s lives (A4). For example, Picornell et al. [89] evaluate the population exposure to \(\mathrm{NO}_2\) on a research published recently. They use CDRs from one of the three most important Spanish mobile phone network operators (MNOs), with around 30% market share. The analysis is conducted for the capital of Spain, Madrid, for the 17th of November 2014, as a typical day in terms of population mobility and \(\mathrm{NO}_2\) levels. Comparing the results with traditional census-based methods, they demonstrate relevant discrepancies at disaggregated levels and underline the importance of integrating CDRs data for the evaluation of population exposure to \(\mathrm{NO}_2\). Lu et al. [90] study people’s behavior affected by climate stress. In particular, by exploring the individuals’ behavioral response to the Cyclone Mahasen, which struck Bangladesh in May 2013, they find out that anomalous patterns of mobility and calling frequency correlate with rainfall intensity, showing the affected regions and when the storm moves. Lu and Bengtsson [91, 92] analyze the movement of 1.9 million mobile phone users before and after the 2010 Haiti earthquake, and they show that CDRs can be a valid data source for estimates of population movements during disasters. Wilson et al. [93] build a tool within nine days of the Nepal earthquake of 2015, to provide spatiotemporally detailed estimates of population displacements from CDRs based on movements of 12 million mobile phones users. Nyarku et al. [94] use CDRs to explore whether mobile phones could be reliably used to monitor individual exposure to selected air pollutants when moving between indoor and outdoor microenvironments. In particular, data are collected from two BROAD life mobile phones, which are equipped with sensors for direct measurements of air pollutants. The two phones bring similar results, both for particles and formaldehyde, making them potentially suitable for applications in polluted environments, even if there seem to be some exceptions where the readings of the two phones do not correspond well to each other. Liu et al. [95] map personal trajectories using mobiles in an urban environment to assess the impact of traffic-related air pollution in society. They estimate traffic pollution exposure to individuals based on the exposure along the individual human trajectories in the estimated pollution concentration fields by utilizing modeling tools and manage to identify trajectory patterns of particularly exposed human groups. In addition, Decuyper et al. [96] use CDRs to study food security indicators finding a strong correlation between the consumption of vegetables rich in vitamins and airtime purchase.

Other studies focus on the safety dimension (A5). Bogomolov et al. [97] use CDRs for 3 weeks from the 9th to the 15th of December 2012 , and from the 23rd December 2012 to the 5th of January 2013, in combination with demographic data from December 2012 to January 2013, to predict crime in the city of London. Experimental results show 70% of accuracy in predicting whether an area could be a crime hotspot or not. Similarly, Ferrara et al. [98] study criminal networks to detect and characterize criminal organizations in networks reconstructed from the CDRs. They also introduce an expert system to support law enforcement agencies in unveiling the underlying structure of criminal networks.

GPS and transportation data

Since the 1990s, Global Positioning Systems (GPS) have been used for tracking the movements of the individuals [99,100,101,102]. In particular, GPS data provide time and location coordinates information, which can be used to link locations with environments and to calculate the speed of movements [103]. For insurance reasons, some vehicles have a black box installed. The device records the position of the vehicle at regular intervals and sends it to the database. Table 3 illustrates an example of the structure of GPS records.

Table 3 Example of GPS records

GPS data can also cover rural areas, as opposed to other data, mostly collected among citizens of urban areas [104]. Comparing to the traditional ways of measuring mobility, usually by self-reports assessed with questionnaires, GPS does not bring any biases and misclassification, [104, 105], as it eliminates the social desirability usually brought by self-report participants [106, 107]. Another advantage of GPS data is that they provide real-time monitoring. However, while there are studies based on GPS data covering hundreds of thousands of individuals [108] most of the GPS studies are conducted with fewer than 300 participants [104, 109], usually due to privacy issues. Apart from this drawback, when a GPS is used indoors, the spatial accuracy of the measurements is fairly detected [110], which creates problems in specific fields, such as on epidemiology research.

GPS data are used to explore several dimensions of objective well-being, notably health (B1), socioeconomic development (B3), and safety (B5).

Table 4 The table contains a subset of the information returned by a Twitter API

Health (B1) exploration has also attracted the interest of researchers. For example, Saelens et al. [111] track the movements of an individual through GPS devices and bring to the surface growing evidence that transit users are more physically active than non-transit users, which could potentially lead to the health improvement of the first ones. Similarly, Rundle et al. [112] explore health in terms of physical activity, and conclude that neighborhood walkability influences other residents’ choice of space utility and is also associated with higher weekly physical activity. Additionally, Sadler et al. [113] use GPS data to understand children’s exposure to junk food in Canada and compare the results to a validated food environment database. They demonstrate that official results underestimate exposure to junk food up to 68%, which should be taken into consideration by policy-makers. Finally, Canzian and Musolesi [114] analyze mobility patterns from GPS traces to answer whether mobile phones can be used to monitor individuals affected by depressive mood disorders. They develop a smartphone application that periodically collects the locations of the users and the answers to daily questionnaires that quantify their depressive mood. They find a significant correlation between mobility trace characteristics and the depressive moods of individuals.

Some of these works using GPS data focus on exploring socioeconomic development (B3). Marchetti et al. [115] perform a study at regional level, analyzing GPS tracks from cars in Tuscany to extract measures of human mobility at province and municipality level. They find that there is a strong correlation between the mobility measures and a poverty index independently surveyed by the Italian official statistics institute. Smith et al. [116] use an automated fare collection data set of journeys made on the London rail system to build a classifier that identifies areas of the city with high economic deprivation. They highlight that, given its high precision, the classifier provides potential benefits for city planning and policy-making. Lathia et al. [117] use the same data set to find that more deprived areas tend to receive passenger flow from a higher number of other areas compared to less deprived areas, also uncovering some evidence of social segregation.

Another objective well-being dimension that is explored with GPS data is safety (B5). Robinson et al. [118] compare the spatial distribution of objective crime incidents and self-reported physical activity among adolescents in Massachusetts, between 2011 and 2012, and show that there is a positive association between them (\(r=0.72\), \(p<0.0001\)). Ariel et al. [119] use GPS data to replicate findings published from US official research on the effect of hot spots policing for the prevention of crime in England and Wales and demonstrate that victim-generated crimes (the primary outcome measured in previous studies) increase in both the near vicinity and in catchment areas.

Social media data

Social media, such as Twitter, Facebook, and Instagram, can be considered as a digital database of information about online users, hence rendering individuals’ online activities accessible for analysis. Given this enormous potential, researchers, governments, and corporations are turning their interest on social media to understand human behavior and interactions better [120]. Among all social media, Twitter is the most popular, since it provides public access to data through APIs with the least restrictive policy. The Twitter APIs return information about locations, date of the event, interactions with other users, or tags inserted in the tweet. Twitter also returns some information about the user profile. Table 4 illustrates an example of the structure of Twitter records.

Despite their indubitable usefulness, social media data may also encounter some concerns [121]. First of all, they may reflect social desirability biases, since individuals manage their online profiles [122]. Besides, social media users may not be as representative of the general population as traditional anonymized self-reports conducted through a chosen representative sample [123].

All dimensions of objective well-being are monitored through social media data, i.e., health (C1), job opportunities (C2), socioeconomic development (C3), environment (C4), safety (C5) and politics (C6).

Several studies provide valuable insights into how the analysis of social media data can lead to next-generation automated methodologies for public health (C1). As an example, Eichstaedt et al. [123] use Twitter data, in combination with atherosclerotic heart disease (AHD) mortality rates and country-level socioeconomic variables. They predict country-level heart disease mortality since the language expressed on Twitter reveals important psychological characteristics that are significantly associated with heart disease mortality risk. Besides, De Choudhury et al. [124] use Twitter data in combination with traditional depression screening test data for the detection and diagnose of the individuals’ major depressive disorders and even to predict the likelihood of depression of individuals. Signorini et al. [125] use data from Twitter to track rapidly-evolving public sentiment concerning H1N1 and to measure actual disease activity. They show that Twitter can be used as a measure of public interest or concern about health-related events and that estimates of influenza-like illness derived from Twitter chatter accurately track reported disease levels. Paul et al. [126] incorporate in their forecasting models the historical influenza data and Twitter data. Lampos et al. [127] measure the prevalence of flu-like symptoms in the general UK population, based on the contents of Twitter, searching for symptom-related statements, turning this information into a flu-score and they obtain on average a statistically significant linear correlation which is higher than 95%. In a later work, the authors [128] instead of choosing the keywords and phrases themselves, they use machine learning algorithms to find out which words in the database of tweets occurred more often at times of elevated levels of flu, and they obtained very positive results. They claim that flu epidemics can be detected based on Twitter content. Chen and Yang [129] use individuals’ tweets with spatiotemporally tagged information to demonstrate that people’s healthy diet is elicited by exposure to their immediate food environment.

Regarding the monitoring of job opportunities (C2), Llorente et al. [130] quantify the extent to which deviations in diurnal rhythm, mobility patterns, and communication styles across regions relate to unemployment. For this purpose, they examine country-wide Twitter data describing 19 million geo-located messages and find that the regions exhibiting more diverse mobility fluxes, earlier diurnal rhythms, and more correct grammatical styles display lower unemployment rates. Antenucci et al. [131] use data from Twitter, from July 2011 to early November 2013, to create indexes of job loss, job search, and job posting. They derive signals by counting job-related phrases in tweets such as “lost my job”. They construct social media indexes from the principal components of these signals and manage to track events that affect the job market in real-time, such as Hurricane Sandy and the federal government shutdown.

A large number of works in the literature focus on monitoring socioeconomic development from social media data (C3). Bollen et al. [132], in a further study, analyze data from Twitter and consider the emotions of traders, rather than their information gathering processes, suggesting that changes in the calmness of Twitter messages could be linked to changes in stock market prices. Still, regarding socioeconomic development, social media data are also extensively used to nowcast and forecast stock market prices and traded volumes. Seminal works in this field leverage information contained within investment discussion boards and blogs. For example, Bar-Haim et al. [133] use StockTwits data to uncover relevant correlations between Web-derived indicators and the stock market. In detail, they leverage sentiment scores of messages shared in the Yahoo message boards to find correlations with the stock market. In a different web platform study, De Choudhury et al. [134] try to find correlations between the stock market and blog communications. Last, Cresci et al. [135, 136] assess the risks and vulnerability of stock markets to automation, manipulation, and disinformation, with the ultimate goal of safeguarding people’s investments.

Researchers also use social media for the exploration of the environment dimension (C4). Avvenuti et al. [137] claim that the analysis of social media proves valuable for quickly acquiring situational awareness and estimates of the impact of disasters. As an example of the predictive power of social media, Kryvasheyeu et al. [138], Avvenuti et al. [139] and Mendoza et al. [140] demonstrate the viability of predicting or nowcasting the damage produced by earthquakes by analyzing social media communications in the aftermath of the event. The results of these models can also be displayed in real-time, interactive maps that highlight stricken areas and provide support to emergency responders. Notable examples of this kind are the systems developed by Avvenuti et al. [141, 142]. Preis et al. [143] find that the number of photos taken and subsequently uploaded to Flickr with titles, descriptions, or tags related to Hurricane Sandy bears a striking correlation to the atmospheric pressure in the US state New Jersey. They claim that appropriate leverage of such information could be useful to policy-makers and emergency crisis managers.

Safety is another dimension that can be monitored using data from social media (C5). For example, Chen et al. [144] use Twitter data and create a model that predicts the specific time and location a crime occurs. This model combines kernel density estimation based on historical crime incidents and prediction via linear modeling with sentiment and weather predictors. By adding the latter determinants, they show that their model improves significantly with respect to existing models. Similarly, Boni et al. [145] use spatio-temporally tagged tweets and create a model for crime prediction. In particular, they combine real crime data with individuals’ micro-level movement patterns extracted from Twitter and demonstrate improved predictions. Likewise, Kadar et al. [146] describe urban crime by using Foursquare and considering these data as a measurement for the ambient population of a neighborhood, to further describe crime levels. They also confirm that such models improve the traditional models, based on census data. Additionally, the city of Chicago applies text analytics on Twitter and 311 (the local emergency number) records to detect and prevent phenomena like rat infestations and to track civil unrest and violent crimes (CrimeScan and CityScan software) [147,148,149].

Finally, the politics dimension (C6) is extensively studied, in particular, during the last years with the rise of the political crisis across the world. Colleoni et al. [150] investigate the political homophily on Twitter to classify users as Democrats or as Republicans based on their tweets. They show that, in general, the former exhibit higher levels of political homophily than the latter. Goh et al. [151] use Facebook pages of a group of 12 politicians and demonstrate that political engagement can be achieved by creating social media consumption habits, as supported by the habit formation in consumption from macroeconomics. Similarly to the field of socioeconomic and financial analyses, social media data can be easily manipulated also for achieving political goals [152, 153]. As such, results of political analyses based on social media should be carefully weighed to minimize issues related to biases and manipulations.

Health and fitness data

These data mainly consist of Electronic Health Records (EHRs) and mobile application data that are mainly used for monitoring the health dimension (D1). EHRs, initially created for the facilitation of the billing and patient care, are widely used for clinical studies and clinical risk prediction. Table 5 reports an example of clinical records concerning the hospitalization of some patients.

Table 5 The table shows an example of clinical records, including the pathology for which a patient is admitted to the hospital, the duration of hospitalization and the medicines she/he took

Out of a systematic review, Goldstein et al. [154] demonstrate both opportunities and challenges of EHRs. On the one hand, compared to the traditionally used cohort data developed and collected for research purposes (such as the Framingham Heart Study [155]), EHRs are cost-effective. In contrast with cohort data, EHRs can indeed be used for multiple health studies and, since they are collected at a high frequency, they allow a better prediction of near-term risk of events. On the other hand, EHRs include only individuals that have been ill or at least have had a clinic visit, which could generate a problem of representativeness. Moreover, they are not publicly available and might include limited time slots.

Table 6 Subset of the main fields provided by GDELT platform

Researchers use EHRs to monitor several aspects of personal health (D1). For example, Sultana et al. [156] use the Integrated Primary Care Information (IPCI) database to look for elements that could contribute to traditional methodologies. For example, multimorbidity and polypharmacy are elements that could help in identifying frailty methodologies. They demonstrate that the Mini-Mental State Examination score, which is the most commonly recorded data item, could be potentially used as a frailty identifier. Ghaderighahfarokhi et al. [157] use medical records of newborns in the educational Hospitals affiliated to the Ilam University of Medical Sciences (from April 2015 to April 2016) to identify accurate predictors of Low Birth Weight (LBW). They demonstrate that LBW is a multi-factorial condition requiring a systematic and accurate program to be reduced, such as education through mass media, repeated monitoring of pregnancy, and others. Metzger et al. [158] use EHRs with Emergency Department patient visits in 2012, from Lyon University Hospital, to demonstrate that machine learning can contribute to more accurate estimations of suicide attempts in France, in relation to the current national surveillance system based on manual coding by emergency practitioners. Mhaskar et al. [159] investigate the 30 minutes prediction of blood glucose levels based on Continuous Glucose Monitoring (CGM). In particular, they use data from the DirecNet Central Laboratory, containing time series for 25 patients, who are less than 18 years old. By training a deep learning model on a data set designed to explore the performance of CGM devices in children with Type I diabetes, they demonstrate how deep neural networks can outperform shallow networks on this task. In addition, Santillana et al. [160] use a clinician’s database, named as UpToDate, to predict influenza epidemics in the United States promptly. They show that digital disease surveillance tools based on experts’ databases may be able to provide an alternative, reliable, and stable signal for accurate predictions of influenza outbreaks. Besides EHRs, mobile app data, such as lifestyle habits data concerning eating and physical activity behaviors, are used for the monitoring of objective well-being in terms of health (D1). These data demonstrate for once more that smartphones can contribute to research with valuable new insights, although they might apply biases towards people with lower socioeconomic status or towards people who are more interested in their health. In addition, such data collected through web surveys for research purposes might bring the disadvantages discussed before. A critical study using mobile app data is conducted by Althoff et al.[161]. They use a data set consisted of physical activity for 717,527 Apple iPhone smartphone users of the Azumio Argus app, which tracks users’ diet and fitness and other healthy behaviors, between July 2013 and December 2014. They demonstrate inequality in how the activity is distributed within countries and that this inequality is a better predictor of obesity than average activity level. Similarly, Hayeri [162] uses continuous glucose monitors (CGM) and fitness wearables (Fitbit) to predict blood glucose values. The study uses data gathered from each participant for 60 days, where the data from the first 30 days are used to train the algorithm and the remaining 30 days to test the predictions. On average, the software is able to predict a user’s future glucose values with a 93% accuracy rate for 60-mins ahead of time.

News

News data sources, such as the GDELT database [163], contain information extracted from the news of newspapers around the world. News records generally describe a variety of subject domains (e.g., economic events, political events), represent a wide range of targets (e.g., opposing politicians) [164] and are continuously updated, containing even archived historical news of the last decades. Nevertheless, such data contain three main biases [165]: the gatekeeping bias, i.e., the editors or the journalists decide on which event to publish; the coverage bias, related to the coverage of an event (e.g., western countries are over-covered, whereas African countries are under-covered); the statement bias, when the content written by the journalist, even if tried to be objective, is favorable or unfavorable towards certain events. Table 6 shows an example of news records.

News records are used to measure health (E1), socioeconomic development (E3), environment (E4), and politics (E6) dimensions of objective well-being.

Emerging infectious diseases and the rise of modern technology have generated new demands and possibilities for disease surveillance and response (E1). Growing numbers of outbreak reports must be assessed rapidly so that control efforts can be initiated. For example, the World Health Organization (WHO) sets up a process for timely disease outbreak verification to convert large amounts of data from some 600 sources, including all major news wires, newspapers, and biomedical journals, into accurate information for suitable action [166, 167]. Brownstein et al. [168] in a similar effort, create HealthMap, a freely accessible, automated real-time system that monitors, organizes, integrates, filters, visualizes, and disseminates online information about emerging diseases. Wilson et al. [169] use the HealthMap project to monitor listeriosis. Chunara et al. [170] use social and news media to validly estimate the 2010 Haitian cholera outbreak.

News records on financial affairs and financial markets are intrinsically interlinked (E3). Alanyali et al. [171] quantify the relation between movements in financial news and movements in financial markets by exploiting a corpus of six years of financial news from 2007 to 2012 from the Financial Times. Their results suggest that greater interest in a company in the news is related to greater interest in the corresponding company in stock markets. Lillo et al. [172] show that the flux of news of the previous day affects the trading activity of companies, households, and foreign investors and the dynamics of volatility.

News can also help capturing the environmental dimension of well-being (E4). As an example, Kleinschmit et al. [173] investigate 394 articles on forest and climate change published in the Swedish newspaper Dagens Nyheter from 1992 to 2009. They show that there has been an increasing discussion on forests in a changing climate over the last 18 years from both scientists and politicians. The increased number of these news events correlate with real environmental events happening internationally. Similarly, Boykoff [174] uses data extracted from the Vanderbilt University Television News Archive, consisting of television news from US news broadcasts (e.g., ABC World News Tonight) for the period between 1995 and 2004. He demonstrates that 70% of the US television news provide balanced coverage on anthropogenic contributions to climate change compared to natural radiative forcing. He also shows that there is a significant difference between this television coverage and scientific consensus on the topic.

News records are also used to understand the coverage of political issues (E6). Van Aelst and De Swert [175] use daily news of politics of campaign periods, extracted from the Electronical News Archive over the 2003 to 2006 period, and show that campaign periods have a high impact on the amount, style and actors of the political news in Belgium. To the best of our knowledge, the dimension politics (E6) has not yet been adequately explored through news data and constitutes inspiration for future research.

Scanner data

Scanner data are generated by point-of-sales terminals in shops and provide information at the level of the single product. Sales terminals record each transaction, and the resultant data can provide considerable insights into consumer purchasing patterns. They can be obtained from a wide variety of retailers: supermarkets, pharmacies, do-it-yourself stores, home electronics or clothing shops, and many others [176].

Scanner data are used from social researchers, as they can offer useful detailed information and the possibility to model the dynamic behavior of households, as well as to control for unobservable time-invariant characteristics [177]. Also, scanner data provide information over long periods of time than only one day or a couple of weeks. This happens because the final data used are produced from customers that purchase several items on each store visit, for several store visits, over a period of time [178, 179]. It is also worth mentioning that scanner data can contribute to the improvement of the quality of the Harmonized Index of Consumer Prices (HICP) [180]. However, using scanner data is challenging since researchers are dependent on the retailer’s permission, and they should also overcome the legal constraints in order to obtain them [179]. Table 7 shows an example of supermarket records.

Table 7 Subset of the main fields provided by a supermarket database for the purchases in different shops

Scanner data are used to measure health (F1), socioeconomic development (F3), and environment (F4) dimensions of objective well-being.

To begin with, researchers use scanner data to monitor several aspects of public health (F1). For example, pharmaceutical sales may be used to predict changes in clinical conditions with a useful time lead. Magruder et al. [181] find a 90% correlation between flu-related drug sales and physician diagnoses of acute respiratory conditions, at several subregions of the National Capital Area. They show that these sales occur approximately three days before the physician-patient encounter. Scanner data are also used to study the nutrients and saturated fat of several food categories and their implications on personal health. For example, Griffith et al. [177] use supermarket scanner data from the UK to study the nutrients in foods. They show that there is a lot of variation in nutrients at individual product level, even with food categories such as butter, which are very narrow. Bonnet et al. [182] use data from French supermarkets to explore consumer behavior with respect to the consumption of saturated fat, while Griffith et al.[183] model the potential impact of a tax on saturated fats. Finally, Janssen et al. [184] use scanner data from the Nielsen Consumer Panel data set that covers the years from 2004 to 2017. They aim to identify households with a pregnant household member and also to estimate the effect during and after pregnancy on alcohol purchases and relative expenditure on fruit and vegetables. Results show that during and after pregnancy, households reduce their alcohol purchases by 22–27%. In contrast, the relative expenditure on fruit and vegetables does not increase during pregnancy but decreases post-pregnancy by 19%.

The majority of studies with scanner data focus on exploring the socioeconomic development (F3). Van der et al. [186] introduce a new method for computing the Dutch Consumer Price Index (CPI) based on supermarket scanner data. In the meanwhile, in 2017, Eurostat issued a practical guide for processing supermarket scanner data to calculate the CPIs of EU countries in order to ensure the comparability of the values across Europe, as well as to modernize the official statistics [179]. Silver et al. [187] outline the potential use of scanner data from retailers for the measurement of inflation. They use monthly scanner data for television sets in 1998 in the UK to study the two primary forms of bias in CPIs. Moreover, Pennacchioli et al. [188] study the retail activity of the customer subset of an Italian supermarket chain. They discover that highly ranked customers, with more sophisticated needs, tend to buy niche products, i.e., low-ranked products. On the other hand, low-ranked, low purchase volume customers tend to buy only high-ranked products, very popular products that everyone buys. In addition, Sobolevsky et al. [189] use a complete set of bank card transactions in 2011 in Spain and demonstrate that there is a clear correlation between individual spending behavior and official socioeconomic indexes denoting the quality of life.

Finally, researchers use scanner data to monitor the impact of humans on the environment (F4). Panzone et al. [190] use scanner data from the largest UK food retailer for the creation of an Environmentally Sensitive Shopper (ESS) index measuring the environmental sustainability of food consumption at household level. In addition, Gadema et al. [191] use data from UK supermarket shoppers to examine whether carbon footprinting and labeling food products are tools that could facilitate consumers to make greener purchasing decisions. They claim that this could be a sensible way to potentially achieve a low carbon future. Food waste is a significant problem in modern society and carries considerable social, economic, and environmental costs. For example, Brancoli et al. [192] use scanner data to analyze the impacts of food waste at a supermarket in Sweden. They discover the importance of not only measuring food waste in terms of mass but also in terms of environmental impacts and economic costs. They also show that meat and bread waste contribute the most to the environmental footprint of the supermarket. Last, Scholz et al. [193] analyze food waste data of six Swedish supermarkets from 2010 to 2012 in terms of mass and carbon footprint. They calculate the wastage carbon footprint for fresh products such as meat, deli, cheese, dairy, and fruits and vegetables.

Web search queries

Web search queries data report the frequency of specific terms over time, entered into a web search engine from users to satisfy their information needs. Data are represented as time series of the frequency, and therefore we do not provide an example of search queries records in this paper.

Comparing to other data sources that require customized and often complicated collection strategies, search data can be collected for many domains simultaneously. They can also be easily analyzed across several countries or regions in real-time. Search data are often helpful in making forecasts. However, their utility for predicting real-world events is based on convenience, speed, and flexibility and has less to do with their superiority over other data sources. Goel et al. [194] provide a useful survey in this area and describe some of the limitations of this data source. First, for different domains, the size of the relevant population varies considerably, along with difficulty in identifying relevant queries. Additionally, in specific domains, searching may be more closely tied to the measured outcomes than in others.

Web search queries data are used to measure health (G1), job opportunities (G2), socioeconomic development (G3), safety (G5), and politics (G6) dimensions of objective well-being.

Public health is a dimension of well-being that is explored through web search queries (G1). In order to improve early detection, researchers monitor health-seeking behavior in the form of web search queries, which are submitted by millions of users around the world every day. For example, Cooper et al. [195] study Yahoo! search activity related to cancer in the USA. They find out that the Yahoo! search activity associated with cancer correlates with the estimated cancer incidence and estimated cancer mortality. Polgreen et al. [196] show that search volume for handpicked influenza-related queries is correlated with the reported number of cases over the period from 2004 to 2008. Hulth et al. [197] find similar results in a study of search queries submitted on a Swedish medical Web site. Yuan et al.[198] monitor influenza epidemics in China with search queries from Baidu. Additionally, an automated procedure for identifying informative queries is described by Ginsberg et al. [199]. Based on that, Google Flu Trends [200] was introduced by Google in 2008 to provide real-time estimates of flu incidence for more than 25 countries and to help predict outbreaks of flu. Nsoesie et al. [201] present a framework for near real-time forecast of influenza epidemics using web-based estimates of influenza activity from Google Flu Trends for 2004–2005, 2007–2008, and 2012–2013 flu seasons. Yang et al. [202] use Google Flu Trends and historical data to infer the evolving epidemiological features of influenza and its impacts among the large population during 2003–2013, including the 2009 pandemic. Wilson et al. [203] use data from Google Flu Trends to study the spread of the pandemic H1N1 influenza in New Zealand during 2009. Furthermore, Chan and Althouse [204, 205] use Google queries to monitor Dengue epidemics, Dukic et al. [206] to predict hospitalizations for methicillin-resistant Staphylococcus aureus infections and Ocampo et al. [207] for malaria surveillance. Moreover, Yang et al. [208] evaluate the association between suicide and Google searches trends for 37 suicide-related terms representing major known risks of suicide in Taipei City, Taiwan, from 2004 to 2009. Their results show that a set of suicide-related search terms, the trends of which either temporally coincided or preceded trends of suicide data, are associated with suicide death. Searches for “major depression” and “divorce”, for example, account for at most, 30.2% of the variance in suicide data. McCarthy [209] uses annually-averaged Google search activity for “suicide” from the same period, from 2004 to 2009 to study suicide rate data in the United States. The study shows that searches for most medical, familial, and socioeconomic terms precede suicide deaths, and most searches for psychiatric-related terms coincide with suicide data. In a later work, Kristoufek et al. [210], use Google data from 2004 to 2013 in combination with suicide occurrences data to estimate the number of suicide occurrences in England. Finally, Adler et al. [211] combine official statistics on demographic information with data generated through search queries from Bing, between November 2016 and February 2017, to gain insight into suicide rates per state in India. In this way, their search data work as a proxy for unmeasured (hidden) factors corresponding to suicide rates.

Table 8 Example of the information provided by users of influenzanet

The first to explore the job opportunities dimension (G2), are Ettredge et al. [212] as they find that counts of the top 300 search terms during from 2001 to 2003 are correlated with US Bureau of Labor Statistics unemployment figures. Later on, Askitas et al. [213], D’Amuri et al. [214], Suhoy et al. [215] confirm the value of search data in forecasting unemployment in the US, Germany, and Israel. Baker et al. [216] use Google search data to examine how job search responds to extensions of unemployment payments. Finally, McLaren et al. [217] summarise how online search data can be used for economic nowcasting by central banks. They show that the volume of online searches can be used as indicators of economic activity, more specifically for unemployment and housing markets in the United Kingdom.

Researchers use search queries to monitor socioeconomic development (G3) as well. Choi and Varian [218, 219] consider Google Trends as a source of data on real-time economic activity, and they show that by using its query indices accurate predictions can, for example, be made for retail, automotive, etc., and could be helpful for short-term economic prediction or nowcasting. Koop and Onorante [220] use Dynamic Model Selection (DMS) methods, which allow for model switching between time-varying parameter regression models. They extend the DMS methodology by allowing Google variables to determine the nowcasting model to be used at each point in time. Guzman [221] examines Google data as a predictor of inflation. Additionally, Preis et al. [222] provide evidence that search engine query data and US stock market fluctuations are correlated. In a later [223] work, they analyze changes in Google query volumes for search terms related to finance, and they find patterns that may be interpreted as “early warning signs” of stock market moves. Furthermore, Curme et al. [224] present a method that allows identifying topics for which levels of online interest change before large movements of the Standard & Poor’s 500 index (S&P 500). They find that search volumes from Google related to politics and business can be linked to subsequent stock market moves. This demonstration of a connection between stock market transaction volume and search volume is also replicated using Yahoo! data, where Bordino et al. [225] show that query volumes precede in many cases peaks of trading by one day or more. Finally, Moat et al. [226] show that data on views of Wikipedia pages can also be related to market movements, providing evidence that increases in the number of views of financially related pages on Wikipedia can be detected before stock market falls.

Search data are also used for the exploration of safety (G5). Qi et al. [227] show that a simple low-level indicator of civil unrest can be obtained from online data at an aggregate level through Google Trends or similar tools. The study covers countries across Latin America from 2011 to 2014 in which diverse civil unrest events took place. In each case, they find that the combination of the volume and momentum of searches from Google Trends surrounding pairs of simple keywords, tailored for the specific cultural setting, provide useful indicators of periods of civil unrest. Qi et al. [228] study online search activity from Google Trends surrounding the topics of social unrest over several countries in Latin America from 2011 to 2014. They find that the volume and momentum of searches surrounding mass protest language, can detect—and may even pre-empt—the macroscopic on-street activity. They also find that the most crucial search keywords differ subtlety from country to country, even though the language may be the same. They explain this by the fact that civil unrest is a time-varying coordinated interaction between individuals, groups, or populations within a given cultural and socioeconomic setting.

Finally, the politics dimension is explored with search data (G6). Chykina et al. [229] study how Google Trends can be used to examine issue salience for hard-to-survey mass populations in the US, from 2010 to 2017. They apply this method to immigrant concerns over deportation. They show that anxieties over removal increase in response to (potential) policy changes, such as immigration policies that are considered in the wake of Donald Trump’s election. Reilly et al. [230] use Google search activity for ballot measures’ names and topics in a state one week before the 2008 Presidential election, and they find that they correlate with actual participation on those ballot measures. Their result demonstrates that the more Internet searches there are for a ballot measure, the less likely voters are to roll-off (not answering the question) and establish the validity for this data for a critical topic in state politics research.

Crowdsourced data

Kleemann and Rieder [231], in 2008, have defined crowdsourcing as the “the intentional mobilization for commercial exploitation of creative ideas and other forms of work performed by consumers”. In other words, crowdsourcing involves obtaining work, information, or opinions from a large group of people who submit their data via the Internet, smartphone apps, etc. Naturally, crowdsourcing brings several advantages. Crowdsourcing can provide researchers with a huge amount of data, which can be accessed quickly and at a relatively low cost. Besides, comparing to traditional research (such as studies using traditional surveys), the use of crowdsourcing can provide researchers with data from samples that are more diverse [232]. However, crowdsourcing yields various challenges, as well. Firstly, crowdsourcing may bring relatively low-quality results, e.g., a participant of a crowdsourced study may intentionally give wrong answers. Secondly, mobile platforms pose new challenges for crowdsourced data management. Table 8 shows an example of crowdsourced data.

Crowdsourced data are used to capture all dimensions of objective well-being, i.e., health (H1), job opportunities (H2), socioeconomic development (H3), environment (H4), safety (H5) and politics (H6) dimensions of well-being.

To improve early detection, researchers started monitoring the health of individuals (H1) through crowdsourced self-reporting mobile apps, such as Influenzanet (Europe) [233], Flutracking (Australia) [234], and Flu Near You (United States) [235]. Hashemian et al. [236] introduce iEpi, an end-to-end system for epidemiologists and public health workers to collect, visualize, and analyze contextual microdata through smartphones. Additionally, Madan et al. [237] use data from a smartphone application provided to university students to study their health state. Participants fill out self-report surveys related to their health habits, diet, exercise, weight changes, daily symptoms related to common colds, fever, influenza, and mental health. The researchers find that phone-based features can be used to predict changes in health, such as common colds, influenza, and stress. For longer-term health outcomes such as obesity, they find that weight changes of participants are correlated with exposure to peers who gain weight in the same period. Finally, Martinucci et al. [238] study Gastroesophageal Reflux Disease (GERD) symptoms among Italian university students from a data set collected from a web-app. The app allows users a self–diagnosis for the gastrointestinal disturbances through a simple questionnaire and data about the students’ food consumption at the university canteen. They show that 792 students reported typical GERD symptoms to occur at least weekly. Among all users, females, smokers, and high in BMI students tend to show increased GERD values.

Researchers use crowdsourced data to explore the job opportunities dimension (H2) and the direct socioeconomic benefits associated with it. For example, Green et al. [239] use the crowdsourced employer review website named Glassdoor, an online crowdsourced employer branding platform, to explore employees’ satisfaction and work–life balance. This exploration is preliminary for the direct economic benefit and most important finding of the study; companies experiencing improvements in employer ratings are significantly associated with future stock returns, comparing to companies with declines in employer rating. Similarly, Dabirian et al. [240] analyze reviews of the highest and lowest-ranked employers on Glassdoor. Using IBM Watson to analyze the data, they show how employers could use crowdsourced employer branding intelligence to turn into a workplace that attracts highly qualified employees. Furthermore, Könsgen et al. [241] analyze employee reviews data, listed on the German employee review site named Kununu.de, combined with \(2\times 2\times 2\) between-subjects experimental design. Results show that such studies can complement the research on the online reputation by underlying the relevance of discrepant reviews for job candidates’ application intentions.

Crowdsourced data are also used to estimate the socioeconomic (H3) well-being. For example, Tingzon et al. [242] show the feasibility to map poverty by combing crowdsourced geospatial information with nighttime lights, daytime satellite imagery, and human settlement data. In particular, they use the popular geospatial data crowd-sourcing platform named OpenStreetMap [243] to map poverty in the Philippines. Similarly, Piaggesi et al. [244] use OpenStreetMap [243] crowdsourced data merged with official data at a city scale. They demonstrate the possibility of estimating the socioeconomic conditions of different neighborhoods of five different cities in North and South America. In order to increase the efficiency of direct money transfers to impoverished villages in Kenya and Uganda, Abelson et al. [245] develop and deploy a crowdsourcing interface to obtain labeled satellite imagery training data. They train and deploy a predictive model for detecting impoverished villages. Their estimations are leveraged to build a fine-scale heat map of poverty that is used to recommend donations to the most impoverished villages.

Crowdsourcing is also used to capture the environmental dimension of well-being (H4). There are plenty of examples of crowdsourcing platforms for emergency management, such as Ushahidi [246], where volunteers provide updated environmental information in the aftermath of mass emergencies. These platforms are shown to contribute significantly to organizing a prompt emergency response [247]. Another category of crowdsourced platforms is the so-called citizens’ observatories [248], a community-based network of environmental monitoring and information systems. On these platforms volunteers monitor and provide data about a plethora of environmental dimensions, such as comprising water availability and water quality, air pollution, land use, and flood risk management [249]. As an example, Schneider et al. [250] combine crowdsourced data from the EU-funded CITI-SENSE project, which measures the air-quality with data obtained from statistical or deterministic air quality models. Their goal is to present a novel data fusion-based technique for combining real-time crowdsourced observations with model output that maps the urban air quality in detail. This could help users find the least polluted routes or control their exposure to pollution while moving around the city. Besides, Meier et al. [251] use crowdsourced atmospheric data from Netatmo weather stations in the city of Berlin, as well as available metadata to explore the urban atmosphere. Results show a distinctive urban heat island pattern in Berlin during the night and are also validated, confirming that crowdsourced atmospheric data can contribute to advancement in climate research. Similarly, Chapman et al. [252] use Netatmo weather station crowdsourced data to quantify the urban heat island in the city of London over the summer of 2015. Their results are similar to previous studies with official data and are therefore validated.

Crowdsourced data are considered an important data source for studying safety (H5). Suzanne Goodney et al. [253] map violence against women with the use of a crowdsourced app named as Safecity.in, which includes anonymous reporting of violence against women. The goal of the study is to highlight the importance of crowd mapping violence, as it can make women aware of potentially dangerous locales, encourage violence reporting, and provide advice on practical solutions for navigating street harassment and assault in public buses. Furthermore, Gosselt et al. [254] use the Internet Movie Database (www.imdb.com) to study the violent behavior and victimization of male and female film characters over time in the United States. In particular, using IMDb synopsis texts, they analyze reviewers’ movie descriptions. They demonstrate that both perpetrators and victims are mainly male, as well as that violence becomes less severe and more often non-deadly over the years. Researchers underline the future potentiality of using such data sources to explore matching results with actual crime figures. Additionally, Ozkan et al. [255] use crowdsourced police-involved killings data from FatalEncounters.org, as well as media data, to control whether police killings is counted and reported correctly in the aforementioned unofficial data, as compared to official data in the city of Dallas. Results mostly show consistency between all data sources. In conjunction with social media and crowdsourcing data sources, as well as environmental and safety dimensions, Avvenuti et al. [256] collect targeted and detailed information from people involved in natural disasters through crowdsourcing surveys via social media. These data are used to monitor unfolding disasters better and to monitor their consequences (i.e., damage caused)

Last, crowdsourced data are also used to study the politics dimension (H6) of objective well-being. For instance, crowdsourced data have been used within NGOs to set strategic priorities and involvement in the referendum activities based on participants’ responses to a survey [257]. Yasseri and Bright [258] use Wikipedia traffic data for electoral prediction. In particular, they get insights about changes in overall turnout at elections and changes in vote share for certain parties. Furthermore, Gellers [259] explores whether crowdsourcing can overcome the democratic deficit in global environmental governance. He uses data from the United Nations MY World survey, a multi-year (2012–2015) global poll designed to identify post-2015 development priorities, as well as e-discussions data, organized by the UNDG and the thematic consultation on environmental sustainability ran from November 2012 to July 2013. Results suggest that although crowdsourcing may present an attractive technological approach to enhance participation in global governance, ultimately, the representativeness of this participation and the legitimacy of the policy results depend on the way the contributions are sought and filtered by international organizations.

Measuring subjective well-being

“Subjective well-being”, the scientific term of happiness, is a central value in people’s lives, and reflections for its definition have arisen ever since antiquity. Aristotle has expressed his interest on the topic claiming that human well-being, labeled as eudaimonia (\(\varepsilon v\delta \alpha \iota \mu o \nu \acute{\iota }\alpha \): Eu=Good, Daimon=spirit), is an activity of the soul expressing complete virtue [260]. During the last decades, researchers have focused on identifying the critical dimensions and the relevant determinants that can positively or negatively affect human well-being, hence providing a perspective different from the philosophical definition that Aristotle has been contemplating about. Since humans are conscious beings, they can subjectively evaluate their appreciation of life, labeled “subjective well-being” or happiness. In particular, happiness can be defined as satisfaction with life in general, or as sociologist Veenhoven (1984) suggests, as the degree to which an individual judges the overall quality of her life-as-a-whole favorably. Similarly, psychologist Diener [261] defines happiness as people’s affective and cognitive evaluations of life. Veenhoven [262] shows that people use two sources of information to evaluate their appreciation with life-as-a-whole: affects and thoughts. The first source of information captures people’s feelings, emotions, and moods, the so-called hedonic level of affect (or simply called emotional component). In particular, he underlines that to avoid neglecting crucial information about precedent and subsequent events, researchers should separate between positive and negative affects. On the other hand, the second source of information is the contentment component (or simply called structural component), concerning people’s thoughts and capturing whether people’s life expectations have been fulfilled, according to their cultural or societal standards, and lead them to evaluate their life satisfaction. These two components, the hedonic level of affect and the contentment component, determine the overall happiness.

Table 9 Pros and cons for each traditional data source and new data source used for the measurement of subjective well-being

This concept of happiness, compared to the traditional macroeconomic measurements, such as GDP, inflation and national income (see, e.g., Alesina et al. [263]) can capture the variations of people’s perceived well-being [11, 12]. It is also worth mentioning the controversy surrounding the relationship between national income and national happiness, identified by Easterlin [30]. According to the Easterlin paradox, temporary changes in income both within and between nations directly affect happiness, but over time happiness does not trend upward as income continues to grow.

Considering its subjective nature, researchers frequently measure happiness by self-report rating scales. Nevertheless, the most widely used are global reports, using the single-item scale, such as the Positive And Negative Affect Scale (PANAS) [264, 265]. Self-report measures are reliable since they provide accuracy and temporal stability, they are valid for community surveys and cross-cultural comparisons, and they can capture happiness as life-as-a-whole, as well as domain satisfactions [266,267,268,269]. Examples of self-reported surveys are the Gallup World Poll (e.g., study by Deaton [270]) and the World Values survey (e.g., study by Easterlin et al. [271]), which capture the worldwide happiness; the Gallup-Healthways Well-being index (e.g., study by Kahneman and Deaton [272]), the British Household Panel Survey (e.g., study by Frijters et al. [273]) and the Eurobarometer (e.g., study by Stevenson et al. [274]), which capture the happiness at local level. Although self-report surveys are widely used for the measurement of happiness, some factors might influence the results. For example, the type of questions asked before the happiness questions, as well as the individuals’ mood at the time of the well-being rating, might disturb the results. Deaton and Stone [275] demonstrate a high item-order effect because of political questions coming before happiness questions. Also, substantial current-mood effects on happiness judgments are generated because of weather conditions, since they affect people’s thoughts, feelings, and behavior [276, 277]. Finally, because global reports are abstract general ratings of happiness over a long period, they neglect temporal resolution.

Diversely, researchers use Ecological Momentary Assessment (EMA) and Day Reconstruction Method (DRM) that are momentary diary self-report measures of happiness. They are designed to capture the affective components of happiness and reduce recall biases and heuristics [269]. In particular, EMA is a longitudinal research methodology that asks participants to report their feelings, thoughts, and emotions at the moment or right after each of their activities, avoiding retrospective biases and maximizing the accuracy of the assessments [278]. Similarly, DRM asks participants to reconstruct their daily life activities systemically and their experiences of the preceding days. It does not capture the moment-to-moment variation of emotions, as EMA does, but it avoids disturbing normal activities, requires less respondent burden [279] and captures time-budget information more efficiently [280]. Shiffman et al. [281] show that global reports of happiness are more predictive of future behaviors than momentary methodologies. Therefore, taking into consideration the pros and cons discussed above, researchers suggest a multi-method assessment, combing both global and momentary methods, to reach valid and accurate results [269, 282].

The first rows of Table 9 provide a summary of the traditional data sources, as well as their pros and their cons, as discussed previously. The remaining rows are explained later.

The dimensions of subjective well-being

Over the years, researchers studied subjective well-being and have identified the dimensions and the relevant determinants that can positively or negatively affect human well-being. Some studies rely on small data sets (e.g., review by Diener and Seligman[283]) reflecting the psychologists’ interest, such as personality, and some others use larger data sets, such as panel data (e.g., review by Dolan et al. [29]) reflecting the economists’ interest. These studies, conducted with the use of traditional data sources, and in particular with surveys, have shed more light on identifying in detail the determinants of happiness, which we divide into five main dimensions explained below:

Human genes

Evidence shows that one of the most important predictors of happiness is human genes, which is fairly heritable, with 30% to 50% range, since there is a variation on the results across studies [13,14,15,16,17,18,19,20]. Therefore, on average, about 40% of the variance of individual differences in happiness scores is accounted for by genes. Personality, which falls under our genetic makeup, can distinguish between happy and unhappy personalities. For example, extraverted individuals are happier to anxious and worried ones [284]. People higher in self-esteem are less likely to suffer from depression [29]. In addition, studies undertaken with data across different countries and periods of time, find influences of the following results: age has a U-shaped effect on happiness, with the highest level of happiness on the youngest and the oldest age and the lowest level of happiness on the middle age, between 32 and 50 years [29]; women are either happier than men, or there is no significant difference between them in almost all 73 countries investigated [285]. However, these results should be carefully interpreted. For example, Deaton and Tortora [286] show that the U-shaped relationship between happiness and age in West countries turns into a linear relationship in sub-Saharan countries, where there is unavailability of social services for older people.

Universal needs

According to the evolutionary theory [22] and human’s inherent growth tendencies [23], basic and psychological needs play an important role on happiness and are considered to be universal. In fact, Tay et al. [24], in a research conducted across 123 countries, show that life evaluation is associated with having basic and psychological needs, such as food and shelter, met (\(r=0.31\))Footnote 1; positive affects are associated with the fulfillment of social needs \((r=0.29)^1\) and the respect gained from other people \((r=0.36)^1\); negative affects are associated with the fulfillment of basic needs (\(r=-0.17\))\(^1\), respect gained from others \((r=-0.20)^1\) and autonomy needs \((r=-0.18)^1\) in terms of the degree of freedom in life. Therefore, according to Veenhoven and Ehrhardt’s Livability theory [287], some societies have a better quality of life because they highly satisfy the aforementioned universal needs. It should be noted that each of these basic and psychological needs is independent of one another, meaning that each of them is influencing happiness beyond the effects of others.

Fig. 2
figure 2

The figure relates the sources of data (left) with the dimensions of the subjective well-being (right)

Social environment

Many determinants fall under this dimension and can explain changes in the reported level of happiness. To begin with, education is an important determinant, which needs to be carefully studied since there is controversial evidence of its effects on happiness. Some studies of happiness economics suggest an insignificant relationship between higher education and happiness, whereas some others show a negative relationship between them [25,26,27,28]. On the other hand, other studies show that educated individuals tend to report more positive emotions and less negative ones, as well as more satisfaction with most domains of their life, such as financial, employment opportunities, etc., even when controlling for non-economic factors, such as marriage [288, 289]. Besides, studies show that health is an important determinant, with psychological health to be more strongly correlated to happiness, than physical health (e.g., review by Dolan et al. [29]). Climate is another determinant, which appears to have effects on happiness. Rehdanz and Maddison’s [290] study gives a reasonable indication that extreme weather is damaging to happiness. Moreover, living in an urban or rural area seems to influence happiness. In particular, living in big cities negatively affects happiness, whereas living in rural areas positively affects it (e.g., Hudson and Kyklos [291] for Europe; Hayo [292] for Eastern Europe). On the contrary, Rehdanz and Maddison [290] show different results on urbanization and ruralization. They demonstrate that population density does not affect happiness. Another important determinant is exercising. Naturally, Ferrer-i-Carbonell and Gowdy [293] show that people who exercise tend to have higher levels of happiness.

Economic environment

Income is one of the most discussed economic determinants of happiness. Easterlin [30] argues that while happiness and income show a positive relationship within nations, they show weak or no association between nations. He also shows that, across countries, although a relationship between happiness and income holds in the short-run, this is not the case over time. However, time-series and panel analyses across countries show that there is a positive relationship between income and happiness also in the long-run [32,33,34]. Veenhoven [31] challenges Easterlin’s findings by arguing that people’s happiness highly depends on the satisfaction of basic and psychological needs covered by income, which is more an absolute standard, than a relative standard. To the present day, studies support both arguments. Another source contributing to this debate is individual-level income or wealth and happiness data studies. For example, a longitudinal study with a sample of about 33,000 individuals shows that after two years, lottery winners rated higher happiness than non-lottery winners [294]. These contradicting findings confirm the complexity of the interpretation of the role of income and wealth on happiness, since the potential positive relationship between them may be moderated by other factors. For example, in the case of natural disasters, wealthier countries are more economically capable of providing financial aid to the people affected by the event [295]. Employment falls under the economic dimension category (see, e.g., [296, 297]), likewise income. Evidence shows that unemployed individuals report lower happiness than employed individuals [298]. In particular, Knabe et al. [299] demonstrate that unemployment has a substantial relationship with diminished cognitive well-being, but does not decrease affective well-being.

Political environment

As discussed previously, there are also political determinants associated with happiness. For example, Radcliff et al. [35] examine the effect of direct democracy, and in particular, the effect of the use of initiatives on happiness. They show that an individual’s happiness is higher in states where not only initiatives are permitted, but also policy-makers depend on these initiatives to form the political system. Political freedom falls under this dimension, as well. Veenhoven [36] shows that political freedom is highly correlated with happiness in developed countries. Another political determinant associated with happiness is social hierarchy in terms of the differences in power and prestige. Brule and Veenhoven [300] show that in northern and southern European countries, people are less happy in hierarchical societies. Last, social trust [301] and government quality [302] are political determinants that are substantially associated with happiness.

Data sources for monitoring the dimensions of subjective well-being

Similarly to Fig. 1 on objective well-being, Fig. 2 describes the new data sources (left) that have been used to estimate one or more dimensions of subjective well-being (right). The presence of a link in Fig. 2 between a data source and a dimension indicates that there are papers in the literature on monitoring that dimension with that data source. For example, b4 indicates the link between Google Trends data (b) and economic environment (4).

In this section, we describe, for each data source, its features (e.g., the process of data collection, its biases and limitations) and the main works in the literature that use it to measure several dimensions of subjective well-being. Table 9 provides a summary of the new data sources used to explore happiness, including the traditional data sources, as discussed previously. We aim to highlight the advantages and disadvantages of using each data source as a useful guide for future research on happiness.

Table 10 The table contains a subset of the information returned by a Twitter API

With the growth of technology, researchers are inclined to use more innovative approaches for the measurement of happiness. In fact, over the last years, researchers use novel methodologies and data sources, which offer new opportunities to study happiness and to circumvent the limitations carried from traditional methodologies and data sources. Fowler and Christakis study [303] is one of the first and most important to help the transition of happiness research from the traditional to the innovative era. The researchers computerised information from archived handwritten administrative tracking sheets from the Framingham Heart Study. They study happiness as a network phenomenon, by using data of 4739 people, from 1983 to 2003. Comparing to previous traditional work on happiness, which main focus is on socioeconomic, political, and genetic factors, this study is the first one to study happiness as a spreading phenomenon and its characteristics. In particular, they suggest that happiness is a network phenomenon, which clusters happy and unhappy people and spreads across various social relationships (e.g., relatives, friends) up to three degrees of separation (e.g., to one’s friends’ friends’ friends). Additionally, individuals that are central in the network are more likely to be happy in the future.

There are more than the study mentioned above in the innovative era, predominantly with the use of innovative big data sources. Although measuring happiness with new data approaches appears to be adequate in predicting the emotional component of happiness, most studies seem to neglect the structural component of happiness [304]. Below new data sources are described, and relevant studies are provided. We would like to underline that in comparison to objective well-being studies, researchers of subjective well-being usually explore more than one dimension.

Social media

Nowadays, people are highly involved in social media, and they are motivated to share their emotions and thoughts online, leaving a large and continuously updated user-generated content. Studying happiness from users’ posts may eliminate the social desirability effect that traditional self-reports bring, due to participants’ inaccurate and dishonest evaluation of happiness [305]. Thus, researchers and policy-makers are attracted by these intellectual opportunities to explore happiness, with wider use of Twitter data accessed through Twitter’s public API. Twitter has the least barriers in data extraction, while the other social media have strict policies, and the acquisition of data has turned to be difficult. Social media data may also encounter some concerns. They may reflect social desirability biases since individuals manage their online profiles [122]. Also, Twitter users may not be as representative of the general population [123] as anonymized self-reports conducted through a chosen representative sample. Table 10 illustrates an example of the structure of Twitter records.

There are several studies on social media (mostly on Twitter) showing the variations on happiness as influenced by the universal needs (a2), and in particular, the interaction with other people. For example, Quercia et al. [306] use Twitter data in order to monitor the gross community happiness in the city of London. In particular, they suggest that Twitter friends, on average, have similar sentiment. They also show that the relationship between sentiment and well-being can hold at individual and community level. Bollen et al. [307] use the OpinionFinder (OF) subjectivity lexicon [308] in order to analyze the sentiment of an online social network of 39,110 Twitter users. They show the first direct observation of a significant Happiness Paradox, meaning that on average most of the individuals are less happy than their friends are. Similarly, by using the OF, Bollen et al. [309] analyze the emotional content of a set of Twitter users over 6 months, to examine whether happiness is assortative in online social networks. They find significant levels of happiness assortativity across Twitter, since users might be propense to connect to users with similar happiness values (homophilic attachment) or converge on their friends’ happiness level (contagion). This result suggests that real social networks may work similarly. With the use of Facebook, Kramer et al. [310] test whether emotions are contagious between users without the awareness of the influenced individuals. Indeed, by reducing the amount of emotional content in the Facebook News Feed on an experiment conducted on Facebook users, they demonstrate that emotional contagion can also happen without direct interaction between the users and even without non-verbal cues.

Social media is also used for the exploration of happiness as influenced by the social environment dimension (a3). For example, Lim et al. [311] collect a set of geotagged tweets, of users in Melbourne, Australia, between the period of November 2016 to January 2017. They use sentiment analysis to demonstrate that people show more positive emotions and less negative emotions in green spaces or close to them. This could potentially be taken into consideration by policy-makers aiming to improve the societal well-being by urban greening interventions. Besides, Mitchell et al. [312] use Twitter data to study happiness and the 2010 United States Census Bureau’s MAF/TIGER database to define the urban areas. They use the Language Assessment by Mechanical Turk (labMT) sentiment analysis tool to study the similarities in word use in urban areas in the United States, to map areas according to the happiness level and score individual states and cities for average word happiness. Golder and Macy [313] identify individual-level diurnal and seasonal mood rhythms in cultures across the globe, using data from Twitter between February 2008 and January 2010. They find that people like the weekend as people are much happier on Saturdays and Sundays. They also find that even individuals’ good mood deteriorates as the day progresses, which is consistent with the effects of sleep and circadian rhythm. They also show that seasonal change in baseline positive affect varies with change in day length. Landsdall et al. [314] turn their attention to the issue of the public mood or sentiment—the mood of the nation. They use tweets sampled from the 54 largest cities in the UK from July 2009 to January 2012, and they associate each of the basic emotions (fear, joy, anger, sadness) with a list of words. They find out that each of the four key emotions changes over time in a manner that is partly predictable (or at least interpretable). Joy rises in Christmas, fear in Halloween, and especially negative mood started in October 2010, where massive cuts were announced in the UK. Cresci et al. [315] use Instagram data to explore, among others, the differences that the cultural and social environment bring on people’s smiles. They perform face recognition in a case study of over 2 million selfies shared from January to February 2015. In particular, they use a Face++ algorithm function to measure the smiling degree of the individuals in their selfies. Results reveal that El Salvador, Brazil, and Panama have the highest smiling average.

Other researchers use social media to study the variations of happiness as influenced by more than one dimension. For example, Bollen et al. [316] conduct sentiment analysis on Twitter data from 2008. They find that events in the social and cultural (a3), political (a5), and economic sphere (a4) have a significant effect on happiness. Dodds et al. [317] construct the Hedonometer to measure temporal patterns of societal happiness, as influenced by basic needs (a2), as well as by various social (a3), economic (a4) and political (a5) determinants. For indicating happiness using Hedonometer, they create a data set of users’ tweets over 3 years (from September 2008 to September 2011 approximately). The results show that in general, at an annual level, the average happiness appears to increase till April 2009 and then to decrease gradually. On a weekly basis, the average happiness peaks during the weekend and on an hourly basis, the happiest hour of the day is between 5 to 6 a.m. (US local time). Another example is Iacus et al. [318], who analyze tweets from Italy, written in the Italian language. In particular, they use the iSA (integrated Sentiment Analysis) method [319, 320] to capture a set of determinants that influence happiness, such as self-esteem (a1) and family relationships (a2), and aggregate them into an index labeled SWBI (Social Well Being Index). Results suggest that the environmental and health conditions (a3) anticipate several determinants of happiness as measured by SWBI. This study is one of the few to study both the emotional and structural components of happiness. Curini et al. [321] use tweets posted in 2012 in Italy to build a happiness index, labeled iHappy. They demonstrate that variables such as the overall quality of institutions (a5) seem to have a minor effect on the average level of happiness of the Italian provinces. In contrast, meteorological variables, such as rain and snow (a3), as well as events related to specific days, such as the payday (a4), have a stronger impact on happiness. Furthermore, Durahim et al. [322] use Twitter data to create the Gross National Happiness (GNH) for the country of Turkey. The GNH created measures people’s happiness as varied due to specific events, such as Saint Valentine’s Day (a2), Starting day of Gezi Park Protests (#occupygezi), and Day of Ergenekon lawsuit verdict (a5). Last, Coviello et al. [323] compare what people post on Facebook to data they have on the weather (a3), specifically the rainfall amount. They find that people tend to post less happy messages on Facebook if it rains. This emotion seems to pass along their network (a2). For example, if a friend on Facebook is in a rainy area and this affects the emotional content of her posts on Facebook, then more likely, her friends might post a sadder message, even though where they are the weather is better.

Google trends

Another new data source is Google Trends, which provides data on the frequency of specific search terms over time. Algan et al. [324] present Google Trends as a new data source for exploring happiness and its relevant dimensions. They consider it a promising data source for its timeliness, since it provides computational social scientists with immediate data, as well as offers the possibility to observe people’s behavior, as compared to analyzing textual opinions. On the other hand, working with Google Trends challenges researchers since the value of the series obtained directly from Google Trends is difficult to interpret, and this value on a given day cannot be compared between terms since they are normalized to the maximum value by term. In this study [324], researchers cover 300 weeks from January 6, 2008, to January 4, 2014. Results reveal that happiness is associated with job security, financial security (b4), family life (b2), and leisure determinants (b3). An example of Google Trends data set is not provided since data are represented as time series of the frequency.

Crowdsourced data

Crowdsourcing, as discussed in Sect. 2.2, involves obtaining work, information, or opinions from a large group of people who submit their data via the Internet, smartphone apps, etc. In particular, smartphones are lately appealing to happiness researchers since they give access to previously inaccessible data related to daily social behavior [325, 326]. Innovative smartphone sensor technology, such as accelerometers, GPS, and Bluetooth, are used in combination with self-reports, such as mood tracking self-reports, in the form of EMA. However, such methodologies bring the limitations of the traditional data sources (see the first rows of Table 9), since happiness fluctuations are collected through self-reports. Moreover, when hiring individuals to participate in crowdsourcing platforms, the crowd is not anymore for free, and the study might result in high costs. It is, therefore, hard to keep a trade-off between initial objectives with results of quality and cost [327]. Additionally, some studies are conducted with a small number of data and might need to be replicated. Table 11 shows an example of crowdsourced data.

Table 11 The table contains a subset of the information returned by HappyDB, a crowdsourced database capturing happy moments

For example, Lathia et al. [328] collect data of over 10,000 individuals, by combining smartphone-based self-reports (in the form of EMA) and the accelerator in the smartphones, to investigate the relationship between happiness and physical activity (c3). Results show that there is indeed a relationship between happiness and physical activities, including the non-exercise ones, such as standing and walking. Asai et al. [329] study 100,000 happy moments from HappyDB over 3 months, to find which are the short and long term determinants of happiness. In particular, HappyDB is a database created through Amazon Mechanical Turk, for capturing people’s happy moments by asking every 24 h and once over 3 months, people’s happiness status, and analyzing with NLP people’s responses. Results show that exercise, nature, and leisure (c3) are short-term determinants, whereas social relationships with loved ones (c2) and achievements (c3) are long-term determinants. Bogomolov et al. [330] exploit a data set of 117 individuals, who are equipped with a sensing software between 2010 and 2011. This software collects smartphone activity data of call logs, SMS and proximity data (acquired by scanning nearby phones and other Bluetooth devices every five minutes). It also collects personality traits (the “Big Five” [331]) and daily happiness data by self-report questionnaires. Results demonstrate that by using mobile phone data reflecting social interactions (c2), information concerning weather conditions (c3), and personality traits (c1), individuals’ daily happiness can be predicted.

News data

Similarly to objective well-being, news data are a new promising data source for the further exploration of subjective well-being. Its advantages and its disadvantages, as well as a data set example, are discussed and presented in Sect. 2.2. Carlquist et al. [332] study happiness with the use of news data. In particular, they study the concept of well-being in Norwegian society by examining word use patterns in four electronically archived Norwegian newspapers media from 1992 to 2014. They demonstrate that about half of the words referring to affective approaches, cognitive or life satisfaction approaches, eudaimonic and humanistic approaches, and character strengths show systematic and statistically significant patterns of change. The most notable rise concerns the eudaimonic words (related to mastery, motivation, and self-development), which show increasing trends in all newspapers. The authors state that certain happiness terms appearing more frequently could be interpreted as an increased and liberating focus on individual opportunity (d1) [333] or could demonstrate neoliberal ideology (d5) [334].

Discussion

In this study, we provide researchers with the theoretical background on both the objective and the subjective well-being or happiness, as well as their relevant dimensions necessary for the conduction of a meaningful study. In addition, we present a review of the data sources used for the exploration of well-being, and we discuss existing related studies. More specifically, we present the structure and the opportunities that each data source offers and the problems that researchers might encounter when working with these data.

The paper is primarily targeted at researchers interested in “Data Science for Social Good” (DS4SG) or similarly “Artificial Intelligence for Social Good” (AI4SG). Harnessed correctly, artificial intelligence can inform and empower the social good decision-making [335, 336]. DS4SG or AI4SG is a vague concept, and there is not an adequate definition yet. However, Shi et al. [39] propose several societal application domains to shed light on this concept, such as healthcare and well-being. In this study, we specifically aim to contribute to the exploration of well-being through data science. Researchers from various disciplines, from social science to computer science, could use this paper to understand data science for well-being better and make a positive and tangible social impact.

We would like to underline that this is not a complete review of studies conducted on well-being with the use of innovative data sources. We aim to provide some examples of the most important evidence on these data sources and well-being dimensions so that this study works as a reference point for future research. We do not fully cover existing research on a given link that is present in Figs. 1 and 2, but to the best of our knowledge, a missing link entails that there is no existing study connecting the two nodes. For example, there is no adequate literature on news data for the exploration of the safety dimension (E5) of objective well-being. Therefore, since nowadays, safety is an important dimension, due to constant conflicts around the world (e.g., political instability, terrorist attacks), it shows great potential for future research.

Moreover, new data sources seem to be particularly promising for a more in-depth exploration of subjective well-being. Taking into consideration the subjective nature of happiness, it has been traditionally measured through self-reports. Although they have been proved to be valid, they are very costly, and depending on the study might neglect to capture either the emotional or the structural component of well-being. Therefore, new data sources could be used, and innovative methodologies, such as text analysis, could be applied for a complete, according to its definition, measurement of subjective well-being. Still, most studies using new data sources tap into the emotional component of subjective well-being and neglect the structural component. Consequently, we suggest further exploration of the novel data sources for the measurement of subjective well-being, capturing both components.

Undoubtedly, the research opportunities opened up by the innovative data sources discussed in this paper are plenty. However, with the use of these data sources, researchers are called to deal with new challenges comparing to traditional research. Since, usually, the data used are personal, if not sensitive, and are analyzed to shape policy and to make decisions [337, 338], ethical concerns may arise, such as privacy and respect to human rights. In the European Union, additional attention to the topic has been brought after the implementation of the General Data Protection Regulation (GDPR). Researchers need to take into consideration the ethical challenges and not overlook them but address them successfully. Only by facing ethical problems, researchers can maximize the contributing value of data science studies for society.