1 Introduction

Nowadays, the identification of frail individuals is quite a popular research theme. Despite the growing interest regarding this topic, a unique definition for frail individuals still does not exist, and frailty is still defined as a syndrome in desperate need of description and analysis (Gillick 2001). Indeed, the identification of both common criteria and guidelines for frailty has been described as a highly complex and demanding task (Bortz 2002). This is because frailty is a complex and multidimensional concept that involves several functional domains of the elderly (Gobbens et al. 2010).

Attention toward the definition, measurement, and identification of frailty originates from the aging phenomenon. In fact, the average age of the population worldwide has increased, with the diffusion of age-related chronic conditions. In Europe, the number of people over 65 y is approximately 101 million (out of the total population of 512 million). Between 2018 and 2050, this number is expected to increase by 17.6% for those aged 65–74 y and by 60.5% for those aged 75–84 y. Italy has the highest percentage of people over 55 y, comprising over one-third its population. The portion of the population with a longstanding illness or health problem ranged from 56.9% (for people aged 65–74 y) to 72.5% (for people aged 85 y or more) (Eurostat 2019).

As such, in 2016, the Italian Ministry of Health issued a National Plan for Chronic Disease (NPCD) (Ministero della Salute 2016), a strategic policy framework for action aimed at improving the organization of healthcare services for old and chronic-disease patients, focusing on clinical decisions based on patients’ needs and calling into action all the involved stakeholders.

To reach these goals, the first step established was the stratification of the population. Algorithms and tools that exploit administrative healthcare data-flows to identify subgroups of the population that are homogeneous were defined in terms of health needs and characteristics. The added value of the initial stratification undertook policy measures that can be tailored based on the subgroup of the population to which the individuals belong. Moreover, the stratification represents a tool that can be exploited by all the actors of the healthcare sector, as it is based on administrative healthcare data. Specific disease registries based on administrative data are used for the stratification of the more frequent chronic diseases (e.g., diabetes registries).

However, among the older population and population suffering from chronic diseases, frail subjects are an additional category of individuals with special care needs. The identification of this specific population is particularly important to implement prevention policies and improve their overall quality of life (Bergman et al. 2002). Thus, the construction of a tool for measuring frailty among individuals using administrative healthcare data-flows assumes a fundamental role in both national and international scenarios.

This topic was already addressed in an article by XXX et al. (2019), where a composite indicator for frailty was proposed. The study used administrative healthcare data-flows coming from an Italian Local Health Unit (LHU) in the northern part of the Italian city of Padova (LHU 15 “Alta Padovana,” now part of LHU 6 “Euganea”).

The composite indicator proposed by XXX et al. (2019) is based on the fact that frail subjects are characterized by an increased susceptibility to fall into negative outcomes related to frailty condition (Fried et al. 2001). Indeed, given the absence of a unique and shared definition for frailty, the ability to predict adverse outcomes is considered the highest standard for a successful definition of a frail individual (Rockwood 2005). The two most cited negative outcomes are death and emergency hospitalization (Falasca et al. 2011). These two outcomes are also used for the definition of composite indicator in the work of XXX et al. (2019). These were defined using nine variables collected from administrative healthcare data-flows and were aggregated using the partially ordered set theory (poset).

However, from the perspective of a policy maker, the identification of frail subjects as individuals at a higher risk of dying or being urgently hospitalized because of unplanned reasons offers little room for improvement in patients’ health conditions. On the contrary, the NPCD advocates concrete actions directed to a global improvement of the quality of life. Indeed, taking into account the less severe and partially independent outcomes in the definition of the “frail elderly” may result in a more useful tool in stratifying the population. This may also be crucial in intervening with prevention policies to keep elderly health conditions from worsening. This is why we decided to enlarge the list of outcomes considered in the construction of the frailty indicator. This was done through the inclusion of four more possible outcomes: access to the emergency room with red code, avoidable hospitalization, hip fracture, and disability. By including multiple health outcomes, the indicator achieves the additional goal of representing the complexity of the frailty condition better. However, the inclusion of more outcomes likewise increases the complexity of the statistical methodology and some computational aspects involved in the variables selection.

This paper described the methodological proposal for a frailty indicator that faces complexity in the theoretical definition of frailty. Its representation using administrative data-flows of the Italian Piedmont region, the management of several outcomes, and its methodological and computational issues were also tackled. This paper aims to disentangle all these issues to propose a frailty indicator that involves a small set of variables that are easy to collect from administrative healthcare data-flows. Therefore, it becomes easy to implement and replicate in any Italian LHU. In particular, we answered the need for a composite indicator that is able to detect the frailest individuals compared with the population to which they belong using the poset methodology.

2 Steps for the Construction of the Frailty Indicator

This section summarizes the main steps that were undertaken for the construction of the frailty indicator (Fig. 1).

Fig. 1
figure 1

Steps for the construction of the frailty indicator

Full size image

The first step consists of literature recognition, looking for frailty definitions therein. As explained, there is no consensus as to a unique definition for frailty. Thus, the composite indicator is based on the fact that frail individuals have an increased susceptibility to adverse negative outcomes related with frailty condition (more details are provided in Sect. 3.1).

As a consequence, the second step for the construction of the frailty indicator comprises the choice of outcomes that are usually associated with frailty condition according to the literature. After careful consideration, six outcomes were selected as being related with the frailty condition: death, urgent unplanned hospitalization, access to the emergency room (ER) with red code, avoidable hospitalization, hip fracture, and disability (explained in depth in Sect. 3.2). These six outcomes are collected from administrative healthcare data-flows. Thus, their practical definition goes together with the structure of administrative databases. However, in some cases, it is quite delicate. For instance, the disability outcome was coded as the incidence (not prevalence) of disability in 2016. A brief description of data sources used in this work can be found in Sect. 4.

Outcomes are not directly included in the computation of the frailty indicator. However, they are still fundamental in the selection of the explanatory variables that constitute the composite indicator. Thus, the third step is strictly connected with the selected outcomes. It comprises a bibliographic research of predictive variables for the six outcomes (Sect. 3.3). The crucial point in this step is the codification of the variables and their construction with administrative healthcare databases. Furthermore, the practical definition of variables is quite delicate and strictly connected with some practical aspects related to the structure of administrative databases. For instance, chronic conditions shall be presented as dichotomous variables equal to 1 if the subject suffers from the disease. Variables referring to chronic conditions are the result of the union of both the information coming from different data sources, such as the participation in the prescription charges, as well as the territorial drug prescriptions and diagnosis assigned in hospital discharge records and in accident and emergency databases.

The final dataset contains 76 explanatory variables collected and computed through a deterministic record linkage (performed with a personal identification code and described in Sect. 4.3) of several different data sources (2014–2015) and the six negative outcomes observed on administrative databases of 2016.

The fourth step was on variables selection, based on their ability to predict all the six considered outcomes. This task is carried on through the estimation of 100 logistic regression models on balanced samples of the whole population for every outcome, using a stepwise selection criterion (described in detail in Sect. 5.1). The stepwise criterion allows the observation of both the presence of every variable in estimated models and the order of entrance of each variable in the models. Thus, the selection of variables is guided by both the median order of entrance and the percentage of presence of every variable in the models (the reduced set of variables contains 15 variables).

The fifth step involved the aggregation of variables using poset theory, fully described in Sect. 5.2. However, the number of variables selected is still too high both because of computational reasons and because of the inclusion of all of them would worsen the performance of the indicator. Thus, the sixth step involves variable selection again. However, this time it follows a forward approach to maximize the sum of the Area Under the ROC Curves (AUC) of the six outcomes. The steps of this procedure are described in Sect. 5.3.

The final composite indicator for frailty comprises seven variables aggregated through the use of poset theory. The final step for its construction includes performance evaluation and validation, as described in Sect. 6. This step mainly involves checking for some characteristics that are in accordance with the literature (Rockwood 2005), making a successful definition of frailty.

In the following sections, the steps for the construction of the frailty indicator are described in detail.

3 Frailty: Theoretical Framework Definition

3.1 Definition of Frailty

Within the epidemiological literature, the concept of frailty emerges with a multidimensional nature (Gobbens et al. 2010). As such, it does not refer to a unique context. Rather, it includes a combination of physical, sensory, psychological, and social domains. The presence of multiple facets that need to be considered together generates the challenge of determining a unique conceptual definition of frailty, which is currently still lacking.

In terms of usage and citations, the most popular definition of frailty was given by Fried et al. (2001), which has considered it in its clinical aspect. For Fried et al. (2001), frailty is defined as a biologic syndrome of decreased reserve and resistance to stressors, resulting from cumulative declines across multiple physiologic systems, causing vulnerability to adverse outcomes. In fact, the measurement system proposed by Fried et al. (2001) follows this clinical approach: the diagnosis is medical and based on the presence of symptoms of physical weakness, including weak muscle strength, slow gait speed, unintentional weight loss, and low physical activity. Another well-known definition is the one made by Rockwood (2005), who highlighted a more comprehensive view. For Rockwood (2005), frailty is a combination of aging, diseases, and other factors that make some people vulnerable, whose consequences affect the functional status as well as physical and cognitive deficits and psychosocial factors. Other definitions have been added (Buchner et al. 1992; Campbell et al. 1997; Strawbridge et al. 1998). Based on the experiment performed by Gobbens et al. (2010), the conceptual definition accepted the most by experts was the one produced by Schuurman et al. (2004). In his work, frailty was defined as a loss of resources in several domains of functioning, which leads to a declining reserve capacity for dealing with stressors. Eventually, Gobbens et al. (2010) himself has produced a more complete definition of frailty, which entails, beyond the physical and psychological aspect, also a social one (Gobbens et al. 2010).

Depending on the definition adopted, frailty prevalence can range from 7% (Fried et al. 2001) to 45% (Roppolo et al. 2015) among adults aged 65 y and older, with the occurrence of frailty increasing with age. Independently from the definition used, frailty is associated with the increased need for assistance in mobility, self-care, and activities in daily life. It is likewise associated with the progressive loss of self-confidence, leading to social isolation, reduced physical activity, progressive isolation, and decreased social interaction. These effects further exacerbate the level of frailty. Moreover, frailty is recognized as a continuum process, a relative state that can change over time (Gobbens et al. 2010). As such, it is possible to intervene in that pathway so that frailty is delayed, reduced, or prevented from worsening (Morley et al. 2002).

Two main points recur in researchers’ works and are crucial for the decision on our operational definition of frailty. First, all definitions of frailty refer to a state of susceptibility to adverse health outcomes. Second, experts’ opinions converge on the treatment of frailty as a catalyst in the process of worsening health conditions. In fact, frailty is said to confer significantly increased risk toward poor health outcomes, such as hospitalization and mortality.

Together, the two elements form the general meaning of frailty as a personal condition of having unfavorable outcomes in the future. Despite these two common characteristics of paramount importance, frailty indeed encloses all the previously mentioned facets. Thus, it requires to be tackled as an umbrella term. The next section will explain how the health outcomes considered in this analysis aim to include the multidimensional nature of frailty.

3.2 Adverse Health Outcomes Related to Frailty Condition

As mentioned in the previous section, frailty does not have a unique conceptual definition. Likewise, frailty has not yet run into a unique measurement system, and the present study’s contribution involves the research of a better method to identify the term and to reach consensus in the clinical community. Our approach to determine our operational definition is to focus on the envisaged goal. We reckon that the objective of acting upon frailty in a specific context conducts to the proper operational definition of frailty for that context. In our case, the need for the definition of frailty is driven by the implementation of the NPCD at the regional level. This calls for action in the anticipation and the slowing down of the clinical worsening of elderly people. Hence, the main point we are interested in is frailty being considered a global deterioration of health conditions, which in turn may cause adverse clinical events. The higher the level of frailty, the higher the probability that people may incur one or more of these events. Through the mapping of these clinical events, subjects who tend to be frailer can be identified. As such, a preventive and supportive policy can be addressed toward them to reduce the probability of them incurring additional negative health events.

Table 1 summarizes the main outcomes that have been used in recent works on frailty. In particular, since our aim is to use only administrative healthcare data, we reported only those papers that use prediction models based on this type of data.

Table 1 Literature sources for the negative outcomes related with frailty conditions
Full size table

The most widely used outcome is death, which ultimately resumes all the processes that can worsen the health status. However, the inspiring principle of the NPCD is to slow down the process which leads to such an outcome before it becomes unavoidable. Hence, we decided to add to our analysis of other types of outcomes to highlight the aspects of frailty that are preventable and can be dealt with by the NPCD. In particular, three types of events are of interest to this approach: (1) emergency admissions and urgent hospitalizations, as they often refer to the exacerbation of a chronic disease, thus indicating failures in the continuity of care; (2) avoidable hospitalizations, which represent the volume of hospital care potentially preventable through access to timely and effective primary and outpatient care within the local health system organization. Its variability also depends on the individual characteristics of the patients; the same, likewise, allows for the identification of a subgroup of vulnerable people for whom the performance of primary and outpatient care is sub-optimal; and (3) events related to a new disability, i.e., progression in functional limitations, and to unfavorable housing conditions or poor family support, usually triggered by failure in health–social integration. All these events often hide failures either in organization or in the coverage of essential care, which can be subjected to improvement policies, such as those that the new NPCD seeks to take charge of. We decided to exclude outcomes related to the mental status of individuals, as they denote a more complex status that includes a combination of more biological and social aspects (all of which are different in terms of prediction or control).

Thus, referring to the three areas of interest, we decided to include the following six adverse events: death, urgent hospitalizations, admissions in emergency department with red code, avoidable hospitalizations, hip fracture interventions (as a proxy of safety risks at home), and disability. In this way, the multidimensional nature of frailty is accounted for. Moreover, we are able to reinforce the idea of frailty as a continuum process, the progress of which can be positively influenced by actions directed to the population and individuals.

3.3 Explanatory Variables

There are a number of risk factors within each of the dimensions that have been described as involved in the frailty process. Among them, several and complex interactions exist, such as the total level of frailty not being equivalent to the sum of its components (de Vries et al. 2011).

Four main categories can be outlined as containing all the risk factors that have been considered in the literature: personal, social, clinical, and psychological or behavioral risk factors. The majority of the risk factors that have been associated with frailty refer to evaluations that can be performed only through face-to-face visits. Because we used administrative healthcare data in our algorithm, there are certain limitations, such as the absence of information on the physical status of people, their lifestyles, and more. Table 2 reports the risk factors that have been explored in the literature and can be measured with administrative healthcare data.

Table 2 Variables considered for the frailty indicator, divided by category
Full size table

To ensure that all aspects related with the frailty condition were taken into account in our analysis, all variables found in the literature, which are collectible from administrative healthcare data-flows, were considered in the first stage of the variables selection.

4 Data

4.1 Definition of the Population

Our analysis involves people aged 65 y and older, considering their ages computed as of December 31, 2016. This specific age group was chosen mainly because the health conditions and morbidities of people differ depending on their age. Our observation period covered three years: 2014 and 2015 were used to observe explanatory variables to be included in the indicator, while 2016 was used for the observation of the negative outcomes related to the frailty condition. We used the Piedmont population-assisted register to select people aged 65 y and older as of the end of December 2016. The total population was 1,095,613.

4.2 Administrative Healthcare Databases

This research has been performed using the longitudinal study of the Piedmont (SLP) resident population (comprising over 4 million). The SLP is a record linkage among population census (2011 data) and the main administrative healthcare databases. The variables used to create the frailty index were observed in 2014 and 2015. The outcomes were then observed in 2016. More specifically, we used several databases to describe individual health conditions:

  1. 1.

    Archives of hospital discharges (2014–2015): These contain all the information collected on

  2. 2.

    hospitalizations, including the principal diagnosis and up to five secondary diagnoses.

  3. 3.

    Archives of drug prescriptions (2014–2015): These list all the prescribed drugs and their quantities, and classify each drug according to its therapeutic, pharmacological, and chemical properties based on the Anatomical Therapeutic Chemical Classification system.

  4. 4.

    First aid (2014–2015): These records contain all the information collected in ERs, including the principal diagnosis, triage color code access, and color code exit.

  5. 5.

    Income exemptions (2015): This dataset collects information on service activation, beneficiaries, starting and ending dates, and exemption codes.

  6. 6.

    Home care service (2015): This dataset collects information on service activation, beneficiaries, starting and ending dates, treatments, and diagnoses.

  7. 7.

    Medical exemptions (2015): This dataset collects information on service activation, beneficiaries, starting and ending dates, and medical exemption codes.

To describe social characteristics, we used the population census from 2011. This census collected individual information about education, occupation, civil status, family type, and house condition. To define death outcome, we used the mortality register containing information on peoples’ date of death and the causes of death.

4.3 Record Linkage

The abovementioned datasets are designed for administrative purposes; thus, records represent events, rather than patients. Pertinent to our study, the first step needed to use these data was to rearrange the rows of data so they represent individuals. For example, the Territorial Drug Prescriptions dataset can contain several rows, one for each prescription or for the same patient. Ideally, we needed a single row containing all the drug prescriptions for a given individual.

The next step consists of the linking of all the datasets to obtain all the available data for each individual. A deterministic record linkage of all the training datasets with the Piedmont population-assisted register was run. The linkage was based on individual codes, and it generated a single table that contained all relevant events (from 2014 and 2015), characteristics of people living in the sample area, and their health outcomes in 2016.

5 Methods

5.1 Variables Selection

We commenced the variables selection process with 76 variables. This set of variables included age (divided into six five-year age classes), some categorical and ordinal variables describing mostly socio-economic characteristics, some dummy variables indicating the presence of a condition (i.e., chronic disease), and some counting variables quantifying the use of some services (i.e., hospitalizations, access to the ER according to the color code, and prescriptions).

However, because we needed to select a small set of the most meaningful variables that best described the frailty condition of the whole population, we first discarded low-prevalence characteristics (less than 1% cases in the whole population) and variables with no significant association with the outcomes. Indeed, low-prevalence characteristics, although representing critical aspects for some individuals, do not help in the description of the health of the majority of the population. They may be also somehow otherwise represented by other more general variables. In this way, the number of considered variables was reduced to 57.

Some of the variables that we took into account are counting variables. These variables should be reduced in classes to be included in the construction of a composite indicator. To end up with ordinal variables with a restricted number of meaningful classes, we conducted a decision tree analysis that guided us in identifying the groups of values that better classify individuals according to the risk of fall into the considered outcomes. Having a large number of outcomes, the consensus about the best overall classification is difficult to reach. Thus, in doubtful situations, death, as the most severe outcome, was used to make the ultimate choice.

The final goal of this step was to identify a parsimonious set of variables capable of simultaneously predicting the six outcomes we have considered. However, the six outcomes may have quite different risks. Hence, the first step was the separate observation and selection of explanatory variables for each outcome. In order to do so, we used logistic regressions where the outcome was the dependent variable. All the 57 variables were considered for a regression and then selected using a stepwise selection criterion. Having a large population, a great number of variables had enough significance to be included in the final model. For this reason and in order to overcome over-fitting, we decided to implement a three-step procedure to select important explanatory variables for every outcome:

  1. 1.

    Selection of a random sample without replacing 75% of the whole population to avoid over-fitting.

  2. 2.

    Balancing of the sample to have the same amount of subjects that fell into the outcome (cases) and of those subjects that did not experiment the same (controls). The balanced sample contains all the cases and a random sample of controls of a size equal to the number of cases.

  3. 3.

    Logistic regression with a stepwise selection criterion. This step allowed us to observe if the variable was included in the final model and its rank of inclusion, as a proxy of its importance in the prediction of the outcome.

These three steps were repeated a 100 times for each outcome to get general results, not strictly linked with the analyzed population.

Two measures were obtained from the results repeated for each outcome for every explanatory variable: percentage of presence and the median order of entrance (Table 3). The percentage of presence is the number of times a variable was selected in the final version of the 100 models for a given outcome. The median order of entrance is the median rank of entrance in the models, considering all the 100 logistic regression models performed for every outcome. To obtain this measure, we assigned the maximum possible rank (that theoretically is 57, as the total number of considered variables) when the variable was not selected in the final model. However, these extreme values did not affect the median. When the median was equal to the maximum possible rank, it is denoted using an asterisk in Table 3.

Table 3 Percentage of presence and the median order of entrance of the selected variables for the six outcomes
Full size table

Having two measures that summarize the importance of every variable in the prediction of every outcome, the following step comprises the selection of a unique, relevant set of variables with respect to all the six outcomes. Thus, we selected a small set of variables relevant to the prediction of all the six outcomes according to the two defined measures above: variables with percentage of presence ≥ 75% for at least 3 outcomes and the median order of entrance ≥ 15 for at least 3 outcomes. In this way, 15 relevant variables were selected (enlisted in Table 3). Although these criteria were chosen empirically to select the variables located in the best quarter, according to both the percentages of presence and median order of entrance, we also performed a sensitivity analysis to guarantee their robustness (described in detail in Sect. 5.3.1).

5.2 Partially Ordered Set Theory to Aggregate Variables

Once the variables are selected, we need to aggregate dichotomous and ordinal variables to create a composite indicator for frailty. To this end, poset theory is a powerful tool.

In mathematics, poset is a set of elements wherein a binary relation indicating an order can be traced. The word “partial” refers to the fact that not every pair of elements could be comparable.

To easily explain the poset theory and its use in the computation of the composite indicator for frailty, we propose a toy example that involves six subjects and three dichotomous characteristics (Fig. 2).

Fig. 2
figure 2

A toy example to understand the poset theory: part (a) contains a group of observations; part (b) represents the Hasse diagram of the subjects in table (a); part (c) lists all the linear extensions for the subjects; and part (d) presents the exact average rank for the individuals of table (a) and their normalized values

Full size image

Suppose that we have a population with six individuals characterized by three dichotomous variables, as represented in Fig. 2a: age (which takes a value of 0 for individuals who are between 60 and 70 y old and 1 if they are older); drugs (which takes a value of 0 if individuals had no drug prescriptions in 2015 and 1 otherwise); and first aid (which takes a value of 1 if individuals had access to ER at least once in 2015 and 0 otherwise). These variables are ordered, where a value equal to 1corresponds to the worsening health conditions. We are assuming that they are able to summarize individuals’ frailty condition. Thus, every subject is represented in the poset by its profile given by the set of its characteristics.

The comparison of the individuals in the population gives rise to a list of comparabilities and incomparabilities (Davey and Priestley 2002). These can be represented in a graphic form called the Hasse diagram (Fig. 2b). This diagram represents the elements in a poset: Each node is an element, two or more equal elements still form one node, and every line segment is an order relation between comparable objects. When two individuals are comparable, they are connected by line segments in the diagram, like A and B, B and E, or C and E, whereas there is no ascending or descending path between incomparable elements, like B and C. Indeed, B and E are the same age, but E has more drug prescriptions than B. Thus, we expect E to be frailer than B. While C and E have the same value for drug prescriptions, C is younger than E. Thus, we expect E to be frailer than C. However, the comparison between C and B is not possible because C is younger than B but has a higher value for drug prescriptions.

As a consequence of incomparabilities, several possible rankings of subjects according to their frailty level are possible. All the possible rankings of elements in the poset that respect its comparabilities (the connections in the Hasse diagram) and incomparabilities (Brüggemann and Patil 2011) are called linear extensions (Fig. 2c). To summarize all the information provided by linear extensions and to generate an idea about the position that each node (profile) assumes in the whole poset of elements, it is possible to compute the average rank (AR). The AR of a node represents the mean of all the ranks that the same element occupies in all possible linear extensions, starting from the known order relations, as listed in Fig. 2d.

The AR can then be normalized to make it vary between 0 and 1, for sake of interpretability, using the min–max normalization method (Fig. 2d). Therefore, the normalized AR becomes an indicator that describes the relative position of an individual in the distribution of the latent concept—in this case, it is frailty condition.

If the numbers of individuals and variables increase, the linear extensions become too many to be examined thoroughly. Hence, it becomes almost computationally impossible to find the exact AR, as in the example in Fig. 2. That said, researchers interested in the computation of AR have used two main approaches to obtain a computationally efficient calculation of the AR by sampling linear extensions (Fattore 2016; Lerche and Sorensen 2003) or by the definition of an approximation formula (Brüggemann and Carlsen 2011; De Loof et al. 2011).

Different approximation formulas have been proposed in literature, such as the Local Partial Order Model (Brüggemann and Carlsen 2011) or the one based on Mutual Probabilities (De Loof et al. 2011). The present work is based on the De Loof’s approach (2011), because it provides better results in terms of accuracy with a large sample size as compared with other methods (De Loof et al. 2011). The approximated AR was computed using the R software, also utilizing the package proposed by Caperna (2019) that can cope with large datasets (e.g., Boccuzzo and Caperna 2017; Caperna and Boccuzzo 2018; Caperna 2016).

Given that the main aim of our composite indicator is to detect the frailest individuals compared with the population to which they belong, the AR fits the purpose, as it arranges individuals in order according to their frailty level. However, it is important to remember that the AR summarizes the relative position of the subjects in the specific population they belong to. Therefore, the indicator is strictly population dependent, and it is not possible to interpret it in absolute terms. It is also not possible to use it to measure the distance between individuals, because it represents ranks.

5.3 Variables Selection Based on Poset

A poset created with all the 15 selected variables could become too complicated, leading to information potentially being lost due to an excess of incomparabilities between the profiles. Moreover, from a practical perspective, the collection of such a high number of variables to replicate the indicator may be too demanding. Thus, it is preferable to reduce the number of variables that compose the frailty indicator. This second selection process follows a forward logic to select the most parsimonious set of variables that maximizes the indicator’s performance. At this stage, the indicator’s performance was evaluated by observing the AUC with respect to the six outcomes. Furthermore, to have a unique measure to compare performances, we considered the sum of the six AUCs. To evaluate the relative contribution of each variable to the increase in the sum of the six AUCs (one for every outcome o) we computed, for every step s and for every variable v the relative contribution (RCs, v) of each variable in percentage is as follows:

$$ RC_{s,v} = \frac{{(\mathop \sum \nolimits_{o = 1}^{6} AUC_{o,s,v} - \mathop \sum \nolimits_{o = 1}^{6} AUC_{o,s - 1,v} )}}{{\mathop \sum \nolimits_{o = 1}^{6} AUC_{o,s - 1,v} }} \times 100 $$

This selection procedure consisted of the computation of the frailty indicator several times with a growing number of variables. At every iteration, the variables were selected, maximizing the sum of the AUCs of the six outcomes, thus choosing the variable v with the highest RCs, v.

Given the fact that the age variable is fundamental in identifying frail individuals, we selected age as the first variable and computed 14 composite indicators with only 2 variables (i.e., age and another variable chosen from among the remaining 14). The indicator composed of age and poli-prescriptions was the one with the highest RC2, v among the 14 indicators considered. Therefore, to select the third variable, we computed 13 composite indicators with 3 variables (i.e., age, poli-prescriptions, and another variable chosen from among the remaining 13). The selection process is summarized in Table 4, where all the AUCs at every step and the added variables are reported.

Table 4 Steps of the variables selection process with a forward approach: the AUCs for all the six outcomes and added variables
Full size table

We continued with the same procedure until we produced all the indicators with eight variables. We noticed that even the indicator that maximizes the sum of the AUCs worsens the performance of the indicator with respect to the one with seven variables (as shown in Fig. 3). Thus, the RC8, v was negative for every variable v. This is because the addition of the eighth variable increases the incomparabilities and the entropy of the poset. Moreover, the computational time to produce the composite indicator with eight variables is approximately 30 min (as shown in Fig. 3). Such time is considerably higher than the time used by the same R function to compute for the indicator with only one variable less (which was approximately 10 min).

Fig. 3
figure 3

The AUC sum and the computation time for every step of the variables selection process with a forward approach

Full size image

For these reasons and for the sake of parsimony, we preferred the indicator that involved only seven variables: age, poli-prescriptions, Charlson Index, disability, number of accesses to the ER with yellow code, number of accesses to the emergency room with green code, and the presence of Parkinson’s disease. These 7 variables produce 2592 profiles that may be theoretically present in the population. However, only 1697 were actually represented. Thus, the frailty indicator assumed 1697 different values.

5.3.1 Sensitivity Analysis and Variables Selection

To verify the robustness of the selection procedure of the 15 variables considered and further selected in the previous section, we performed a sensitivity analysis using different thresholds for both the percentages of presence as well as the median order of entrance and their combinations. As shown in Table 5, we evaluated the combination of five thresholds for the percentages of presence (≥ 60%, ≥ 70%, ≥ 75%, ≥ 80%, and ≥ 90%), with three-limit values for the median order of entrance (≤ 10, ≤ 15, and ≤ 20) and all their combinations. As a consequence, variables were divided into four main groups:

  • Group A includes eight variables common to almost all the combinations of the considered criteria (one of these variables, Neoplasia, is discarded in all combinations where the percentage of presence is set to be greater than or equal to 90%);

  • Group B consists of a total of 15 variables, including group A and seven more variables that are common to almost all cases where the threshold for the median order of entrance is ≤ 15 or ≤ 20 (except for the Neoplasia variable where the percentage of presence is set to be greater than or equal to 90%);

  • Group C includes group B and four more variables that are included in at least one of the considered combinations; and

  • The group of discarded variables that was not included, on the basis of the choosing criteria.

Table 5 Criteria considered in the sensitivity analysis for variables selection and variables subgroups selected in each combination (the criterion used in the variables selection is written in bold)
Full size table

Corresponding to each combination of thresholds, we performed the same variables selection based on poset. Likewise, we worked with a forward logic that is described in the previous paragraph. The choice of variables is driven, step by step, by the relative contribution of each variable in percentages (RCs,v). The selection procedures are shown in Fig. 4.

Fig. 4
figure 4

Relative contribution of each variable (%) for every step of the sensitivity analysis of the variables selection process

Full size image

The selection is identical for all the combinations of thresholds until the fourth step. The fifth variable to be included in the frailty indicator belongs to group B but not to group A. Thus, for median order of entrance lower than or equal to 10, the fifth variable to be chosen is the second best according to the RCs,v criterion. Considering this variation in the selection process and the fact that variables included only in group C have a lower relative contribution than some other variables, only two possible versions of the frailty indicator are computed in step 7. The better between the two versions, with the highest sum of AUCs, is the final version of the frailty indicator described at the end of the previous paragraph.

In conclusion, the set of variables that are included in the frailty indicator is robust to the arbitrary and empirical choice of thresholds for the percentage of presence as well as the median order of entrance and their combinations.

6 Results

6.1 Descriptive Statistics

The frailty indicator varies between 0 and 1, as explained in Sect. 5.2. However, it assumes quite low values on an average. Indeed, its mean is 0.104 in the whole population. More than 10% of the population has indicator values equal to 0 (the lowest level of frailty). These individuals are 65–69 y old and do not present any of the risk factors included in the composite indicator. Approximately 2% of the whole population has indicator values higher than 0.5. Indeed, the 99th percentile is equal to 0.586 (Table 6).

Table 6 Distribution of the frailty indicator in the whole population
Full size table

Among women, the mean of the indicator is equal to 0.109. The value is quite high in comparison to that of men (the same situation is represented for median values in Fig. 5), which is 0.097. However, observing the means standardized by age (assuming an age structure equal to the whole population for both men and women), the relationship is inverted, with the standardized means equal to 0.101 for women and 0.107 for men.

Fig. 5
figure 5

Frailty indicator by individual characteristics (Gender, Disability, and Co-morbidities measured by the Charlson Index and age); medians and interquartile ranges

Full size image

The median of the frailty indicator for the whole population is 0.048, which is quite low with respect to the total range of the indicator. However, we may observe some differences in the median if we consider the different subgroups of the population. Figure 5 represents the medians and interquartile ranges for some variables that, except for gender, are included in the computation of the composite indicator. Variables included in the indicator are ordinal with the same direction. Thus, higher values of the variables correspond to higher values of the indicator. Moreover, the values of the indicator in the groups shown in Fig. 5 are significantly different for all comparisons with a significance level of 99%, according to the Mann–Whitney test.

Values for individuals with and without disability are separated. This means that the presence of disability is highly distinctive in the evaluation of the level of frailty of every individual.

The same distinction is also present between the values of the indicator for those subjects with and without co-morbidities. Individuals with no co-morbidities had no hospitalizations in 2014–2015 or had hospitalizations with only the principal diagnosis and no co-morbidities as secondary diagnosis. Mild and severe co-morbidities are assigned in the graph to individuals with at least one hospitalization and at least one co-morbidity as a secondary diagnosis that may be mild (1 or 2 on the Charlson Index) or severe (higher than 2 on the Charlson Index).

Interquartile ranges of mild and severe co-morbidities are quite large and overlapping. We can likewise observe overlaps in the interquartile ranges of the age variables and medians that increase with age values. Indeed, a healthy (without other negative conditions included in the composite indicator) individual over 80 y of age will have a lower value on the frailty indicator as compared to an individual who is 65 y old with other conditions considered in the indicator.

Another aspect we want to underline is that interquartile ranges of the age variable grow larger as age increases. It may be explained by the fact that it is more common for older people to have combinations of negative conditions included in the composite indicator that may considerably raise the assigned value of the composite indicator because of implicit interactions among variables.

In conclusion, Fig. 5 presents some important characteristics of the frailty indicator, such as the presence or absence of overlapping interquartile ranges according to variables’ distinctive power, the threshold relationship between the indicator and co-morbidities, and the presence of implicit interactions among variables.

The aforementioned characteristics of the indicator come from the aggregation method that we used to create it, which was based on the poset theory. Indeed, as the aggregation method is not additive, values assumed by the indicator corresponding to different combinations of variables are not given by the weighted sum of several variables. Rather, it depends on the reciprocal position of the complete health profile of individuals in the population. This is basically why the indicator is also able to take into account the presence of implicit interactions among variables. An example can be seen in Fig. 6, where there are represented medians and interquartile ranges for combinations of age and co-morbidities (values of the indicator are significantly different corresponding to all combinations, with a significance level of 99%, according to the Mann–Whitney test). The marginal impact of these two variables on the indicator changes corresponding to their combinations. Values of the indicator for individuals with and without co-morbidities are quite different, especially for subjects older than 80 y. At the same time, values of the indicator for individuals with mild and severe co-morbidities are more overlapping for subjects older than 80 y age as compared with younger age classes.

Fig. 6
figure 6

Frailty indicator by age and co-morbidities measured by the Charlson Index; medians and interquartile ranges

Full size image

However, a more precise idea about the role each variable plays in the identification of frail individuals is given by Fig. 4 as well, where the effect to the sum of AUCs in terms of relative contribution is reported for every step of the selection of variables. In general, the contribution of an additional variable decreases together with the increase in the number of variables being included in the frailty indicator. The drop is not linear. It depends on how much the profiles created by every combination of variables included in the indicator are informative and are able to predict the negative outcomes related with frailty conditions.

6.2 Performance of the Indicator

As explained before, we built the frailty indicator considering the fact that frail individuals have an increased susceptibility to adverse negative outcomes related with the frailty condition. Thus, the first step to evaluate the performance of the indicator was to observe its relationship with the outcomes. The indicator is associated to high AUC with respect to all the six outcomes (0.816 for death, 0.706 for access to the ER with red code, 0.695 for urgent unplanned hospitalization, 0.773 for avoidable hospitalization, 0.753 for hip fracture, and 0.781 for disability), as shown in Fig. 7.

Fig. 7
figure 7

ROC curves for the six outcomes

Full size image

Moreover, the frailty indicator likewise classifies well the individuals that experienced at least one outcome in 2016, regardless of which outcome or outcomes they incurred. Indeed, the AUC for having at least one outcome is 0.769. However, this increases if we consider a higher number of outcomes until the value of 0.845 for individuals that had at least 4 outcomes in 2016 (Fig. 8).

Fig. 8
figure 8

ROC curves for the increasing number of outcomes suffered in 2016

Full size image

The choice of variables to be included in the frailty indicator was driven by their ability to predict the selected outcomes. Therefore, the good performance in terms of ability to predict negative outcomes related with frailty comes by construction. However, the frailty indicator assumes different values in different subgroups of the population. For instance, individuals with chronic diseases have higher values on the indicator than other subjects (significantly different for all chronic diseases shown in Fig. 9, with a significance level of 99%, according to the Mann–Whitney test). This is true even for diseases that are not directly included in the computation of the frailty indicator (Fig. 9). Parkinson’s disease is one of the variables that are included in the computation of the composite indicator for frailty. Thus, medians and interquartile ranges are quite far and do not overlap with respect to individuals with or without the disease. However, this is not the only case in which we find such differences between healthy and sick groups. Similar situations arise for other chronic conditions not included in the frailty indicator, such as dementia, chronic obstructive pulmonary disease, and chronic heart failure.

Fig. 9
figure 9

Frailty indicator by chronic conditions; medians and interquartile ranges

Full size image

Even if some socio-economic variables were available from the 2011 census, the variables selection process discarded all of them in favor of variables that represent the use of services and chronic conditions. Some concerns about this indicator may be with regard to its ability to represent and also take into account the socio-economic aspects of individuals and their conditions. The empirical cumulative distribution functions of the indicator’s values for three different education levels are represented in Fig. 10. It also distinguishes between individuals with no title, those with just primary education, and those with secondary or higher education levels. Mean values of the indicator are quite different and are inversely proportional with respect to the three education levels: 0.076 for secondary or higher degrees, 0.123 for primary degree, and 0.177 for individuals with no titles. Some of these differences are due to the different age structures in the three groups. Thus, we computed and compared also the standardized means (assuming an age structure equal to the whole population for all the education levels). The relationship between the frailty indicator and education remains, but is slightly weakened.

Fig. 10
figure 10

Empirical cumulative distribution functions of the frailty indicator by education level

Full size image

In conclusion, observing the differences in the indicator’s values for subgroups that are not directly represented as variables in the computation of the indicator means the small set of the seven chosen variables in the two described selection processes is extremely powerful. Even if there are only seven variables used for the computation of the frailty indicator, our composite indicator for frailty is able to predict the six outcomes as well as describe several individual characteristics (such as chronic conditions and education level).

6.3 Success of our Proposal

Given that a unique definition for frailty does not exist and that it is not possible to observe and measure the real value of frailty for every individual, the validation of the frailty indicator is not trivial. However, we may get some indications from Rockwood’s (2005) work. His work enlists some characteristics that an operative definition of frailty should have to be successful. These criteria for a successful definition of frailty are grouped into three categories: content validity, construct validity, and criterion validity.

Content validity refers to whether the definition makes sense with respect to first principles (Rockwood 2005). Among the criteria to assess content validity, there are parameters that the definition should include multiple determinants but should also simultaneously be dynamic and computationally tractable. Our proposed frailty indicator meets these characteristics. Indeed, it includes multiple determinants in three different ways. First, it includes seven variables that represent different aspects of the elderly, such as age, use of services, chronic conditions, and health characteristics. Second, the variables are selected according to six different outcomes that represent different stages of frailty. Third, as indicated in Sect. 6.2, the indicator also indirectly includes some other aspects, such as chronic conditions and education level. The dynamicity of the indicator is given by the poset methodology. Indeed, even if no assumptions are required for its computation, it is able to catch interactions and non-linearity in the data. Moreover, assuming that the same set of variables (that were chosen in a very robust way) is important to measure frailty of any population, it is possible to compute our frailty indicator even in different populations or in the same population in different years. This may be done without the need to repeat the variables selection. Using the R function (Caperna 2019), it is possible to compute values of the indicator in approximately 10 min (on a population of 1,095,613 subjects). Thus, the indicator is also computationally tractable.

Construct validity exists when the operational definition coheres with other measures of the phenomenon: frailty should be more common among women than among men, higher frailty indicator values at higher ages, and the relation to disability and co-morbidity (Rockwood 2005). As shown in Sect. 6, our frailty indicator is higher for women than for men on an average. Moreover, by construction, its values are strictly related with age, disability, and co-morbidities, because these are the variables included in the computation of the indicator.

Criterion validity deals with the ability of the frailty definition to predict adverse outcomes, especially death (Rockwood 2005). In Sect. 6.2, we reported the AUCs of our indicator with respect to six outcomes related with the frailty condition. Our indicator has the best performance with respect to death (AUC = 0.816) but shows good performances with respect to the other outcomes. Thus, we may say that it likewise meets criterion validity. Moreover, an important strength of our indicator is its ability to predict six outcomes at the same time with good performances. Common methods used to measure frailty usually are only focused on one outcome at a time. Thus, the comparison of the performance of our indicator with other measures of frailty in terms of AUCs is meaningless. This is because any other frailty measure with a multiple-outcome approach does not exist in the literature.

In conclusion, our frailty indicator has some generally good properties and meets many of the requirements that successfully form the definition of frailty with respect to literature.

7 Conclusion

In the context of the regional implementation of the National Plan for Chronic Diseases, there was a need to identify multi-morbid patients at a higher risk of clinical worsening, for whom preventive actions could be taken and who may benefit from better disease management. We developed a frailty indicator that provides a score for all individuals over 65 y of age in the Piedmont region. It generates a graduated classification of individuals useful for this purpose.

This indicator provides a summary measure for frailty, apart from its complex and multidimensional nature. It can be particularly useful for the implementation of the NPCD at the local level, because it is built with a small set of variables obtained from current administrative healthcare data-flows, which are usually available in all Italian LHUs. Although the methods used are quite sophisticated, the indicator is easy to calculate. This is because of the construction of a user-friendly application that returns the individual frailty indicator after a guided loading of a suitable input data file. Moreover, the same frailty indicator might also become an individual variable to be used for further analysis, as it represents individual health conditions and predicts negative outcomes related to frailty condition.

As mentioned earlier, the validation of the frailty indicator is not an easy task. As the first step toward a more thorough validation analysis, we compared our results with the subjective judgment of a small sample of 10 general practitioners (GPs). These participants joined the project on a voluntary basis. We, therefore, selected all the patients aged over 65 y and asked the GPs to classify them according to their knowledge of the clinical and social conditions of their patients. They were ultimately divided into three classes: not frail, at risk of frailty, and definitely frail. We then analyzed the performance of our frailty indicator in the three classes of patients identified by the GPs. The results showed statistically significant differences among the groups: The mean score was 0.09 in the first class of patients identified as “not frail” (n = 1402), 0.14 in the second class of patients identified as being “at risk of frailty” (n = 742), and 0.23 in the class of patients identified as “definitely frail” (n = 839). Although based on a small and selected sample of GPs, these results are in the expected direction of an increasing score with respect to increasing frailty. There is no reason to think that the algorithm should perform differently among other groups of patients.

The next steps of validation will involve both the validation of the indicator in different populations and times as well as a sensitivity analysis with respect to the construction of variables.

In conclusion, this frailty indicator appears to be a promising tool for the identification of patients to whom preventive actions should be addressed as a priority. Once its validity and applicability have been further evaluated, it could become a useful tool for the clinical management of multi-chronic patients and thus can be likewise transferred to other Italian regions.