A predictive modeling approach to increasing the economic effectiveness of disease management programs

Abstract

Predictive Modeling (PM) techniques are gaining importance in the worldwide health insurance business. Modern PM methods are used for customer relationship management, risk evaluation or medical management. This article illustrates a PM approach that enables the economic potential of (cost-)effective disease management programs (DMPs) to be fully exploited by optimized candidate selection as an example of successful data-driven business management. The approach is based on a Generalized Linear Model (GLM) that is easy to apply for health insurance companies. By means of a small portfolio from an emerging country, we show that our GLM approach is stable compared to more sophisticated regression techniques in spite of the difficult data environment. Additionally, we demonstrate for this example of a setting that our model can compete with the expensive solutions offered by professional PM vendors and outperforms non-predictive standard approaches for DMP selection commonly used in the market.

Appendix:Definition and interpretation of different predictive measures

In this appendix, we introduce two kinds of predictive measures that permit a comparison of cost prediction techniques with regard to an efficient DMP selection:

• a) Measures for the accuracy of a forecast that indirectly assess how many of the patients with the highest saving potential can be identified

• b) Measures for the sorting capacity of a forecast that directly examine the same question

The latter group of measures might be more suitable for analyzing the direct cost benefit in the actual data situation described. For comparing the general ability of an approach to optimize the selection of DMP participants by forecasting claimed amounts, the measures in group a) are equally important.

• a) Measures for the accuracy of a forecast

Two predictive measures quantifying the prediction error are the mean predictive squared error (MPSE) and the mean predictive absolute error (MPAE) defined as

$$\begin{array}{rll} \text{MPSE} &:=& \frac{1}{n}\sum\limits_{i=1}^{n} \left(y_{i}^{*}-y_{o,i}\right)^{2} \quad \text{and} \\ \text{MPAE} &:=& \frac{1}{n}\sum\limits_{i=1}^{n} \left|y_{i}^{*}-y_{o,i}\right|. \end{array} $$

Both measures analyze the differences between predicted $(\boldsymbol{y}^* = (y_{1}^{*},\ldots,y_{n}^{*})')$ and actually observed ($\boldsymbol {y}_{o} = (y_{o,1},$ $\ldots ,y_{o,n})'$) costs. The MPSE is based on the quadratic loss function. This means that predictions that avoid extreme discrepancies between $\boldsymbol {y}^{*}$ and $\boldsymbol {y}_{o}$ are rated best. By contrast, the MPAE that is based on the absolute loss function favors predictions that are good “on average”. The MPSE assures precise predictions for high-cost members. This is very relevant for DMP selection, because not recognizing a member who will-without preventive interaction-produce exploding medical costs in the near future means that there is no possibility of realizing the individual saving potential related to a DMP.

A disadvantage of both MPSE and MPAE is that these measures are not normed like, for example, the coefficient of determination or model R-squared that measures goodness-of-fit in linear models and ranges between 0 and 1. Hence, it is desirable to define a normed measure that is bound to a limited interval of possible values and measures the absolute predictive quality of a model. For this purpose, we define the so-called predictive R-squared $R^{2^{*}}$ according to the formulation of the model R-squared (that assesses the squared linear correlation between $\boldsymbol {\hat {y}}$ and $\boldsymbol {y}$) in order to measure the squared linear correlation between $\boldsymbol {y}^{*}$ and $\boldsymbol {y}_{o}$:

$$R^{2^{*}} := \frac{\sum_{i=1}^{n} \left(y_{i}^{*}-\bar{y}^{*}\right)\left(y_{o,i}-\bar{y}_{o}\right)}{\sqrt{\sum_{i=1}^{n} \left(y_{i}^{*}-\bar{y}^{*}\right)^{2}} \sqrt{\sum_{i=1}^{n} \left(y_{o,i}-\bar{y}_{o}\right)^{2}}}. $$

$\bar {y}^{*}$ and $\bar {y}_{o}$ denote the arithmetic means of $\boldsymbol {y}^{*}$ and $\boldsymbol {y}_{o}$, respectively. Unlike the model R-squared, the predictive R-squared does not measure goodness-of-fit and cannot be interpreted as percentage of explained variance or deviance in the classical sense, since the decomposition of variance or deviance [10, 48] that holds for estimated values $\boldsymbol {\hat {y}}$ does not hold for predicted values $\hat {\boldsymbol {y}}^{*}$. However, $R^{2^{*}}$ is also bound to the interval $[0;1]$ with values closer to one indicating a higher absolute predictive quality.

• b) Measures for the sorting capacity of a forecast

Two measures directly characterizing the sorting capacity of a forecast are the Spearman rank correlation coefficient $R_{\text {Sp}}$ and the area under the “matching curve” $AUC_{m}$.

The Spearman or rank correlation coefficient $R_{\text {Sp}}$ measures the monotone correlation between $\boldsymbol {y}^{*}$ and $\boldsymbol {y}_{o}$:

$$R_{\text{Sp}} = \frac{\sum_{i=1}^{n} \left(\text{rank}\left(y_{i}^{*}\right)-\overline{\text{rank}}(\boldsymbol{y}^{*})\right)(\text{rank}(y_{o,i})-\overline{\text{rank}}(\boldsymbol{y}_o))}{\sqrt{\sum_{i=1}^{n} \left(\text{rank}\left(y_{i}^{*}\right)-\overline{\text{rank}}(\boldsymbol{y}^{*})\right)^{2}} \sqrt{\sum_{i=1}^{n} \left(\text{rank}(y_{o,i})-\overline{\text{rank}}(\boldsymbol{y}_o)\right)^{2}}}. $$

where $\overline {\text {rank}}(\boldsymbol {y})$ denotes the average rank of all elements of the vector $\boldsymbol {y}$. $R_{\text {Sp}}$ ranges between $-$1 and $+1$ where values close to $+1$ indicate a high positive monotone correlation meaning that predicted and observed claimed amounts are similarly ordered. For DMP selection, this is a desirable property.

The idea of the matching curve m is derived from the concept of the ROC (receiver operating characteristic) curve that is used to assess the predictive quality of binary regression models [42]. $m(i)$ is defined as the percentage of those i members with the highest observed values who can also be found among the i members with the highest predicted values where i runs from 1 to n:

$$m(i) := \frac{1}{i} \sum\limits_{j=1}^i I\left(c_{o,(j)} \in c_{(1)}^*,\ldots,c_{(i)}^*\right), \quad i=1,\ldots,n.$$

In this definition, $c_{o,(1)},\ldots ,c_{o,(n)}$ represents the vector of member codes sorted in descending order by the corresponding observed claims totals $\boldsymbol {y}_{o}$. In parallel, $c_{(1)}^{*},\ldots ,c_{(n)}^{*}$ denotes the vector of member codes sorted in descending order by the corresponding predicted claims totals $\boldsymbol {y}^{*}$. $I(\cdot )$ is an indicator function that is equal to 1 if the condition in brackets is fulfilled and 0 if it is not. Thus, $m(i)$ indicates the percentage of matching member codes (matches) among the first i elements of the vectors $c_{o,(1)},\ldots ,c_{o,(n)}$ and $c_{(1)}^{*},\ldots ,c_{(n)}^{*}$. We obtain the area under the matching curve $AUC_{m}$ by calculating $\frac {1}{n} \sum _{i=1}^{n} m(i)$. The maximum $AUC_{m}$ is 1, which occurs if the members have the same order in respect of observed and predicted values.

Figure 8 shows an example of a matching curve $m(i)$ for our GLM (evaluated on the grid i = 50,100,. . . , 9,150) and the expected matching curve of a randomly ordered sample. In the context of DMP selection, it is particularly interesting to compare different values of the matching curve $m(i)$ in the high-cost region in order to measure what percentage of i members who actually have the highest claimed amounts can be identified by the prediction approach. This is why $m(i)$ is also called the identification or hit ratio. Based on the identification ratio and some experience-driven assumptions on the average saving potential per cost group, a health insurer can easily compare the potential overall savings between different methods.

Fig. 8

Matching curve m of our GLM (solid line) with area under curve $AUC_{m}$ (gray) and expected matching curve of a randomly ordered sample (dashed line)