The Intrinsic Estimator, Alternative Estimates, and Predictions of Mortality Trends: A Comment on Masters, Hummer, Powers, Beck, Lin, and Finch
- 1.4k Downloads
In this article, we discuss a study by Masters et al. (2014), published in Demography. Masters and associates estimated age, period, and cohort (APC) effects on U.S. mortality rates between 1959 and 2009 using the intrinsic estimator (IE). We first argue that before applying the IE, a grounded theoretical justification is needed for its fundamental constraint on minimum variance of the estimates. We next demonstrate IE’s high sensitivity to the type of dummy parameterization used to obtain the estimates. Finally, we discuss challenges in the interpretation of APC models. Our comments are not restricted to the article in question but pertain generally to any research that uses the IE.
In their article “Long-Term Trends in Adult Mortality for U.S. Blacks and Whites: An Examination of Period- and Cohort-Based Changes,” Masters et al. (2014) used the intrinsic estimator (IE) to estimate age, period, and cohort (APC) effects on mortality rates for whites and blacks in the United States. While concerns have been raised about the use of the IE technique (Fienberg 2013; Luo 2013a; O’Brien 2011), Masters et al. (2014:2066) argued that the IE is the “preferred solution” to find the true APC effects on mortality. However, as Luo (2013a) demonstrated, without external information, we do not know whether the IE recovers the parameters that generated the data. In addition, acknowledging that the IE is sensitive to the choice of the reference—or, more accurately, omitted—categories for each age, period, and cohort effect, Masters et al. found their results highly consistent using different omitted categories (Masters et al. 2014: Online Resource 1). However, these sensitivity analyses tend to be consistent as long the same parameterization framework is used (Luo et al. forthcoming; Pelzer et al. 2015).
Our comments on the Masters et al. (2014) article are threefold. First, the IE estimates in their article do not necessarily recover the parameters that generated these mortality data; that is, the estimates could very well be biased. Second, Masters and associates provided limited sensitivity analysis. We will show that the IE estimates can change dramatically with different choices of omitted categories of age, period, and cohort. Third, because of the perfect dependency between age, period, and cohort, we raise concerns about the meaning and interpretation of the estimated mortality rates.
Theoretical Basis for the Intrinsic Estimator’s Fundamental Assumption
The literature warns researchers that the identification problem (i.e., age = period – birth cohort) is inherent in APC models and that no technical solution overcomes this problem (Bell and Jones 2015; Fienberg 2013; Fienberg et al. 2015; Held and Riebler 2013; Luo 2013a, b; Luo et al. forthcoming; Winship and Harding 2008). Therefore, researchers may need to choose an estimator based on theoretical grounds. Without such theoretical justification, it remains questionable whether the estimates obtained are unbiased and consistent of the true age, period, and cohort parameters that generated the data.
The first (row) vector in Eq. (1) is called the null vector (b0), and its numbers are solely a function of the number of age, period, and cohort categories in a given data set. The second (column) vector (bIE) holds the IE estimates. The IE constraint is that the multiplication of both vectors equals zero.
Thus, the IE estimates obtained by Masters and colleagues are valid if and only if the underlying data-generating mechanism holds parameters for age, period, and cohort that also have the lowest variance possible. If, however, the parameters do not follow this constraint (see the dashed line in Fig. 1), then the IE estimates (the solid line in Fig. 1) are biased. Masters and associates provide little external information or theoretical consideration to verify the validity of the IE constraint of minimum variance in Eq. (1) and shown in Fig. 1.
Masters and colleagues contend that the IE can be applied in their research because the mortality data they use contain nonlinear effects, whereas “Luo set the functional forms of all APC effects on the outcome Y to be exactly linear” (2014: Online Resource 1). However, Luo (2013a) demonstrated that the IE estimates are also invalid when the IE’s constraint is not satisfied in simulated data with both linear and nonlinear effects (see, e.g., Luo 2013a: figure 1, 1963–1964). The IE produces unbiased and consistent estimates only when the minimum variance constraint also applies to the data generating parameters. This is equally true when the age, period, and cohort effects all are perfectly linear (see, e.g., Data Set 1 in Luo 2013a: table 3, 1954).
Masters and colleagues performed some interesting simulation analyses and argued that the IE can recover the “true” age, period, and cohort effects specified in the simulation function (Masters et al. 2014: figure S3, Online Source 1). However, the IE performed well in their simulations simply because all of their simulation functions satisfy the IE’s constraint of minimum variance. In other words, it is their method of simulating the data, not the statistical properties of the IE, that guarantees the IE’s performance.
Specifically, Masters and associates specified the data-generating mechanism “us[ing] the estimated coefficients from IE models,” and then “refit the IE estimator models to the simulated data to determine how well the IE can retrieve the original effects in the presence of the simulated error” (Masters et al. 2014: Online Resource 1). However, the numeric values of the IE estimates are used in the function that generates the simulated data. As a result, the simulated vectors with IE estimates all have close to minimum variance and therefore, as we have already discussed, the IE produces a set of estimates on the grounds that Eq. (1) is correct—that is, the IE’s constraint is satisfied by the data-generating mechanism. Using the numeric values of these IE estimates in the simulation function means that for the resulting simulated data, the IE constraint in Eq. (1) is at least approximately satisfied. Because the IE is an unbiased estimator when its constraint conforms to the truth (Luo 2013a:1958), the IE will appear to yield reliable estimates as it did in Masters et al. (2014: figure S3, Online Resource 1).
This simulation procedure is therefore not very useful to demonstrate the unbiasedness of the IE estimates because any APC estimator can recover the age, period, and cohort parameters when the data are generated with the constraint applied by that estimator, as O’Brien (2011:436) noted. For example, one may model the same mortality data using the constrained generalized linear model (CGLM; Mason et al. 1973) while setting the parameters of the first two age groups as equal. The set of age, period, and cohort effects obtained this way are then used in the data-generating function while adding some random error. Finally, one reestimates the model using the CGLM constraint on the first two age groups to find that the CGLM recovered the parameters that were used to simulate the data set. Of course, the CGLM estimates are only reliable in this example because the data were generated to satisfy its constraint in the first place (i.e., to have equal effects for the first two age groups). In fact, an infinite number of possible solutions exist that can produce the same mortality data with some random error, and the IE and the CGLM are just two of them. If the data-generating mechanism satisfies the IE constraint, as in the simulation in Master et al.’s article, then the IE can yield valid estimates; otherwise, the IE can be wildly biased. We elaborate on this crucial argument against the use of the IE in the next section.
A Matter of Choice: IE’s Dependence on Parameterization
Masters et al. obtained their IE estimates using a dummy parameterization (see their table S4 in Online Resource 1) known as effect coding (Hardy 1993). This coding scheme results in estimates for all age, period, and cohort dummy variables that are deviations from the scaled grand mean. The choice to use effect coding should be arbitrary because under normal conditions, the way of coding the dummy variables is of no real importance. For instance, in dummy coding, estimates are scaled as deviations from the reference category, and they differ from effect-coded estimates only by a scaling factor. However, because the effect coding is part of the constraint (of minimum variance) in the IE to solve the APC identification problem, the IE estimates obtained from effect coding and the IE estimates from dummy coding can differ substantially from each other (Luo et al. forthcoming; Pelzer et al. 2015).
Pelzer et al. (2015) demonstrated that each parameterization framework relates to a particular IE constraint, which causes the estimates to be fundamentally different (see Fig. 2). In the default IE that Masters and associates used, the effect coding framework is part of the constraint: the estimates obtained this way have the lowest variation possible. That is, the variance of the set of IE estimates is kept at its minimum (see Fig. 1). In this sense, the IE is a member of a larger family of constrained models that have been developed in the past, and the main criticism of those models applies here as well: verifying that the true parameters for age, period, and cohort in the data generating process also have minimum variation requires external information to determine which particular framework is the one that generated the data—a task that may be impossible. It is therefore questionable whether the default IE estimates presented by Masters et al. (2014) are unbiased: do they reflect the unknown parameters or not?
Pitfalls in Interpreting APC Estimates in Regression Models
A less-discussed aspect of APC models is the potential difficulty in interpreting the resulting estimates. In regression models, the effect of a predictor variable is typically interpreted as the pure or independent effect of that predictor when keeping the other predictors constant. However, this type of interpretation makes less sense in APC models, in which one of the predictors (e.g., birth cohort) does not vary when the other two (age and period) are kept constant. Consequently, it is unfeasible to demonstrate change in the dependent variable by manipulating the value of only one predictor.
In Fig. 5, the sample mean of the estimated mortality rates is equal to the overall mortality rate for blacks and whites, respectively. This way, the comparison of the trends between blacks and whites no longer depends on which categories are chosen. Compared with the results in Masters et al. (2014: figure 3), our results tell a different story about differences between blacks and whites, which underpins our concerns about presenting estimated mortality rates with fixed data points.
- Held, L., & Riebler, A. (2013). Comment on “Assessing validity and application scope of the intrinsic estimator approach to the age-period-cohort (APC) problem.” Demography, 50, 1977–1979.Google Scholar
- Luo, L., Hodges, J. S., Winship, C., & Powers, D. (Forthcoming). The sensitivity of the intrinsic estimator to coding schemes: A comment on Yang, Schulhofer-Wohl, Fu, and Land. American Journal of Sociology.Google Scholar
Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.