# Nonparametric Estimation from Incomplete Observations

• E. L. Kaplan
• Paul Meier
Chapter
Part of the Springer Series in Statistics book series (SSS)

## Abstract

In lifetesting, medical follow-up, and other fields the observation of the time of occurrence of the event of interest (called a death) may be prevented for some of the items of the sample by the previous occurrence of some other event (called a loss). Losses may be either accidental or controlled, the latter resulting from a decision to terminate certain observations. In either case it is usually assumed in this paper that the lifetime (age at death) is independent of the potential loss time; in practice this assumption deserves careful scrutiny. Despite the resulting incompleteness of the data, it is desired to estimate the proportion P(t) of items in the population whose lifetimes would exceed t (in the absence of such losses), without making any assumption about the form of the function P(t). The observation for each item of a suitable initial event, marking the beginning of its lifetime, is presupposed.

For random samples of size N the product-limit (PL) estimate can be defined as follows: List and label the N observed lifetimes (whether to death or loss) in order of increasing magnitude, so that one has $$0 \leqslant t_1^\prime \leqslant t_2^\prime \leqslant \cdots \leqslant t_N^\prime .$$ Then $$\hat P\left( t \right) = \Pi r\left[ {\left( {N - r} \right)/\left( {N - r + 1} \right)} \right]$$, where r assumes those values for which $$t_r^\prime \leqslant t$$ and for which $$t_r^\prime$$ measures the time to death. This estimate is the distribution, unrestricted as to form, which maximizes the likelihood of the observations.

Other estimates that are discussed are the actuarial estimates (which are also products, but with the number of factors usually reduced by grouping); and reduced-sample (RS) estimates, which require that losses not be accidental, so that the limits of observation (potential loss times) are known even for those items whose deaths are observed. When no losses occur at ages less than t the estimate of P(t) in all cases reduces to the usual binomial estimate, namely, the observed proportion of survivors.

## Keywords

American Statistical Association Nonparametric Estimation Effective Sample Size Indeterminate Result Observation Limit
These keywords were added by machine and not by the authors. This process is experimental and the keywords may be updated as the learning algorithm improves.

## References

1. [1]
Bailey. W.G.. and Haycocks, H.W, “A synthesis of methods of deriving measures of decrement from observed data.” Journal of the Institute of Actuaries, 73 (1947). 179–212.Google Scholar
2. [2]
Bartlelt, M.S.. “On the statistical estimation of mean lifetime,” Philosophical Magazine (Series 7 ), 44 (1953), 249–62.Google Scholar
3. [3]
Berkson, J., and Gage, R.P. “Calculation of survival rates for cancer,” Proceedings of the Staff Meetings of the Mayo Clinic. 25 (1950), 270–86.Google Scholar
4. [4]
Berkson, J, and Gage, R.P., “Survival curve for cancer patients following treatment.” Journal of the American Statistical Association. 47 (1952), 501–15.
5. [5]
Berkson. J., “Estimation of the interval rate in actuarial calculations: a critique of the person-years concept.” (Summary) Journal of the American Statistical Association 49 (1954). 363.Google Scholar
6. [6]
Böhmer, P.E., “Theorie der unabhängigen Wahrscheinlichkeiten,” Rapports. Mémoires et Prooès-verbaux de Septième Congrès International d’Actuaires, Amsterdam. 2 (1912), 327–43.Google Scholar
7. [7]
Brown, G.W. and Flood, M.M.. “Tumbler mortality,” Journal of the American Statistical Association. 42 (1947). 562–74.
8. [8]
Cornfield, J., “Cancer illness among residents in Atlanta, Georgia,” Public Health Service Publication No. 13, Cancer Morbidity Series No. 1. 1950.Google Scholar
9. [9]
Davis, D.J., “An analysis of some failure data,” Journal of the American Statistical Association 47 (1952), 113–150.
10. [10]
Epstein, Benjamin, and Sobel, Milton, “Lifetesting,” Journal of the American Statistical Association. 48 (1953). 486–502.
11. [11]
Epstein, Benjamin, and Sobel, Milton, “Some theorems relevant to lifetesting from an exponential distribution,” Annals of Mathematical Statistics. 25 (1954), 373–81.
12. [12]
Fix, Evelyn, “Practical implications of certain stochastic models on different methods of follow-up studies,” paper presented at 1951 annual meeting of the Western Branch, American Public Health Association, November 1, 1951.Google Scholar
13. [13]
Fix. Evelyn, and Neyman, J., “A simple stochastic model of recovery, relapse, death and loss of patients.” Human Biology, 23 (1951), 205–41.Google Scholar
14. [14]
Goodman, L.A., “Methods of measuring useful life of equipment under operational conditions.” Journal of the American Statistical Association. 48 (1953) 503–30.
15. [15]
Greenwood, Major. “The natural duration of cancer.” Reports on Public Health and Medical Subjects, No. 33 (1926), His Majesty’s Stationery Office.Google Scholar
16. [16]
Greville, T.N.E., “Mortality tables analyzed by cause of death.” The Record, American Institute of Actuaries, 37 (1948). 283–94.Google Scholar
17. [17]
Hald, A., “Maximum likelihood estimation of the parameters of a normal distri¬bution which is truncated at a known point.” Skandinavisk Aktuarietidskrift, 32 (1949) 119–34.
18. [18]
Harris, T.E., Meier, P.. and Tukey, J.W., “Timing of the distribution of events between observations,” Human Biology, 22 (1950), 249–70.Google Scholar
19. [19]
Irwin. J.O., “The standard error of an estimate of expcctational life.” Journal of Hygiene, 47 (1949), 188–9.
20. [20]
Jablon. Seymour. “Testing of survival rates as computed from life tables,” unpublished memorandum. June 14. 1951.Google Scholar
21. [21]
Kahn. H.A.. and Mantel. Nathan, “Variance of estimated proportions withdrawing in single decrement follow-up studies when cases are lost from observation,” unpublished manuscript.Google Scholar
22. [22]
Littell, A.S., “Estimation of the T-year survival rate from follow-up studies over a limited period of time,” Human Biology, 24 (1952), 87–116.Google Scholar
23. [23]
Merrell, Margaret, “Time-specific life tables contrasted with observed survivorship,” Biometrics. 3 (1947), 129 - 36.
24. [24]
Sartwell, P.E., and Merrell. M. “Influence of the dynamic character of chronic disease on the interpretation of morbidity rates,” American Journal of Public Health, 42 (1952), 579–84.
25. [25]
Savage. I.R., “Bibliography of nonparametric statistics and related topics.” Journal of the American Statistical Association, 48 (1953), 844–906.
26. [26]
Seal, H.L., “The estimation of mortality and other decremental probabilities,” Skamlinavisk Aktuarletidskrift, 37 (1954), 137–62.
27. [27]
Stephan. F.F., “The expected value and variance of the reciprocal and other negative powers of a positive Bernoullian variate,” Annals of Mathematical Statistics, 16 (1945), 50–61.
28. [28]
Wolfowitz. J., “Additive partition functions and a class of statistical hypotheses.” Annals of Mathematical Statistics, 13 (1942), 247–79.

© Springer-Verlag New York, Inc. 1992

## Authors and Affiliations

• E. L. Kaplan
• 1
• Paul Meier
• 2
1. 1.University of California Radiation LaboratoryUSA
2. 2.University of ChicagoUSA