Breakthroughs in Statistics pp 319-337 | Cite as

# Nonparametric Estimation from Incomplete Observations

## Abstract

In lifetesting, medical follow-up, and other fields the observation of the time of occurrence of the event of interest (called a *death*) may be prevented for some of the items of the sample by the previous occurrence of some other event (called a *loss*). Losses may be either accidental or controlled, the latter resulting from a decision to terminate certain observations. In either case it is usually assumed in this paper that the lifetime (age at death) is independent of the potential loss time; in practice this assumption deserves careful scrutiny. Despite the resulting incompleteness of the data, it is desired to estimate the proportion *P*(*t*) of items in the population whose lifetimes would exceed *t* (in the absence of such losses), without making any assumption about the form of the function *P*(*t*). The observation for each item of a suitable initial event, marking the beginning of its lifetime, is presupposed.

For random samples of size *N* the product-limit (PL) estimate can be defined as follows: List and label the *N* observed lifetimes (whether to death or loss) in order of increasing magnitude, so that one has \(0 \leqslant t_1^\prime \leqslant t_2^\prime \leqslant \cdots \leqslant t_N^\prime .\) Then \(\hat P\left( t \right) = \Pi r\left[ {\left( {N - r} \right)/\left( {N - r + 1} \right)} \right]\), where *r* assumes those values for which \(t_r^\prime \leqslant t\) and for which \(t_r^\prime\) measures the time to death. This estimate is the distribution, unrestricted as to form, which maximizes the likelihood of the observations.

Other estimates that are discussed are the actuarial estimates (which are also products, but with the number of factors usually reduced by grouping); and reduced-sample (RS) estimates, which require that losses not be accidental, so that the limits of observation (potential loss times) are known even for those items whose deaths are observed. When no losses occur at ages less than *t* the estimate of *P*(*t*) in all cases reduces to the usual binomial estimate, namely, the observed proportion of survivors.

## Keywords

American Statistical Association Nonparametric Estimation Effective Sample Size Indeterminate Result Observation Limit## Preview

Unable to display preview. Download preview PDF.

## References

- [1]Bailey. W.G.. and Haycocks, H.W, “A synthesis of methods of deriving measures of decrement from observed data.”
*Journal of the Institute of Actuaries*, 73 (1947). 179–212.Google Scholar - [2]Bartlelt, M.S.. “On the statistical estimation of mean lifetime,”
*Philosophical Magazine*(Series 7 ), 44 (1953), 249–62.Google Scholar - [3]Berkson, J., and Gage, R.P. “Calculation of survival rates for cancer,”
*Proceedings of the Staff Meetings of the Mayo Clinic*. 25 (1950), 270–86.Google Scholar - [4]Berkson, J, and Gage, R.P., “Survival curve for cancer patients following treatment.”
*Journal of the American Statistical Association*. 47 (1952), 501–15.CrossRefGoogle Scholar - [5]Berkson. J., “Estimation of the interval rate in actuarial calculations: a critique of the person-years concept.” (Summary)
*Journal of the American Statistical Association*49 (1954). 363.Google Scholar - [6]Böhmer, P.E., “Theorie der unabhängigen Wahrscheinlichkeiten,” Rapports. Mémoires et Prooès-verbaux de Septième Congrès International d’Actuaires, Amsterdam. 2 (1912), 327–43.Google Scholar
- [7]Brown, G.W. and Flood, M.M.. “Tumbler mortality,”
*Journal of the American Statistical Association*. 42 (1947). 562–74.zbMATHCrossRefGoogle Scholar - [8]Cornfield, J., “Cancer illness among residents in Atlanta, Georgia,” Public Health Service Publication No. 13, Cancer Morbidity Series No. 1. 1950.Google Scholar
- [9]Davis, D.J., “An analysis of some failure data,”
*Journal of the American Statistical Association*47 (1952), 113–150.CrossRefGoogle Scholar - [10]Epstein, Benjamin, and Sobel, Milton, “Lifetesting,”
*Journal of the American Statistical Association*. 48 (1953). 486–502.MathSciNetzbMATHCrossRefGoogle Scholar - [11]Epstein, Benjamin, and Sobel, Milton, “Some theorems relevant to lifetesting from an exponential distribution,”
*Annals of Mathematical Statistics*. 25 (1954), 373–81.MathSciNetzbMATHCrossRefGoogle Scholar - [12]Fix, Evelyn, “Practical implications of certain stochastic models on different methods of follow-up studies,” paper presented at 1951 annual meeting of the Western Branch, American Public Health Association, November 1, 1951.Google Scholar
- [13]Fix. Evelyn, and Neyman, J., “A simple stochastic model of recovery, relapse, death and loss of patients.”
*Human Biology*, 23 (1951), 205–41.Google Scholar - [14]Goodman, L.A., “Methods of measuring useful life of equipment under operational conditions.”
*Journal of the American Statistical Association*. 48 (1953) 503–30.zbMATHCrossRefGoogle Scholar - [15]Greenwood, Major. “The natural duration of cancer.” Reports on Public Health and Medical Subjects, No. 33 (1926), His Majesty’s Stationery Office.Google Scholar
- [16]Greville, T.N.E., “Mortality tables analyzed by cause of death.”
*The Record*, American Institute of Actuaries, 37 (1948). 283–94.Google Scholar - [17]Hald, A., “Maximum likelihood estimation of the parameters of a normal distri¬bution which is truncated at a known point.”
*Skandinavisk Aktuarietidskrift*, 32 (1949) 119–34.MathSciNetGoogle Scholar - [18]Harris, T.E., Meier, P.. and Tukey, J.W., “Timing of the distribution of events between observations,”
*Human Biology*, 22 (1950), 249–70.Google Scholar - [19]Irwin. J.O., “The standard error of an estimate of expcctational life.”
*Journal of Hygiene*, 47 (1949), 188–9.CrossRefGoogle Scholar - [20]Jablon. Seymour. “Testing of survival rates as computed from life tables,” unpublished memorandum. June 14. 1951.Google Scholar
- [21]Kahn. H.A.. and Mantel. Nathan, “Variance of estimated proportions withdrawing in single decrement follow-up studies when cases are lost from observation,” unpublished manuscript.Google Scholar
- [22]Littell, A.S., “Estimation of the
*T*-year survival rate from follow-up studies over a limited period of time,”*Human Biology*, 24 (1952), 87–116.Google Scholar - [23]Merrell, Margaret, “Time-specific life tables contrasted with observed survivorship,”
*Biometrics*. 3 (1947), 129 - 36.CrossRefGoogle Scholar - [24]Sartwell, P.E., and Merrell. M. “Influence of the dynamic character of chronic disease on the interpretation of morbidity rates,”
*American Journal of Public Health*, 42 (1952), 579–84.CrossRefGoogle Scholar - [25]Savage. I.R., “Bibliography of nonparametric statistics and related topics.”
*Journal of the American Statistical Association*, 48 (1953), 844–906.MathSciNetzbMATHCrossRefGoogle Scholar - [26]Seal, H.L., “The estimation of mortality and other decremental probabilities,”
*Skamlinavisk Aktuarletidskrift*, 37 (1954), 137–62.MathSciNetGoogle Scholar - [27]Stephan. F.F., “The expected value and variance of the reciprocal and other negative powers of a positive Bernoullian variate,”
*Annals of Mathematical Statistics*, 16 (1945), 50–61.MathSciNetCrossRefGoogle Scholar - [28]Wolfowitz. J., “Additive partition functions and a class of statistical hypotheses.”
*Annals of Mathematical Statistics*, 13 (1942), 247–79.MathSciNetzbMATHCrossRefGoogle Scholar