Abstract
We propose a likelihood ratio statistic for forming hypothesis tests and confidence intervals for a nonparametrically estimated univariate regression function, based on the shape restriction of concavity (alternatively, convexity). Dealing with the likelihood ratio statistic requires studying an estimator satisfying a null hypothesis, that is, studying a concave least-squares estimator satisfying a further equality constraint. We study this null hypothesis least-squares estimator (NLSE) here, and use it to study our likelihood ratio statistic. The NLSE is the solution to a convex program, and we find a set of inequality and equality constraints that characterize the solution. We also study a corresponding limiting version of the convex program based on observing a Brownian motion with drift. The solution to the limit problem is a stochastic process. We study the optimality conditions for the solution to the limit problem and find that they match those we derived for the solution to the finite sample problem. This allows us to show the limit stochastic process yields the limit distribution of the (finite sample) NLSE. We conjecture that the likelihood ratio statistic is asymptotically pivotal, meaning that it has a limit distribution with no nuisance parameters to be estimated, which makes it a very effective tool for this difficult inference problem. We provide a partial proof of this conjecture, and we also provide simulation evidence strongly supporting this conjecture.
Similar content being viewed by others
References
Aït-Sahalia, Y., Duarte, J.: Nonparametric option pricing under shape restrictions. J. Econom. 116(1–2), 9–47 (2003)
Allon, G., Beenstock, M., Hackman, S., Passy, U., Shapiro, A.: Nonparametric estimation of concave production technologies by entropic methods. J. Appl. Econom. 22(4), 795–816 (2007)
Balabdaoui, F., Rufibach, K., Wellner, J.A.: Limit distribution theory for maximum likelihood estimation of a log-concave density. Ann. Stat. 37(3), 1299–1331 (2009)
Banerjee, M.: Likelihood ratio tests for monotone functions. Ph.D. thesis, University of Washington (2000)
Banerjee, M.: Likelihood based inference for monotone response models. Ann. Stat. 35(3), 931–956 (2007). https://doi.org/10.1214/009053606000001578
Banerjee, M., Wellner, J.A.: Likelihood ratio tests for monotone functions. Ann. Stat. 29(6), 1699–1731 (2001)
Birke, M., Dette, H.: Estimating a convex function in nonparametric regression. Scand. J. Stat. 34(2), 384–404 (2007)
Bronšteĭn, E.M.: Extremal convex functions. Sibirsk. Mat. Ž. 19(1), 10–18, 236 (1978)
Brunk, H.D.: Estimation of isotonic regression. In: Nonparametric techniques in statistical inference (Proceedings of Symposium, Indiana University, Bloomington, IN, 1969), pp. 177–197. Cambridge University Press, London (1970)
Cai, T.T., Low, M.G., Xia, Y.: Adaptive confidence intervals for regression functions under shape constraints. Ann. Stat. 41(2), 722–750 (2013)
Dent, W.: A note on least squares fitting of functions constrained to be either nonnegative, nondecreasing or convex. Manag. Sci. 20, 130–132 (1973/74)
Doss, C.R., Wellner, J.A.: Global rates of convergence of the MLEs of log-concave and \(s\)-concave densities. Ann. Stat. 44(3), 954–981 (2016)
Doss, C.R., Wellner, J.A.: Inference for the mode of a log-concave density. Submitted to the Annals of Statistics (2018). arXiv:1611.10348
Doss, C.R., Wellner, J.A.: Log-concave density estimation with symmetry or modal constraints. Submitted to Annals of Statistics (2018). arXiv:1611.10335v2
Dümbgen, L.: Optimal confidence bands for shape-restricted curves. Bernoulli 9(3), 423–449 (2003)
Dümbgen, L., Rufibach, K.: Maximum likelihood estimation of a log-concave density and its distribution function: basic properties and uniform consistency. Bernoulli 15(1), 40–68 (2009)
Dykstra, R.L.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc. 78(384), 837–842 (1983)
Efron, B.: Bootstrap methods: another look at the jackknife. Ann. Stat. 7(1), 1–26 (1979)
Folland, G.B.: Real Analysis, Pure and Applied Mathematics, 2nd edn. Wiley, New York (1999)
Fraser, D.A.S., Massam, H.: A mixed primal-dual bases algorithm for regression under inequality constraints. application to concave regression. Scand. J. Stat. 16(1), 65–74 (1989)
Groeneboom, P.: Lectures on inverse problems. In: Bernard, P. (ed.) Lectures on Probability Theory, Ecole d’Eté de Probabilités de Saint-Flour XXIV-1994, pp. 67–164. Springer, Berlin (1996)
Groeneboom, P., Jongbloed, G.: Nonparametric confidence intervals for monotone functions. Ann. Stat. 43(5), 2019–2054 (2015)
Groeneboom, P., Jongbloed, G., Wellner, J.A.: A canonical process for estimation of convex functions: the “invelope” of integrated brownian motion \(+t^4\). Ann. Stat. 29(6), 1620–1652 (2001)
Groeneboom, P., Jongbloed, G., Wellner, J.A.: Estimation of a convex function: characterizations and asymptotic theory. Ann. Stat. 29(6), 1653–1698 (2001)
Hall, P.: Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density. Ann. Stat. 20(2), 675–694 (1992)
Hannah, L.A., Dunson, D.B.: Multivariate convex regression with adaptive partitioning. J. Mach. Learn. Res. 14, 3261–3294 (2013)
Hanson, D.L., Pledger, G.: Consistency in concave regression. Ann. Stat. 4(6), 1038–1050 (1976)
Hildreth, C.: Point estimates of ordinates of concave functions. J. Am. Stat. Assoc. 49(267), 598–619 (1954)
Hudson, D.J.: Least-squares fitting of a polynomial constrained to be either non-negative non-decreasing or convex. J. R. Stat. Soc. B 31(1), 113–118 (1969)
Johansen, S.: The extremal convex functions. Math. Scand. 34, 61–68 (1974)
Kuosmanen, T.: Representation theorem for convex nonparametric least squares. Econom. J. 11(2), 308–325 (2008)
Lim, E.: Response surface computation via simulation in the presence of convexity. In: Johansson, B., Jain, S., Montoya-Torres, J., Hugan, J., Yücesan, E. (eds.) 2010 Winter Simulation Conference, pp. 1246–1254 (2010)
Lim, E., Glynn, P.W.: Consistency of multidimensional convex regression. Oper. Res. 60(1), 196–208 (2012)
Mammen, E.: Nonparametric regression under qualitative smoothness assumptions. Ann. Stat. 19(2), 741–759 (1991)
Meyer, M.C.: Inference using shape-restricted regression splines. Ann. Appl. Stat. 2(3), 1013–1033 (2008)
Meyer, M.C.: Constrained penalized splines. Can. J. Stat. 40(1), 190–206 (2012)
Monti, M.M., Grant, S., Osherson, D.N.: A note on concave utility functions. Mind Soc. 4(1), 85–96 (2005)
Pal, J.K., Woodroofe, M., Meyer, M.: Estimating a Polya frequency function\({}_2\). In: Complex Datasets and Inverse Problems, IMS Lecture Notes Monograph Series, vol. 54, pp. 239–249. Institute of Mathematics Statistics, Beachwood (2007)
Pflug, G., Wets, R.J.B.: Shape-restricted nonparametric regression with overall noisy measurements. J. Nonparametr. Stat. 25(2), 323–338 (2013)
Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)
Seijo, E., Sen, B.: Nonparametric least squares estimation of a multivariate convex regression function. Ann. Stat. 39(3), 1633–1657 (2011)
Silverman, B.: On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Stat. 10(3), 795–810 (1982)
Sinai, Y.G.: Statistics of shocks in solutions of inviscid burgers equation. Commun. Math. Phys. 148(3), 601–621 (1992)
Topaloglu, H., Powell, W.B.: An algorithm for approximating piecewise linear concave functions from sample gradients. Oper. Res. Lett. 31(1), 66–76 (2003)
Toriello, A., Nemhauser, G., Savelsbergh, M.: Decomposing inventory routing problems with approximate value functions. Naval Res. Logist. 57(8), 718–727 (2010)
Wang, J.C., Meyer, M.C.: Testing the monotonicity or convexity of a function using regression splines. Can. J. Stat. 39(1), 89–107 (2011)
Wasserman, L.: All of Nonparametric Statistics. Springer Texts in Statistics. Springer, New York (2006)
Wu, C.F.: Some algorithms for concave and isotonic regression. Stud. Manag. Sci. 19, 105–116
Author information
Authors and Affiliations
Corresponding author
Additional information
Supported in part by NSF Grant DMS-1712664.
A Appendix: Technical formulas and other results
A Appendix: Technical formulas and other results
Here is a statement of an integration by parts formulas for functions of bounded variation. See, e.g., page 102 of [19] for the definition of bounded variation.
Lemma 5
([19]) Assume that F and G are of bounded variation on a set [a, b] where \(-\infty< a< b < \infty \). If at least one of F and G is continuous, then
Theorem 6
([23, Theorem 2.1]) Let \(\sigma , a > 0\). Let \(X(t) = \sigma W(t) - 4a t^3\) where W(t) is standard two-sided Brownian motion starting from 0, and let Y be the integral of X satisfying \(Y(0)=0\). Thus \(Y_{a,\sigma }(t) = \sigma \int _0^t W(s) ds - a t^4\) for \(t \in \mathbb {R}\). Then, with probability 1, there exists a uniquely defined random continuous function \(H_{a,\sigma }\) satisfying the following:
-
1.
The function \(H_{a,\sigma }\) satisfies \(H_{a,\sigma }(t) \le Y(t)\) for all \( t\in \mathbb {R}\).
-
2.
The function \(H_{a,\sigma }\) has a concave second derivative, \(\widehat{r}_{a,\sigma } := H_{a,\sigma }''\).
-
3.
The function \(H_{a,\sigma }\) satisfies \(\int _{\mathbb {R}} ( H_{a,\sigma }(t)-Y_{a,\sigma }(t) ) dH_{a,\sigma }^{(3)}(t)=0\).
Theorem 7
([24, Theorem 6.3]) Suppose that the regression model (9) holds, that \(\epsilon _{n,1},\ldots ,\epsilon _{n,n}\) are i.i.d. with \(E^{\epsilon _{n,1}^2 t} < \infty \) for some \(t > 0\), that \(r_0 \in \mathcal {C}\), that \(r_0''(x_0)<0\), and that \(r_0''\) is continuous in a neighborhood of \(x_0\). Let Assumptions 1 and 2 hold. Let \(a := |r_0''(x_0)|/ 24\) and \(\sigma ^2 := {{\mathrm{Var}}}( \epsilon _{n,i})\). Then
in \(L^p[-K,K]\) for all \(K > 0\).
Rights and permissions
About this article
Cite this article
Doss, C.R. Concave regression: value-constrained estimation and likelihood ratio-based inference. Math. Program. 174, 5–39 (2019). https://doi.org/10.1007/s10107-018-1338-5
Received:
Accepted:
Published:
Issue Date:
DOI: https://doi.org/10.1007/s10107-018-1338-5