Mathematical Programming

, Volume 174, Issue 1–2, pp 5–39 | Cite as

Concave regression: value-constrained estimation and likelihood ratio-based inference

  • Charles R. DossEmail author
Full Length Paper Series B


We propose a likelihood ratio statistic for forming hypothesis tests and confidence intervals for a nonparametrically estimated univariate regression function, based on the shape restriction of concavity (alternatively, convexity). Dealing with the likelihood ratio statistic requires studying an estimator satisfying a null hypothesis, that is, studying a concave least-squares estimator satisfying a further equality constraint. We study this null hypothesis least-squares estimator (NLSE) here, and use it to study our likelihood ratio statistic. The NLSE is the solution to a convex program, and we find a set of inequality and equality constraints that characterize the solution. We also study a corresponding limiting version of the convex program based on observing a Brownian motion with drift. The solution to the limit problem is a stochastic process. We study the optimality conditions for the solution to the limit problem and find that they match those we derived for the solution to the finite sample problem. This allows us to show the limit stochastic process yields the limit distribution of the (finite sample) NLSE. We conjecture that the likelihood ratio statistic is asymptotically pivotal, meaning that it has a limit distribution with no nuisance parameters to be estimated, which makes it a very effective tool for this difficult inference problem. We provide a partial proof of this conjecture, and we also provide simulation evidence strongly supporting this conjecture.

Mathematics Subject Classification

Primary 62G08 Secondary 62G15 62G10 62G20 


  1. 1.
    Aït-Sahalia, Y., Duarte, J.: Nonparametric option pricing under shape restrictions. J. Econom. 116(1–2), 9–47 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  2. 2.
    Allon, G., Beenstock, M., Hackman, S., Passy, U., Shapiro, A.: Nonparametric estimation of concave production technologies by entropic methods. J. Appl. Econom. 22(4), 795–816 (2007)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Balabdaoui, F., Rufibach, K., Wellner, J.A.: Limit distribution theory for maximum likelihood estimation of a log-concave density. Ann. Stat. 37(3), 1299–1331 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Banerjee, M.: Likelihood ratio tests for monotone functions. Ph.D. thesis, University of Washington (2000)Google Scholar
  5. 5.
    Banerjee, M.: Likelihood based inference for monotone response models. Ann. Stat. 35(3), 931–956 (2007). MathSciNetCrossRefzbMATHGoogle Scholar
  6. 6.
    Banerjee, M., Wellner, J.A.: Likelihood ratio tests for monotone functions. Ann. Stat. 29(6), 1699–1731 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  7. 7.
    Birke, M., Dette, H.: Estimating a convex function in nonparametric regression. Scand. J. Stat. 34(2), 384–404 (2007)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Bronšteĭn, E.M.: Extremal convex functions. Sibirsk. Mat. Ž. 19(1), 10–18, 236 (1978)MathSciNetGoogle Scholar
  9. 9.
    Brunk, H.D.: Estimation of isotonic regression. In: Nonparametric techniques in statistical inference (Proceedings of Symposium, Indiana University, Bloomington, IN, 1969), pp. 177–197. Cambridge University Press, London (1970)Google Scholar
  10. 10.
    Cai, T.T., Low, M.G., Xia, Y.: Adaptive confidence intervals for regression functions under shape constraints. Ann. Stat. 41(2), 722–750 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  11. 11.
    Dent, W.: A note on least squares fitting of functions constrained to be either nonnegative, nondecreasing or convex. Manag. Sci. 20, 130–132 (1973/74)Google Scholar
  12. 12.
    Doss, C.R., Wellner, J.A.: Global rates of convergence of the MLEs of log-concave and \(s\)-concave densities. Ann. Stat. 44(3), 954–981 (2016)MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Doss, C.R., Wellner, J.A.: Inference for the mode of a log-concave density. Submitted to the Annals of Statistics (2018). arXiv:1611.10348
  14. 14.
    Doss, C.R., Wellner, J.A.: Log-concave density estimation with symmetry or modal constraints. Submitted to Annals of Statistics (2018). arXiv:1611.10335v2
  15. 15.
    Dümbgen, L.: Optimal confidence bands for shape-restricted curves. Bernoulli 9(3), 423–449 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  16. 16.
    Dümbgen, L., Rufibach, K.: Maximum likelihood estimation of a log-concave density and its distribution function: basic properties and uniform consistency. Bernoulli 15(1), 40–68 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  17. 17.
    Dykstra, R.L.: An algorithm for restricted least squares regression. J. Am. Stat. Assoc. 78(384), 837–842 (1983)MathSciNetCrossRefzbMATHGoogle Scholar
  18. 18.
    Efron, B.: Bootstrap methods: another look at the jackknife. Ann. Stat. 7(1), 1–26 (1979)MathSciNetCrossRefzbMATHGoogle Scholar
  19. 19.
    Folland, G.B.: Real Analysis, Pure and Applied Mathematics, 2nd edn. Wiley, New York (1999)Google Scholar
  20. 20.
    Fraser, D.A.S., Massam, H.: A mixed primal-dual bases algorithm for regression under inequality constraints. application to concave regression. Scand. J. Stat. 16(1), 65–74 (1989)Google Scholar
  21. 21.
    Groeneboom, P.: Lectures on inverse problems. In: Bernard, P. (ed.) Lectures on Probability Theory, Ecole d’Eté de Probabilités de Saint-Flour XXIV-1994, pp. 67–164. Springer, Berlin (1996)Google Scholar
  22. 22.
    Groeneboom, P., Jongbloed, G.: Nonparametric confidence intervals for monotone functions. Ann. Stat. 43(5), 2019–2054 (2015)MathSciNetCrossRefzbMATHGoogle Scholar
  23. 23.
    Groeneboom, P., Jongbloed, G., Wellner, J.A.: A canonical process for estimation of convex functions: the “invelope” of integrated brownian motion \(+t^4\). Ann. Stat. 29(6), 1620–1652 (2001)CrossRefzbMATHGoogle Scholar
  24. 24.
    Groeneboom, P., Jongbloed, G., Wellner, J.A.: Estimation of a convex function: characterizations and asymptotic theory. Ann. Stat. 29(6), 1653–1698 (2001)MathSciNetCrossRefzbMATHGoogle Scholar
  25. 25.
    Hall, P.: Effect of bias estimation on coverage accuracy of bootstrap confidence intervals for a probability density. Ann. Stat. 20(2), 675–694 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  26. 26.
    Hannah, L.A., Dunson, D.B.: Multivariate convex regression with adaptive partitioning. J. Mach. Learn. Res. 14, 3261–3294 (2013)MathSciNetzbMATHGoogle Scholar
  27. 27.
    Hanson, D.L., Pledger, G.: Consistency in concave regression. Ann. Stat. 4(6), 1038–1050 (1976)MathSciNetCrossRefzbMATHGoogle Scholar
  28. 28.
    Hildreth, C.: Point estimates of ordinates of concave functions. J. Am. Stat. Assoc. 49(267), 598–619 (1954)MathSciNetCrossRefzbMATHGoogle Scholar
  29. 29.
    Hudson, D.J.: Least-squares fitting of a polynomial constrained to be either non-negative non-decreasing or convex. J. R. Stat. Soc. B 31(1), 113–118 (1969)zbMATHGoogle Scholar
  30. 30.
    Johansen, S.: The extremal convex functions. Math. Scand. 34, 61–68 (1974)MathSciNetCrossRefzbMATHGoogle Scholar
  31. 31.
    Kuosmanen, T.: Representation theorem for convex nonparametric least squares. Econom. J. 11(2), 308–325 (2008)CrossRefzbMATHGoogle Scholar
  32. 32.
    Lim, E.: Response surface computation via simulation in the presence of convexity. In: Johansson, B., Jain, S., Montoya-Torres, J., Hugan, J., Yücesan, E. (eds.) 2010 Winter Simulation Conference, pp. 1246–1254 (2010)Google Scholar
  33. 33.
    Lim, E., Glynn, P.W.: Consistency of multidimensional convex regression. Oper. Res. 60(1), 196–208 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  34. 34.
    Mammen, E.: Nonparametric regression under qualitative smoothness assumptions. Ann. Stat. 19(2), 741–759 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  35. 35.
    Meyer, M.C.: Inference using shape-restricted regression splines. Ann. Appl. Stat. 2(3), 1013–1033 (2008)MathSciNetCrossRefzbMATHGoogle Scholar
  36. 36.
    Meyer, M.C.: Constrained penalized splines. Can. J. Stat. 40(1), 190–206 (2012)MathSciNetCrossRefzbMATHGoogle Scholar
  37. 37.
    Monti, M.M., Grant, S., Osherson, D.N.: A note on concave utility functions. Mind Soc. 4(1), 85–96 (2005)CrossRefGoogle Scholar
  38. 38.
    Pal, J.K., Woodroofe, M., Meyer, M.: Estimating a Polya frequency function\({}_2\). In: Complex Datasets and Inverse Problems, IMS Lecture Notes Monograph Series, vol. 54, pp. 239–249. Institute of Mathematics Statistics, Beachwood (2007)Google Scholar
  39. 39.
    Pflug, G., Wets, R.J.B.: Shape-restricted nonparametric regression with overall noisy measurements. J. Nonparametr. Stat. 25(2), 323–338 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  40. 40.
    Rockafellar, R.T.: Convex Analysis. Princeton University Press, Princeton (1970)CrossRefzbMATHGoogle Scholar
  41. 41.
    Seijo, E., Sen, B.: Nonparametric least squares estimation of a multivariate convex regression function. Ann. Stat. 39(3), 1633–1657 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  42. 42.
    Silverman, B.: On the estimation of a probability density function by the maximum penalized likelihood method. Ann. Stat. 10(3), 795–810 (1982)MathSciNetCrossRefzbMATHGoogle Scholar
  43. 43.
    Sinai, Y.G.: Statistics of shocks in solutions of inviscid burgers equation. Commun. Math. Phys. 148(3), 601–621 (1992)MathSciNetCrossRefzbMATHGoogle Scholar
  44. 44.
    Topaloglu, H., Powell, W.B.: An algorithm for approximating piecewise linear concave functions from sample gradients. Oper. Res. Lett. 31(1), 66–76 (2003)MathSciNetCrossRefzbMATHGoogle Scholar
  45. 45.
    Toriello, A., Nemhauser, G., Savelsbergh, M.: Decomposing inventory routing problems with approximate value functions. Naval Res. Logist. 57(8), 718–727 (2010)MathSciNetCrossRefzbMATHGoogle Scholar
  46. 46.
    Wang, J.C., Meyer, M.C.: Testing the monotonicity or convexity of a function using regression splines. Can. J. Stat. 39(1), 89–107 (2011)MathSciNetCrossRefzbMATHGoogle Scholar
  47. 47.
    Wasserman, L.: All of Nonparametric Statistics. Springer Texts in Statistics. Springer, New York (2006)zbMATHGoogle Scholar
  48. 48.
    Wu, C.F.: Some algorithms for concave and isotonic regression. Stud. Manag. Sci. 19, 105–116Google Scholar

Copyright information

© Springer-Verlag GmbH Germany, part of Springer Nature and Mathematical Optimization Society 2018

Authors and Affiliations

  1. 1.School of StatisticsUniversity of MinnesotaMinneapolisUSA

Personalised recommendations