Abundant Inverse Regression Using Sufficient Reduction and Its Applications

  • Hyunwoo J. Kim
  • Brandon M. Smith
  • Nagesh Adluru
  • Charles R. Dyer
  • Sterling C. Johnson
  • Vikas Singh
Conference paper
Part of the Lecture Notes in Computer Science book series (LNCS, volume 9907)


Statistical models such as linear regression drive numerous applications in computer vision and machine learning. The landscape of practical deployments of these formulations is dominated by forward regression models that estimate the parameters of a function mapping a set of p covariates, \(\varvec{x}\), to a response variable, y. The less known alternative, Inverse Regression, offers various benefits that are much less explored in vision problems. The goal of this paper is to show how Inverse Regression in the “abundant” feature setting (i.e., many subsets of features are associated with the target label or response, as is the case for images), together with a statistical construction called Sufficient Reduction, yields highly flexible models that are a natural fit for model estimation tasks in vision. Specifically, we obtain formulations that provide relevance of individual covariates used in prediction, at the level of specific examples/samples — in a sense, explaining why a particular prediction was made. With no compromise in performance relative to other methods, an ability to interpret why a learning algorithm is behaving in a specific way for each prediction, adds significant value in numerous applications. We illustrate these properties and the benefits of Abundant Inverse Regression on three distinct applications.


Inverse regression Kernel regression Abundant regression Temperature prediction Alzheimer’s disease Age estimation 



This research was supported by NIH grants AG040396, and NSF CAREER award 1252725. Partial support was provided by UW ADRC AG033514, UW ICTR 1UL1RR025011, UW CPCP AI117924 and Waisman Core Grant P30 HD003352-45.


  1. 1.
    Friston, K.J., Holmes, A.P., Worsley, K.J., Poline, J.P., Frith, C.D., Frackowiak, R.S.: Statistical parametric maps in functional imaging: a general linear approach. Hum. Brain Mapp. 2(4), 189–210 (1994)CrossRefGoogle Scholar
  2. 2.
    Cao, X., Wei, Y., Wen, F., Sun, J.: Face alignment by explicit shape regression. Int. J. Comput. Vis. 107(2), 177–190 (2014)MathSciNetCrossRefGoogle Scholar
  3. 3.
    Li, K.C.: Sliced inverse regression for dimension reduction. J. Am. Stat. Assoc. 86(414), 316–327 (1991)MathSciNetCrossRefzbMATHGoogle Scholar
  4. 4.
    Loh, P.L., Wainwright, M.J.: High-dimensional regression with noisy and missing data: provable guarantees with non-convexity. In: Proceedings of the Advances in Neural Information Processing Systems, pp. 2726–2734 (2011)Google Scholar
  5. 5.
    Chen, Y., Caramanis, C.: Noisy and missing data regression: distribution-oblivious support recovery. In: Proceedings of the 30th International Conference on Machine Learning, pp. 383–391 (2013)Google Scholar
  6. 6.
    Krutchkoff, R.: Classical and inverse regression methods of calibration. Technometrics 9(3), 425–439 (1967)MathSciNetCrossRefGoogle Scholar
  7. 7.
    Taddy, M.: Multinomial inverse regression for text analysis. J. Am. Stat. Assoc. 108(503), 755–770 (2013)MathSciNetCrossRefzbMATHGoogle Scholar
  8. 8.
    Rabinovich, M., Blei, D.: The inverse regression topic model. In: Proceedings of the 31st International Conference on Machine Learning, pp. 199–207 (2014)Google Scholar
  9. 9.
    Kim, M., Pavlovic, V.: Dimensionality reduction using covariance operator inverse regression. In: Proceedings of the Computer Vision and Pattern Recognition, pp. 1–8. IEEE (2008)Google Scholar
  10. 10.
    Glasner, D., Fua, P., Zickler, T., Zelnik-Manor, L.: Hot or not: exploring correlations between appearance and temperature. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 3997–4005 (2015)Google Scholar
  11. 11.
    Adragni, K.P., Cook, R.D.: Sufficient dimension reduction and prediction in regression. Philos. Trans. R. Soc. Lond. A: Math. Phys. Eng. Sci. 367(1906), 4385–4405 (2009)MathSciNetCrossRefzbMATHGoogle Scholar
  12. 12.
    Cook, R.D.: Fisher lecture: dimension reduction in regression. Statist. Sci. 22(1), 1–26 (2007). doi: 10.1214/088342306000000682 MathSciNetCrossRefzbMATHGoogle Scholar
  13. 13.
    Tuzel, O., Porikli, F., Meer, P.: Region covariance: a fast descriptor for detection and classification. In: Leonardis, A., Bischof, H., Pinz, A. (eds.) ECCV 2006. LNCS, vol. 3952, pp. 589–600. Springer, Heidelberg (2006)CrossRefGoogle Scholar
  14. 14.
    Zhu, Y., Li, Y., Mu, G., Guo, G.: A study on apparent age estimation. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 25–31 (2015)Google Scholar
  15. 15.
    Escalera, S., Fabian, J., Pardo, P., Baró, X., Gonzalez, J., Escalante, H.J., Misevic, D., Steiner, U., Guyon, I.: Chalearn looking at people 2015: apparent age and cultural event recognition datasets and results. In: Proceedings of the IEEE International Conference on Computer Vision Workshops, pp. 1–9 (2015)Google Scholar
  16. 16.
    Minear, M., Park, D.: A lifespan database of adult facial stimuli. Behav. Res. Methods 36(4), 630–633 (2004)CrossRefGoogle Scholar
  17. 17.
    Guo, G.D., Wang, X.: A study on human age estimation under facial expression changes. In: IEEE Conference on Computer Vision and Pattern Recognition (2012)Google Scholar
  18. 18.
    Kim, H.J., Xu, J., Vemuri, B.C., Singh, V.: Manifold-valued Dirichlet processes. In: Proceedings of the International Conference on Machine Learning, pp. 1199–1208 (2015)Google Scholar
  19. 19.
    Cook, P., Bai, Y., Nedjati-Gilani, S., Seunarine, K., Hall, M., Parker, G., Alexander, D.: Camino: open-source diffusion-MRI reconstruction and processing. In: Proceedings of the 14th Scientific Meeting of the International Society for Magnetic Resonance in Medicine, Seattle WA, USA (2006)Google Scholar
  20. 20.
    Hwang, S.J., Collins, M.D., Ravi, S.N., Ithapu, V.K., Adluru, N., Johnson, S.C., Singh, V.: A projection free method for generalized eigenvalue problem with a nonsmooth regularizer. In: Proceedings of the IEEE International Conference on Computer Vision, pp. 1841–1849 (2015)Google Scholar

Copyright information

© Springer International Publishing AG 2016

Authors and Affiliations

  • Hyunwoo J. Kim
    • 1
  • Brandon M. Smith
    • 1
  • Nagesh Adluru
    • 1
  • Charles R. Dyer
    • 1
  • Sterling C. Johnson
    • 1
  • Vikas Singh
    • 1
  1. 1.University of Wisconsin-MadisonMadisonUSA

Personalised recommendations