Predictive Validity Under Partial Observability

  • Eduardo Alarcón-BustamanteEmail author
  • Ernesto San Martín
  • Jorge González
Conference paper
Part of the Springer Proceedings in Mathematics & Statistics book series (PROMS, volume 322)


To assess the predictive capacity of selection tests is a challenge because the response variable is observed only in selected individuals. In this paper we propose to evaluate the predictive capacity of selection tests through marginal effects under a partial identification approach. Identification bounds are defined for the marginal effects under monotonicity assumptions of the response variable. The performance of our method is assessed using a real data set from the university selection test applied in Chile and compared with the marginal effect of the traditional model used in Chile to evaluate the predictive capacity of the selection test.


Selection problem Marginal effects Partial identification Identification bounds 



Eduardo Alarcón-Bustamante was funded by CONICYT Doctorado Nacional grant 2018-21181007. Ernesto San Martín was partially funded by the FONDECYT grant 1181261.


  1. American Educational Research Association, American Psychological Association, National Council on Measurement in Education, & Joint Committee on Standards for Educational and Psychological Testing (U.S). (2014). Standards for educational and psychological testing. Washington, DC: American Educational Research Association.Google Scholar
  2. DEMRE. (2016, 9). Prueba de selección universitaria (Tech. Rep.). Universidad de Chile.Google Scholar
  3. Donoso, S. (1998). La reforma educacional y el sistema de selección de alumnos a las universidades: impactos y cambios demandados. Estudios Pedagógicos(24), 7–30.Google Scholar
  4. Florens, J., & Mouchart, M. (1982). A note on noncausality. Econometrica, 50(3), 583–592.MathSciNetzbMATHCrossRefGoogle Scholar
  5. Geiser, S., & Studley, R. (2002). UC and the SAT: Predictive validity and differential impact of the SAT I and SAT II at the University of California. Educational Assessment, 8(1), 1–26.CrossRefGoogle Scholar
  6. Goldhaber, D., Cowan, J., & Theobald, R. (2017). Evaluating prospective teachers: Testing the predictive validity of the edTPA. Journal of Teacher Education, 68(4), 377–393.CrossRefGoogle Scholar
  7. Grassau, E. (1956). Análisis estadístico de las pruebas de bachillerato. Anales de la Universidad de Chile(102), 77–93.Google Scholar
  8. Guilliksen, H. (1950). Theory of mental tests. New York: Willey.CrossRefGoogle Scholar
  9. Heckman, J. (1976). The common structure of statistical models of truncation, sample selection and limited dependent variables and a simple estimator for such models. The Annals of Economic and Social Measurement, 46, 931–961.Google Scholar
  10. Heckman, J. (1979). Sample selection bias as a specification error. Econometrica, 47(1), 153–161.MathSciNetzbMATHCrossRefGoogle Scholar
  11. Lawley, D. (1943). IV.-A note on Karl Pearson’s selection formulae. Proceedings of the Royal Society of Edinburgh. Section A. Mathematical and Physical Science, 62(1), 28–30.MathSciNetzbMATHCrossRefGoogle Scholar
  12. Leong, C.-H. (2007). Predictive validity of the multicultural personality questionnaire: A longitudinal study on the socio-psychological adaptation of Asian undergraduates who took part in a study-abroad program. International Journal of Intercultural Relations, 31, 545–559.CrossRefGoogle Scholar
  13. Lord, F. (1980). Applications of item response theory to practical testing problems. New York: Routledge.Google Scholar
  14. Manski, C. (1989). Anatomy of the selection problem. The Journal of Human Resources, 24(3), 343–360.CrossRefGoogle Scholar
  15. Manski, C. (1993). Identification problems in the social sciences. Sociological Methodology, 23, 1–56.CrossRefGoogle Scholar
  16. Manski, C. (2003). Partial identification of probability distributions. New York: Springer.zbMATHGoogle Scholar
  17. Manski, C. (2005). Social choice with partial knowledge of treatment response (1st ed.). New Jersy: Princeton University Press.zbMATHGoogle Scholar
  18. Manski, C. (2007). Identification for prediction and decision. Cambridge: Harvard University Press.Google Scholar
  19. Manski, C. (2013). Public policy in an uncertain world: Analysis and decisions. Cambridge: Harvard University Press.CrossRefGoogle Scholar
  20. Manzi, J., Bravo, D., Pino, G. del, Donoso, G., Martínez, M., & Pizarro, R. (2008, 7). Estudio de la validez predictiva de los factores de selección a las universidades del consejo de rectores, admisiones 2003 al 2006 (Tech. Rep.). Comité Técnico Asesor, Honorable Consejo de Rectores de las Universidades Chilenas.Google Scholar
  21. Marchenko, Y. V., & Genton, M. G. (2012). A Heckman Selection-t model. Journal of the American Statistical Association, 107(497), 304–317.MathSciNetzbMATHCrossRefGoogle Scholar
  22. Mendoza, J., & Mumford, M. (1987). Corrections for attenuation and range restriction on the predictor. Journal of Educational Statistics, 12(3), 282–293.CrossRefGoogle Scholar
  23. Nawata, K. (1994). Estimation of sample selection bias models by the maximum likelihood estimator and Heckman’s two-step estimator. Economics Letters, 45(1), 33–40.MathSciNetzbMATHCrossRefGoogle Scholar
  24. Pearson, K. (1903). Mathematical contribution to the theory of evolution-XI on the influence of natural selection on the variability and correlation of organs. Philosophical Transactions of the Royal Society of London, 200(Ser. A), 1–66.Google Scholar
  25. Thorndike, R. (1949). Personnel selection: Test and measurement techniques. New York: Wiley.Google Scholar
  26. Wang, X.-F. (2010). fANCOVA: Nonparametric Analysis of Covariance [Computer software manual]. Retrieved from (R package version 0.5-1).

Copyright information

© Springer Nature Switzerland AG 2020

Authors and Affiliations

  • Eduardo Alarcón-Bustamante
    • 1
    • 2
    Email author
  • Ernesto San Martín
    • 1
    • 2
    • 3
  • Jorge González
    • 4
  1. 1.Department of Statistics, Faculty of MathematicsPontificia Universidad Católica de ChileSantiagoChile
  2. 2.Interdisciplinary Laboratory of Social StatisticsSantiagoChile
  3. 3.The Economics School of LouvainUniversité Catholique de LouvainOttignies-Louvain-la-NeuveBelgium
  4. 4.Facultad de MatematicasPontificia Universidad Católica de ChileSantiagoChile

Personalised recommendations