Abstract
On building a model for estimating response propensity score for survey data adjustment, we carried out computer simulations to examine the effects of seven types of variables, each having differing associations with the sample inclusion probability, response probability and study variable, by comparing the respective cases in which the variables are included and excluded by the model. Then the following main results were obtained. The most important variable for the model is the one that is simultaneously associated with the study variable, the sample inclusion probability, and the response probability. The variables which have no association with the study variable should not be included in the response propensity model. These results support the conclusions of Brookhart et al. (2006), who examined propensity score models in their study on estimating treatment effects. Additionally, a small difference was found in comparing the effects of the variable associated with sample inclusion probability and the study variable to those of the variable associated with the response probability and the study vari able.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
References
Austin, P.C., Grootendorst, P. and Anderson, G.M. (2007). A comparison of the ability of different propensity score models to balance measured variables between treated and untreated subjects: a Monte Carlo study, Statistics in Medicine, 26, 734–753.
Bethlehem, J.G. (2002). Weighting nonresponse adjustments based on auxiliary variables, In Groves R.M., Dillman D.A., Eltinge J.L., Little R.J.A., (Eds.), Survey Nonresponse (pp. 275–287). New York: Wiley.
Brookhart, M.A., Schneeweiss, S., Rothman, K.J., Glynn, R.J., Avorn, J. and Stürmer, T. (2006). Variable selection for propensity score models, American Journal of Epidemiology, 163-12, 1149–1156.
Brunell, T.L. and DiNardo, J. (2004). A propensity score reweighting approach to estimating the partisan effects of full turnout in American presidential elections, Political Analysis, 12, 28–45.
Couper, M.P. (2000). Web surveys: a review of issues and approaches, Public Opinion Quarterly, 64, 464–494.
Dillman, D.A., Eltinge, J.L., Groves, R.M., and Little, R.J.A. (2002). Survey nonresponse in design, data collection, and analysis, In Groves R.M., Dillman D.A., Eltinge J.L., Little R.J.A., (Eds.), Survey Nonresponse (pp. 3–26). New York: Wiley.
Duncan, K.B. and Stasny, E.A. (2001). Using propensity scores to control coverage bias in telephone surveys, Survey Methodology, 27-2, 121–130.
Fukuda, M (2007). Predicting election results using propensity scores estimated based on preelection survey data and exit poll data, Journal of the Japan Statistical Society, 36-2, 117–130. (in Japanese)
Fukuda, M (2008). Current methods of exit poll and related issues, The Japanese Journal of Behaviormetrics, 35-1, 59–71. (in Japanese)
Groves, R.M., Cialdini, R.B. and Couper, M.P. (1992). Understanding the decision to participate in a survey, Public Opinion Quarterly, 56, 475–495.
Hirano, K. and Imbens, G.W. (2001). Estimation of causal effects using propensity score weighting: an application to data on right heart catheterization, Health Services & Outcomes Research Methodology, 2, 259–278.
Hoshino, T. (2003). Application of propensity score to web survey to approximate random sampling, Journal of the Japanese Society for Quality Control, 33-3, 44–51. (in Japanese)
Hoshino, T. and Shigemasu, K (2004). Estimation of causal effect and adjustment of survey data using propensity scores, The Japanese Journal of Behaviormetrics, 31-1, 43–61. (in Japanese)
Hoshino, T. and Maeda, T (2006). Applying propensity-score adjustment to social surveys with non-random sampling and a selection criterion for covariates, Proceedings of the Institute of Statistical Mathematics, 54-1, 191–206. (in Japanese)
Hoshino, T. (2007). An experimental research of reproducibility of bias reduction using Propensity score adjustment for internet marketing surveys, The Japanese Journal of Behaviormetrics, 34-1, 33–48. (in Japanese)
Isaksson, A. and Forsman, G. (2003). A comparison between using the web and using telephone to survey political options, The American Association for Public Opinion Research (AAPOR) 58th Annual Conference.
Lee, S. (2006). Propensity score adjustment as a weighting scheme for volunteer panel web surveys, Journal of Official Statististics, 22-2, 329–349.
Oh, H.L. and Scheuren, F.J. (1983). Weighting adjustment for unit nonresponse, In Madow, W.G., Olkin, I and Rubin, D.B. (Eds.), Incomplete data in sample survey vol.2 (pp. 143–184). New York: TAcademic press.
Perkins, S.M., Tu, W., Underhill, M.G., Zhou, X. and Murray, M.D. (2000). The use of propensity scores in pharmacoepidemiologic research. Pharmacoepidemiology and Drug Safety, 9, 93–101.
Rosenbaum, P.R. and Rubin, D.B. (1983). The central role of the propensity score in observational studies for causal effects, Biometrika, 70, 41–55.
Rosenbaum, P.R. and Rubin, D.B. (1984). Reducing bias in observational studies using subclassification on the propensity score, Journal of the American Statistical Association, 79, 516–524.
Rosenbaum, P.R. (1987). Model-based direct adjustment, Journal of the American Statistical Association, 82, 387–394.
Rosenbaum, P.R. (2002). Observational Studies (2nd ed.), New York: Springer.
Rubin, D.B. and Thomas, N. (1996). Matching using estimated propensity scores: relating theory to practice, Biometrics, 52, 249–264.
Särndal, C.E., Swensson, B. and Wretman, J. (1992). Model Assisted Survey Sampling, New York: Springer.
Schonlau, M., Zapert, K., Simon, L.P., Sanstad, K., Marcus, S., Adams, J. Kan, H., Turber, R. and Berry, S. (2004), A comparison between responses from a propensity-weighted web survey and an identical RDD survey, Social Science Computer Review, 22, 128–138.
Schonlau, M., van Soest, A. and Kapteyn, A. (2007). Are “Webographic” or attitudinal questions useful for adjusting estimates from Web surveys using propensity scoring? Survey Research Methods, 1-3, 155–163.
Smith, P.J., Rao, J.N.K., Battaglia, M.P., Daniels, D. and Ezzati-Rice, T. (2000). Compensating for nonresponse bias in the national immunization survey using response propensities, In Proceedings of the Section on Survey Research Methods, American Statistical Association, 641–646.
Taylor, H., Bremer, J., Overmeyer, C., Siegel, J.W., and Terhanian, G. (2001). The record of internet-based opinion polls in predicting the results of 72 races in the November 2000 US elections, International Journal of Market Research, 43-2, 127–135.
Tsuchiya, T. (2005). A study of the characteristics of nonrespondents -a postsurvey of the reasons for cooperation to the eleventh nationwide survey on the Japanese national character-, Proceedings of the Institute of Statistical Mathematics, 53-1, 35–56. (in Japanese)
Author information
Authors and Affiliations
About this article
Cite this article
Fukuda, M. Effects of Variables in a Response Propensity Score Model for Survey Data Adjustment: A Simulation Study. Behaviormetrika 38, 33–61 (2011). https://doi.org/10.2333/bhmk.38.33
Received:
Revised:
Published:
Issue Date:
DOI: https://doi.org/10.2333/bhmk.38.33