Importance of Adjusting for Multi-stage Design When Analyzing Data from Complex Surveys

  • Trung Ha
  • Julia N. Soulakova
Part of the ICSA Book Series in Statistics book series (ICSABSS)


Social scientists and policy makers commonly use estimates derived from population-based studies, e.g., estimates derived from the Tobacco Use Supplement (TUS) are commonly used in behavioral studies targeted on smoking and quitting behaviors. The U.S. Census Bureau and other agencies designing and administering national surveys provide technical guidelines on suitable statistical methodologies. The guidelines specify the appropriate methods for estimation and prediction. However, when performing secondary data analyses scientists may be prone to simplify analytical strategies and use classical statistical methods, i.e., ignore design specifics and mistreat the complex design used to gather the data as simple random sampling. In this chapter, we illustrate the importance of using the guidelines when analyzing complex surveys. We discuss three methods: method I ignores any weighting, method II incorporates the main weight only, and method III utilizes the main weight and balanced repeated replications with specified replicate weights. We illustrate possible discrepancies in point estimates and standard errors using 2014–2015 TUS data. Presented examples include smoking status, attitudes toward smoking restrictions in public places and cars, and smoking rules at home among single parents in the USA.


Balanced Repeated Replication Complex sampling Complex survey Current Population Survey Fay’s factor Hadamard matrix Multistage sampling National Health Interview Survey Primary sampling unit Replicate weights Single-parent household Smoke-free home Smoke-free workplace Stratum Successive Difference Replication Taylor Linearization Tobacco Use Supplement Ultimate Sampling Unit Unequal probability sampling Variance estimation 



We are thankful to James Holland, College of Medicine, University of Central Florida, for helping us improve the chapter.

Funding: Research reported in this publication was supported by the National Institute on Minority Health and Health Disparities of the National Institutes of Health under Award Number R01MD009718. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.


  1. Allison, D. B., Fontaine, K. R., Heo, M., Mentore, J. L., Cappelleri, J. C., Chandler, L. P., et al. (1999). The distribution of body mass index among individuals with and without schizophrenia. The Journal of Clinical Psychiatry, 60(4), 215–220.CrossRefGoogle Scholar
  2. Ash, S. (2014). Using successive difference replication for estimating variances. How to obtain more information. Survey Methodology, 40(1), 47–59.Google Scholar
  3. Blackwell, D., Lucas, J., & Clarke, T. (2014). Summary health statistics for US adults: National health interview survey, 2012. Vital and Health Statistics, 10(260), 1–161.Google Scholar
  4. Burch, R. C., Loder, S., Loder, E., & Smitherman, T. A. (2015). The prevalence and burden of migraine and severe headache in the United States: Updated statistics from government health surveillance studies. Headache, 55(1), 21–34.CrossRefGoogle Scholar
  5. Centers for Disease Control and Prevention. (2016). Variance estimation guidance, NHIS 2016 (Adapted from NHIS Survey Description Documents). Retrieved from
  6. Cummings, K. M., & Shan, D. (1995). Trends in smoking initiation among adolecents and young adults—United States, 1980-89. Morbidity and Mortality Weekly Report, 44(28), 521–525.Google Scholar
  7. Dahlhamer, J., Galinsky, A., Joestl, S., & Ward, B. (2014). Sexual orientation in the 2013 national health interview survey: A quality assessment. Vital and Health Statistics, 2(169), 1–32.Google Scholar
  8. Fay, R. E., & Train, G. (1995). Aspects of survey and model-based postcensal estimation of income and poverty characteristics for states and counties. In Proceedings of the Section on Government Statistics, American Statistical Association, Alexandria, VA (pp. 154–159).Google Scholar
  9. Fiore, M. C., Novotny, T. E., Pierce, J. P., Hatziandreu, E. J., Patel, K. M., & Davis, R. M. (1989). Trends in Cigarette Smoking in the United States. The changing influence of gender and race. JAMA, 261(1), 49–55.CrossRefGoogle Scholar
  10. Gilpin, E. A., & Pierce, J. P. (1997). Trends in adolescent smoking initiation in the United States: Is tobacco marketing an influence? Tobacco Control, 6(2), 122–127.CrossRefGoogle Scholar
  11. Hansen, M., Hurwitz, W., & Madow, W. (1953). Sample survey methods and theory, Vol. 1, Methods and applications. New York: Wiley.zbMATHGoogle Scholar
  12. Jemal, A., Thun, M., Yu, X. Q., Hartman, A. M., Cokkinides, V., Center, M. M., et al. (2011). Changes in smoking prevalence among U.S. adults by state and region: Estimates from the Tobacco Use Supplement to the Current Population Survey, 1992-2007. BMC Public Health, 11, 512.CrossRefGoogle Scholar
  13. Judkins, D. (1990). Fay’s method for variance estimation. Journal of Official Statistics, 6(3), 223–239.Google Scholar
  14. Lohr, S. L. (1999). Sampling: Design and analysis (2nd ed.). Duxbury Press.Google Scholar
  15. Lohr, S. L. (2012). Using SAS® for the design, analysis, and visualization of complex surveys. Retrieved from
  16. Nassiuma, D. K. (2001). Survey sampling: Theory and methods. Nairobi University Press.Google Scholar
  17. Parsons, V., Moriarity, C., Jonas, K., Moore, T., Davis, K., & Tompkins, L. (2014). Design and estimation for the national health interview survey, 2006-2015. Vital and Health Statistics, 2(165), 1–53.Google Scholar
  18. SAS Institute Inc. (2016). SAS/STAT ® 14.2 user’s guide. Cary: SAS Institute Inc.Google Scholar
  19. Scheaffer, R. L., Mendenhall III, W., Ott, R. L., & Gerow, K. G. (2011). Elementary Survey Sampling. Zhurnal Eksperimental’noi i Teoreticheskoi Fiziki (7th ed.). Brooks/Cole.Google Scholar
  20. Shopland, D. R., Gerlach, K. K., Burns, D. M., Hartman, A. M., & Gibson, J. T. (2001). State-specific trends in smoke-free workplace policy coverage: The current population survey tobacco use supplement, 1993 to 1999. Journal of Occupational and Environmental Medicine, 43(8), 680–686.CrossRefGoogle Scholar
  21. Soulakova, J. N., & Crockett, L. J. (2014). Consistency and recanting of ever-smoking status reported by self and proxy respondents one year apart. Journal of Addictive Behaviors, Therapy & Rehabilitation, 3(4), 1000113. Scholar
  22. Soulakova, J. N., Davis, W. W., Hartman, A., & Gibson, J. (2009). The impact of survey and response modes on current smoking prevalence estimates using TUS-CPS: 1992-2003. Survey Research Methods, 3(3), 123–137. Retrieved from Scholar
  23. Soulakova, J. N., Hartman, A. M., Liu, B., Willis, G. B., & Augustine, S. (2012). Reliability of adult self-reported smoking history: Data from the tobacco use supplement to the current population survey 2002-2003 cohort. Nicotine and Tobacco Research, 14(8), 952–960. Scholar
  24. Soulakova, J. N., Bright, B. C., & Crockett, L. J. (2015a). Perception of time since smoking cessation: Time in memory can elapse faster. Journal of Addictive Behaviors, Therapy & Rehabilitation, 4(4).
  25. Soulakova, J. N., Huang, H., & Crockett, L. J. (2015b). Racial/ethnic disparities in consistent reporting of smoking-related behaviors. Journal of Addictive Behaviors, Therapy & Rehabilitation, 4(4).
  26. Stang, P., & Osterhaus, J. (1993). Impact of migraine in the United States: Data from the International Health Interview Survey. Headache, 33(1), 29–35.CrossRefGoogle Scholar
  27. U.S. Bureau of Labor Statistics & U.S. Census Bureau. (2006). Design and methodology: Current population survey, Technical Paper 66. Retrieved from
  28. U.S. Bureau of Labor Statistics & U.S. Census Bureau. Current population survey FTP Page. Retrieved from
  29. U.S. Census Bureau. (2015). America’s families and living arrangements: 2015. Retrieved from
  30. U.S. Department of Commerce, U.S. Census Bureau. (2015). Current population survey, May 2015: Tobacco use supplement. Retrieved from
  31. U.S. Department of Commerce, U.S. Census Bureau. (2017). Estimating current population survey: Household-level supplement variances using replicate weights. Retrieved from
  32. U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. (2010). The principal source of information on the health of the U.S. population. Retrieved from
  33. Ward, B., Dahlhamer, J., Galinsky, A., & Joestl, S. (2014). Sexual orientation and health among US adults: National Health Interview Survey, 2013. National Health Statistics Reports, (77), 1–10.Google Scholar
  34. Wolter, K. M. (1984). An investigation of some estimators of variance for systematic sampling. Journal of the American Statistical Association, 79(388), 781–790.MathSciNetCrossRefGoogle Scholar
  35. Wolter, K. M. (2007). Introduction to variance estimation (2nd ed.). Springer.Google Scholar

Copyright information

© Springer Nature Switzerland AG 2018

Authors and Affiliations

  1. 1.Burnett School of Biomedical SciencesCollege of Medicine, University of Central FloridaOrlandoUSA

Personalised recommendations