Importance of Adjusting for Multi-stage Design When Analyzing Data from Complex Surveys
Social scientists and policy makers commonly use estimates derived from population-based studies, e.g., estimates derived from the Tobacco Use Supplement (TUS) are commonly used in behavioral studies targeted on smoking and quitting behaviors. The U.S. Census Bureau and other agencies designing and administering national surveys provide technical guidelines on suitable statistical methodologies. The guidelines specify the appropriate methods for estimation and prediction. However, when performing secondary data analyses scientists may be prone to simplify analytical strategies and use classical statistical methods, i.e., ignore design specifics and mistreat the complex design used to gather the data as simple random sampling. In this chapter, we illustrate the importance of using the guidelines when analyzing complex surveys. We discuss three methods: method I ignores any weighting, method II incorporates the main weight only, and method III utilizes the main weight and balanced repeated replications with specified replicate weights. We illustrate possible discrepancies in point estimates and standard errors using 2014–2015 TUS data. Presented examples include smoking status, attitudes toward smoking restrictions in public places and cars, and smoking rules at home among single parents in the USA.
KeywordsBalanced Repeated Replication Complex sampling Complex survey Current Population Survey Fay’s factor Hadamard matrix Multistage sampling National Health Interview Survey Primary sampling unit Replicate weights Single-parent household Smoke-free home Smoke-free workplace Stratum Successive Difference Replication Taylor Linearization Tobacco Use Supplement Ultimate Sampling Unit Unequal probability sampling Variance estimation
We are thankful to James Holland, College of Medicine, University of Central Florida, for helping us improve the chapter.
Funding: Research reported in this publication was supported by the National Institute on Minority Health and Health Disparities of the National Institutes of Health under Award Number R01MD009718. The content is solely the responsibility of the authors and does not necessarily represent the official views of the National Institutes of Health.
- Ash, S. (2014). Using successive difference replication for estimating variances. How to obtain more information. Survey Methodology, 40(1), 47–59.Google Scholar
- Blackwell, D., Lucas, J., & Clarke, T. (2014). Summary health statistics for US adults: National health interview survey, 2012. Vital and Health Statistics, 10(260), 1–161.Google Scholar
- Centers for Disease Control and Prevention. (2016). Variance estimation guidance, NHIS 2016 (Adapted from NHIS Survey Description Documents). Retrieved from https://www.cdc.gov/nchs/data/nhis/2006var.pdf
- Cummings, K. M., & Shan, D. (1995). Trends in smoking initiation among adolecents and young adults—United States, 1980-89. Morbidity and Mortality Weekly Report, 44(28), 521–525.Google Scholar
- Dahlhamer, J., Galinsky, A., Joestl, S., & Ward, B. (2014). Sexual orientation in the 2013 national health interview survey: A quality assessment. Vital and Health Statistics, 2(169), 1–32.Google Scholar
- Fay, R. E., & Train, G. (1995). Aspects of survey and model-based postcensal estimation of income and poverty characteristics for states and counties. In Proceedings of the Section on Government Statistics, American Statistical Association, Alexandria, VA (pp. 154–159).Google Scholar
- Judkins, D. (1990). Fay’s method for variance estimation. Journal of Official Statistics, 6(3), 223–239.Google Scholar
- Lohr, S. L. (1999). Sampling: Design and analysis (2nd ed.). Duxbury Press.Google Scholar
- Lohr, S. L. (2012). Using SAS® for the design, analysis, and visualization of complex surveys. Retrieved from http://support.sas.com/resources/papers/proceedings12/343-2012.pdf
- Nassiuma, D. K. (2001). Survey sampling: Theory and methods. Nairobi University Press.Google Scholar
- Parsons, V., Moriarity, C., Jonas, K., Moore, T., Davis, K., & Tompkins, L. (2014). Design and estimation for the national health interview survey, 2006-2015. Vital and Health Statistics, 2(165), 1–53.Google Scholar
- SAS Institute Inc. (2016). SAS/STAT ® 14.2 user’s guide. Cary: SAS Institute Inc.Google Scholar
- Scheaffer, R. L., Mendenhall III, W., Ott, R. L., & Gerow, K. G. (2011). Elementary Survey Sampling. Zhurnal Eksperimental’noi i Teoreticheskoi Fiziki (7th ed.). Brooks/Cole.Google Scholar
- Shopland, D. R., Gerlach, K. K., Burns, D. M., Hartman, A. M., & Gibson, J. T. (2001). State-specific trends in smoke-free workplace policy coverage: The current population survey tobacco use supplement, 1993 to 1999. Journal of Occupational and Environmental Medicine, 43(8), 680–686.CrossRefGoogle Scholar
- Soulakova, J. N., Hartman, A. M., Liu, B., Willis, G. B., & Augustine, S. (2012). Reliability of adult self-reported smoking history: Data from the tobacco use supplement to the current population survey 2002-2003 cohort. Nicotine and Tobacco Research, 14(8), 952–960. https://doi.org/10.1093/ntr/ntr313.CrossRefGoogle Scholar
- Soulakova, J. N., Bright, B. C., & Crockett, L. J. (2015a). Perception of time since smoking cessation: Time in memory can elapse faster. Journal of Addictive Behaviors, Therapy & Rehabilitation, 4(4). https://doi.org/10.4172/2324-9005.1000145.
- Soulakova, J. N., Huang, H., & Crockett, L. J. (2015b). Racial/ethnic disparities in consistent reporting of smoking-related behaviors. Journal of Addictive Behaviors, Therapy & Rehabilitation, 4(4). https://doi.org/10.4172/2324-9005.1000147.
- U.S. Bureau of Labor Statistics & U.S. Census Bureau. (2006). Design and methodology: Current population survey, Technical Paper 66. Retrieved from https://www.census.gov/programs-surveys/cps/technical-documentation/complete.html
- U.S. Bureau of Labor Statistics & U.S. Census Bureau. Current population survey FTP Page. Retrieved from http://thedataweb.rm.census.gov/ftp/cps_ftp.html#cpssupps
- U.S. Census Bureau. (2015). America’s families and living arrangements: 2015. Retrieved from https://www.census.gov/data/tables/2015/demo/families/cps-2015.html
- U.S. Department of Commerce, U.S. Census Bureau. (2015). Current population survey, May 2015: Tobacco use supplement. Retrieved from https://www.census.gov/programs-surveys/cps/technical-documentation/complete.2015.html
- U.S. Department of Commerce, U.S. Census Bureau. (2017). Estimating current population survey: Household-level supplement variances using replicate weights. Retrieved from http://thedataweb.rm.census.gov/pub/cps/supps/HH-level_Use_of_the_Public_Use_Replicate_Weight_File.doc
- U.S. Department of Health and Human Services, Centers for Disease Control and Prevention, National Center for Health Statistics. (2010). The principal source of information on the health of the U.S. population. Retrieved from https://www.cdc.gov/nchs/data/nhis/brochure2010January.pdf
- Ward, B., Dahlhamer, J., Galinsky, A., & Joestl, S. (2014). Sexual orientation and health among US adults: National Health Interview Survey, 2013. National Health Statistics Reports, (77), 1–10.Google Scholar
- Wolter, K. M. (2007). Introduction to variance estimation (2nd ed.). Springer.Google Scholar