Skip to main content

Advertisement

SpringerLink
Log in

The Effect of Benefit Underreporting on Estimates of Poverty in the United States

  • Original Research
  • Published:
Social Indicators Research Aims and scope Submit manuscript

Abstract

The household income data used most frequently to estimate poverty rates in the United States substantially underreports the value of means-tested transfers. This paper investigates how underreporting affects estimates of the incidence and composition of poverty in the U.S. from 2013 to 2015. Specifically, I apply benefit adjustments for the underreporting of three social transfers to the Current Population Survey (CPS ASEC) to provide more accurate estimates of poverty rates. Diagnostic checks indicate that the imputed benefit adjustments are imperfect, but do provide a more accurate representation of household income than the uncorrected CPS data. In 2015, the benefit adjustments add more than $30 billion of income transfers to the CPS ASEC, primarily concentrated among low-income households with children. I test the effects of the benefit corrections on two conceptualizations of poverty: the U.S. Supplemental Poverty Measure (SPM) and a relative measure of poverty set at 50% of federal median income. In 2015, the SPM poverty rate for the total population falls from 14.3 to 12.7%, a 1.6 percentage point (11%) decline, after adjusting for underreporting. Among children, the SPM poverty rate falls from 16.1 to 12.8%, a 3.3 percentage point (20%) decline. The percent-of-median poverty rate experiences similar declines after applying the benefit imputations. The findings suggest that the uncorrected CPS data meaningfully overestimates the incidence of poverty in the U.S., particularly among households with children. Documentation for applying the benefit adjustments to the CPS is provided for improved estimates in future poverty research.

This is a preview of subscription content, log in via an institution to check access.

Access this article

Log in via an institution

Price excludes VAT (USA)
Tax calculation will be finalised during checkout.

Instant access to the full article PDF.

Institutional subscriptions

Fig. 1

Source of survey data: IPUMS-CPS database (Flood et al. 2018)

Fig. 2
Fig. 3

Similar content being viewed by others

Notes

  1. Each year, some survey respondents do not answer the survey questions on the value of social program programs (item nonresponse). The Census Bureau allocates participation and benefit values to households that do not provide a response, but are estimated to be participating in the given benefit (Wheaton and Tran 2018). Thus, what starts as a ‘missing data’ issue becomes a potential source of measurement error (if the imputed values are different from the true values) in the version of the CPS ASEC made public. The large majority of survey respondents, however, do provide answers to all social transfer questions (Kasprzyk 2005).

  2. If a survey respondent mistakenly reported SSI benefits as SSDI benefits, it is possible that adjusting for the underreporting of SSI while not adjusting SSDI benefits may double count the income source. This would, in turn, affect the TRIM3-adjusted poverty estimates presented in this paper. To account for that possibility, I present revised poverty estimates disaggregated by program in “Appendix 3” type to clarify how TRIM3 would affect poverty rates even if the SSI adjustments were to be excluded. I thank an anonymous reviewer for bringing this to my attention.

  3. I use the “source of welfare income” variable in the CPS ASEC to separate TANF benefits from other public assistance programs, such as state-provided General Assistance. When a respondent indicates that the source of his/her welfare income is both TANF and non-TANF assistance, I opt to include the total value in my calculation of TANF benefits. As the number of such cases is small (0.01% of respondents), removing the combined value from TANF calculations makes no substantive difference to the findings presented here.

  4. TRIM3 creates replicates of immigrant households in its simulations to account for uncertainty in ‘undocumented status’ versus ‘legal permanent resident status’ when allocating transfer benefits. In utilizing the TRIM3-adjusted survey data, researchers can choose to either keep the replicate households and use a TRIM3-provided weight, or to re-aggregate the cloned households back into a single household, in which case the CPS-provided weights can be utilized. As detailed in the replication dofiles in “Appendix 4”, I convert the cloned households back into their original households. I follow TRIM3 guidelines in utilizing the replication weights to aggregate the TRIM3-adjusted benefits back to the original household level.

  5. Access to administrative records on benefit dispersal that can be integrated into survey data are heavily restricted and not available for public use. Moreover, these records are generally available for only a small subset of states, as described before. Therefore, I am unable to compare the distribution of TRIM3 benefits directly to linked administrative-survey records.

  6. Removing SNAP, TANF, and SSI transfers from income provides a clearer account of to whom the TRIM3-adjusted benefits are being transferred. If I were to already take the transfers into account, this would affect the underlying income distribution (the X-axes of Fig. 2). For example, the zero-income households receiving SNAP benefits would no longer appear as zero-income households if SNAP benefits were already included.

References

  • Atkinson, A. (1998). Poverty in Europe. Oxford: Blackwell.

    Google Scholar 

  • Bee, C. A., Gathright, G. M. R., & Meyer, B. D. (2015). Bias from unit non-response in the measurement of income in household surveys. Chicago: University of Chicago Harris School of Public Policy.

    Google Scholar 

  • Bee, C. A., & Mitchell, J. (2017). Do older Americans have more income than we think?. U.S. Census Bureau: social, economic, and housing statistics division working paper, 2017–39.

  • Bishaw, A. (2017). How the Census Bureau measures poverty: With selected sources of poverty data. Suitland: U.S. Census Bureau.

    Google Scholar 

  • Bitler, M., Hoynes, H., & Kuka, E. (2017). Child poverty, the great recession, and the social safety net in the United States. Journal of Policy Analysis and Management, 36(2), 358–389. https://doi.org/10.1002/pam.21963.

    Article  Google Scholar 

  • Bound, J., Brown, C., & Mathiowetz, N. (2001). Measurement error in survey data. In J. J. Heckman & E. E. Leamer (Eds.), Handbook of econometrics (Vol. 5, pp. 3705–3843). Amsterdam: Elsevier.

    Chapter  Google Scholar 

  • Bound, J., & Krueger, A. B. (1991). The extent of measurement error in longitudinal earnings data: Do two wrongs make a right? Journal of Labor Economics, 9(1), 1–24.

    Article  Google Scholar 

  • Brady, D. (2009). Rich democracies, poor people: How politics explain poverty. New York: Oxford University Press.

    Book  Google Scholar 

  • Brady, D., & Destro, L. M. (2014). Poverty. In D. Beland, C. Howard, & K. J. Morgan (Eds.), The Oxford handbook of U.S. social policy (pp. 585–602). New York, NY: Oxford University Press.

    Google Scholar 

  • Brady, D., Finnigan, R., & Hübgen, S. (2017). Rethinking the Risks of poverty: A framework for analyzing prevlances and penalties. American Journal of Sociology., 123, 740–786.

    Article  Google Scholar 

  • Burkhauser, R. V., Hérault, N., Jenkins, S. P., & Wilkins, R. (2018). Survey under-coverage of top incomes and estimation of inequality: What is the role of the UK’s SPI adjustment? Fiscal Studies, 39(2), 213–240. https://doi.org/10.1111/1475-5890.12158.

    Article  Google Scholar 

  • Cantillon, B., & Vandenbroucke, F. (2014). Reconciling work and poverty reduction: how successful are European welfare states? (International policy exchange series). New York: Oxford University Press.

    Google Scholar 

  • Chen, W.-H., & Corak, M. (2005). Child poverty and changes in child poverty in rich countries since 1990. UNICEF innocenti working paper, 2005–02.

  • Corak, M. (2005). Principles and practicalities for measuring child poverty in the rich countries. IZA working papers (DP No. 1579).

  • Decancq, K., Fleurbeay, M., & Maniquet, F. (2015). Multidimensional poverty measurement with individual preferences. CORE discussion papers.

  • Flood, S., King, M., Ruggles, S., & Warren, J. R. (2018). Integrated public use microdata series, current population survey: Version 60 [dataset]. Minneapolis, MN: University of Minnesota.

    Google Scholar 

  • Foster, J. E. (1998). Absolute versus relative poverty. The American Economic Review, 88(2), 335–341.

    Google Scholar 

  • Fox, L., Garfinkel, I., Kaushal, N., Waldfogel, J., & Wimer, C. (2014). Waging war on poverty historical trends in poverty using the supplemental poverty measure. NBER working paper 19789. Cambridge, MA: National Bureau of Economic Research.

  • Fox, L., Heggeness, M., Pacas, J., & Stevens, K. (2017). Using SNAP administrative records to evaluate poverty measurement. U.S. Census Bureau, Working Paper Number: SEHSD-WP2017-49.

  • Fox, L., & Renwick, T. (2016). The supplemental poverty measure: 2015. U.S. Census Bureau, Current Population Reports.

  • Goedemé, T. (2013). How much confidence can we have in EU-SILC? Complex sample designs and the standard error of the Europe 2020 poverty indicators. Social Indicators Research, 110(1), 89–110. https://doi.org/10.1007/s11205-011-9918-2.

    Article  Google Scholar 

  • Gornick, J. C., & Jäntti, M. (2012). Child poverty in cross-national perspective: Lessons from the Luxembourg income study. Children and Youth Services Review, 34(3), 558–568. https://doi.org/10.1016/j.childyouth.2011.10.016.

    Article  Google Scholar 

  • Gray, K. F., Fisher, S., & Lauffer, S. (2015). Characteristics of supplemental nutrition assistance program households: Fiscal year 2015. Washington D.C.: U.S. Department of Agriculture.

    Google Scholar 

  • Haskins, R., & Sawhill, I. (2009). Creating an opportunity society. Washington, D.C.: Brookings Institution Press.

    Google Scholar 

  • Henry, M., Shivji, A., de Sousa, T., Cohen, R., & Abt Associates (2015). The 2015 annual homeless assessment report (AHAR) to congress. The U.S. Department of Housing and Urban Development Office of Community Planning and Development.

  • Jäntti, M., Veli-Matti, T., & Marlier, E. (2013). The use of registers in the context of EU–SILC: Challenges and opportunities. Eurostat statistical working papers.

  • Kapteyn, A. & Ypma, J. (2007). Measurement error and misclassification: A comparison of survey and administrative data. Journal of Labor Economics, 25(3), 513–551. https://doi.org/10.1086/513298

    Article  Google Scholar 

  • Kasprzyk, D. (2005). Measurement error in household surveys: Sources and measurement. Princeton: Mathematica Policy Research.

    Google Scholar 

  • Keith-Jennings, B. (2012). SNAP plays a critical role in helping children. Washington D.C.: Center on Budget and Policy Priorities.

    Google Scholar 

  • Kristensen, N., & Westergard-Nielsen, N. (2006). A large-scale validation study of measurement errors in longitudinal survey data. IZA discussion paper series, 2939 (Institute for the Study of Labor (IZA)).

  • Meyer, B. D., & Mittag, N. (2015). Using linked survey and administrative data to better measure income: implications for poverty, program effectiveness and holes in the safety net. National bureau of economic research working paper series, no. 21676, https://doi.org/10.3386/w21676. Accessed June 2018.

    Google Scholar 

  • Meyer, B. D., & Sullivan, J. X. (2006). Consumption, income, and material well-being after welfare reform. National bureau of economic research working paper series, no. 11976. https://doi.org/10.3386/w11976. Accessed June 2018.

    Google Scholar 

  • Meyer, B. D., Wallace, K. C. M., & Sullivan, J. X. (2009). The under-reporting of transfers in household surveys: Its nature and consequences. National bureau of economic research working paper series, no. 15181. https://doi.org/10.3386/w15181. Accessed June 2018.

    Google Scholar 

  • Meyer, B. D., & Wu, D. (2018). The poverty reduction of social security and means-tested transfers. National bureau of economic research working paper series, no. 24567, https://doi.org/10.3386/w24567.

    Google Scholar 

  • Mittag, N. (2016). Correcting for misreporting of government benefits. IZA discussion paper series (no. 10266).

  • Moore, J. C., Stinson, L., & Welniak, E. J., Jr. (2000). Income measurement error in surveys: A review. Journal of Official Statistics, 16(4), 331–361.

    Google Scholar 

  • Nicoletti, C., Peracchi, F., & Foliano, F. (2011). Estimating income poverty in the presence of missing data and measurement error. Journal of Business & Economic Statistics, 29(1), 61–72.

    Article  Google Scholar 

  • Nolan, B., & Whelan, C. T. (2011). Poverty and deprivation in Europe. Oxford: Oxford University Press.

    Book  Google Scholar 

  • Office of Family Assistance. (2015). State TANF spending in FY 2015. Washington D.C.: U.S. Department of Health & Human Services: Administration for Children & Families.

    Google Scholar 

  • Provencher, A. J. (2011). Unit of analysis for poverty measurement: A comparison of the supplemental poverty measure and the official poverty measure. Paper presented at the 2011 joint statistical meetings.

  • Rainwater, L., & Smeeding, T. M. (2003). Poor kids in a rich country: America’s children in comparative perspective. New York: Russell Sage Foundation.

    Google Scholar 

  • Rodgers, W. L., Brown, C., & Duncan, G. J. (1993). Errors in survey reports of earnings, hours worked, and hourly wages. Journal of the American Statistical Association, 88(424), 1208–1218. https://doi.org/10.1080/01621459.1993.10476400.

    Article  Google Scholar 

  • Social Security Administration. (2017). SSI annual statisitical report, 2015. Social security: Research, statistics, & policy analysis.

  • Stevens, K., Fox, L., & Heggeness, M. (2018). Precision in measurement: Using State-level SNAP administrative records and the transfer income model (TRIM3) to evaluate poverty measurement. U.S. census bureau, working paper number: SEHSD-WP2018-15.

  • U.S. Census Bureau (1992). Measuring the Effect of benefits and taxes on income and poverty: 1992. U.S. census bureau, P60-186RD.

  • U.S. Department of Agriculture. (2017). Supplemental nutrition assistance program (SNAP): National level annual summary. Alexandria: USDA Food & Nutrition Service.

    Google Scholar 

  • Weinberg, D. H. (2004). Income data quality issues in the annual social and economic supplement to the current population survey. Paper presented at the American Enterprise Institute-University of Maryland Seminar on Poverty Measurement, College Park, Maryland.

  • Wheaton, L. (2008). Underreporting of means-tested transfer programs in the CPS and SIPP. Washington D.C.: The Urban Institute.

    Google Scholar 

  • Wheaton, L., & Tran, V. (2018). The antipoverty effects of the supplemental nutrition assistance program. The Urban Institute, research report.

  • Winship, S. (2016). Poverty after welfare reform. New York: Manhattan Institute.

    Google Scholar 

  • Zedlewski, S., & Giannarelli, L. (2015). TRIM: A tool for social policy analysis. Washington D.C.: The Urban Institute.

    Google Scholar 

Download references

Acknowledgements

For providing helpful comments and suggestions, I am grateful to Bea Cantillon, Brian Nolan, Wim Van Lancker, Steven Pressman, Joyce Morton, Minna Nurminen, Jane Waldfogel, Chris Wimer, Michael Wiseman, and participants of the 2017 LIS User Conference. Any errors are my own. Funding was provided by FWO - Research Foundation Flanders.

Author information

Authors and Affiliations

  1. University of Antwerp, Antwerp, Belgium

    Zachary Parolin

Authors
  1. Zachary Parolin
    View author publications

    You can also search for this author in PubMed Google Scholar

Corresponding author

Correspondence to Zachary Parolin.

Additional information

Publisher's Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Electronic Supplementary Material

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 41 kb)

Appendices

Appendix 1: State-Level Estimates of Poverty and Underreporting

See Tables 4, 5.

Table 4 Benefit underreporting by state before and after TRIM3 adjustments (3-year average, 2013–2015)
Full size table
Table 5 State-level estimates of poverty before and after TRIM3 adjustments (3-year average, 2013–2015)
Full size table

1.1 Appendix 2: Change in Conditional Median Benefit Values and Benefit Coverage by Household Status, 2015

  

CPS ASEC before TRIM3

CPS ASEC after TRIM3

Absolute change

SNAP

 

Households with children

Median value

$3288

$3366

$78

Coverage rate

17.9%

28.8%

10.9%*

Households without children

Median value

$1344

$1344

$0

Coverage rate

8.0%

13.5%

5.5%*

Total population

Median value

$2160

$2160

$0

Coverage rate

11.1%

18.3%

7.2%*

TANF

Households with children

Median value

$2736

$3048

$312

Coverage rate

2.0%

4.4%

2.4%*

Households without children

Median value

$3600

$2184

− $1416

Coverage rate

0.2%

0.1%

− 0.2%

Total population

Median value

$2808

$3048

$240

Coverage rate

0.8%

1.4%

0.6%*

SSI

Households with children

Median value

$8772

$8796

$24

Coverage rate

3.5%

6.0%

2.5%*

Households without children

Median value

$8784

$8796

$12

Coverage rate

4.8%

5.5%

0.7%

Total population

Median value

$8784

$8796

$12

Coverage rate

4.4%

5.7%

1.3%*

  1. Median value refers to the median benefit among households receiving any positive value of the benefit in 2015. Coverage rate refers to the share of households receiving any positive value of the benefit in 2015. Asterisk (*) in Absolute Change column indicates change in coverage rates is statistically different from zero (95% confidence). Thus, the difference in coverage rates for TANF and SSI among households without children is not statistically significant. Similarly, the large decline in the conditional median among households without children receiving TANF is of little consequence: the sample size of such households is tiny (29 households) and the difference in the conditional means (not displayed) is insignificant

Appendix 3: Effect of Benefit Underreporting on Poverty Estimates by Program

 

TRIM3 Adjustments only for:

 

Unadjusted (%)

SNAP (%)

TANF (%)

SSI (%)

All (%)

SPM

14.3

13.3

14.2

13.9

12.7

Percentage-point change

–

− 1.0

− 0.1

− 0.4

− 1.6

Share of pre-post difference

–

63.2

8.0

25.8

–

50% federal median

16.0

15.1

15.9

15.8

14.8

Percentage-point change

–

− 0.9

− 0.1

− 0.2

− 1.2

Share of pre-post difference

–

74.2

8.2

18.4

–

  1. 95% confidence intervals on poverty estimates: ± 0.17%. Differences in poverty estimates before and after TRIM3’s TANF adjustments are not statistically significant. “All” column includes TRIM3 benefit adjustments for SNAP, TANF, and SSI. “Share of pre-post difference” indicates contribution of individual program relative to total change in poverty after adjusting for all three programs

Appendix 4: Documentation for Applying Benefit Adjustments to U.S. Current Population Survey

The documentation below provides guidance on adjusting for benefit underreporting in the CPS ASEC and for replicating the analyses in this paper. The replication packages contains five steps:

  1. I.

    Accessing TRIM3

  2. II.

    Accessing the CPS ASEC

  3. III.

    Downloading TRIM3 Files

  4. IV.

    Merging TRIM to the CPS ASEC

  5. V.

    Re-Estimating Poverty Rates

    1. a.

      Supplemental Poverty Measure

    2. b.

      50% of Median Income

  6. I.

    Accessing TRIM3

Researchers can request access the Urban Institute’s TRIM3 simulation model upon registration via online form. The link to register is: http://trim3.urban.org/Registration/. There is no charge to access the baseline data, but users must share their intended use of the TRIM3 data. Registration is necessary to download the TRIM3 data on imputed SNAP, TANF, and SSI benefit receipt.

More information on TRIM: http://trim3.urban.org.

More information on the Urban Institute: http://urban.org.

  1. II.

    Accessing the CPS ASEC

There are multiple ways to access the Annual Social and Economic Supplement of the Current Population Survey (CPS ASEC, also referred to as the March CPS files). Two options include:

U.S. Census Bureau: https://thedataweb.rm.census.gov/ftp/cps_ftp.html.

IPUMS CPS (after registration): https://cps.ipums.org/cps/.

The replication package presented here uses the 2014—2016 CPS ASEC files (referring to reference years 2013–2015) from IPUMS CPS. Researchers using the Census files should be aware that some of the variable labels in the dofiles presented below may need to be converted from IPUMS to Census labeling schema (i.e. changing hseq to h_seq).

To compute Supplemental Poverty Measure (SPM) rates, I utilize historical SPM data from the Center on Poverty & Social Policy at Columbia University. Researchers can access the public-use files after registering at https://www.povertycenter.columbia.edu/historical-spm-data-reg. Download the 2016 “Stata 12 DTA” file (CPS ASEC 2016 referring to reference year 2015) and save locally. The code provided in Dofile 2 (below) will merge the file into the CPS ASEC and allow for re-estimation of SPM poverty rates with TRIM-adjusted benefits.

  1. III.

    Download TRIM3 Files

After registering for and receiving access to TRIM3, you can then download the files needed to adjust for benefit underreporting in the CPS ASEC. To do so, visit the TRIM3 website (trim3.urban.org) and select the TRIM3 Navigator link using your login credentials. On the next page, select “Microdata” or “Microdata Examiner.”

This paper uses the 2013 to 2015 microdata files, but TRIM3 files are available for the 1993 March CPS onward (if using the older files, however, take note that TRIM3 imputation procedures may have changed over time). Here, I detail how to download and merge the 2015 TRIM file into the CPS ASEC, but the same procedure applies for other years.

Select the 2015 (2016 CPS ASEC) input data set. You will then download, at a minimum, the following files, selecting the “extract data” link next to each. The sub-bullets below each file name indicate the variables within each extract that you should select prior to download:

  • Alien2015 Person

    • HOUSEHOLDID

    • FAMILYID

    • PERSONID

    • CpsPersonID

    • LineNumber

    • PersonWeight

  • Alien2015 Household

    • HOUSEHOLDID

    • AlienHouseholdSplit

    • HouseholdWeight

    • HighIncomeClone

    • OldIdentifier

  • Alien2015 Family

    • HOUSEHOLDID

    • FAMILYID

    • CpsIdentifier

  • SN2015_

    • HOUSEHOLDID

    • PERSONID

    • ANNUALBENEFITSRECEIVED

  • SSI2015_

    • HOUSEHOLDID

    • PERSONID

    • ANNUALSSIBENEFITSRECEIVED

  • TF2015_

    • HOUSEHOLDID

    • PERSONID

    • ANNUALBENEFITSRECEIVED

Under “formatting options”, select Stata 2.1 format (if using Stata) and extract each set of data. Follow the link on the proceeding page to begin the download. Save each file into a local folder. If downloading files from multiple years, I recommend saving them into separate folders with the respective year as the folder title. Recommended file names for the 2015 downloads (useful if following the merge instructions in the dofiles below) are, respectively: input2015p, input2015 h, input2015f, snap2015, ssi2015, tanf2015.

  1. IV.

    Merging TRIM3 into CPS ASEC

After downloading the TRIM3 files, you can now merge them into the CPS ASEC. This process consists of three steps: the first is to merge the separate TRIM3 files into one unified file. The second step is to merge the unified TRIM3 file into the CPS. The third step then treats the adjusted CPS ASEC file to account for the presence of high-income clones and alien replicates within the TRIM3 data.

Download the Stata dofile here to view and run the commands:

Dofile 1 of 2: Merging TRIM3 into CPS ASEC

Download: https://github.com/cpstotrim/dofiles/blob/master/replication_dofile1.do.

  1. V.

    Re-Estimating Poverty Rates

The Stata dofile below contains the code to estimate poverty rates with the TRIM3-adjusted benefits and to replicate each of the analyses in this paper.

Download the Stata dofile here to view and run the commands:

Dofile 2 of 2: Estimating Poverty Rates with TRIM3-Adjusted Benefits

Download: https://github.com/cpstotrim/dofiles/blob/master/replication_dofile2.do.

Rights and permissions

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Parolin, Z. The Effect of Benefit Underreporting on Estimates of Poverty in the United States. Soc Indic Res 144, 869–898 (2019). https://doi.org/10.1007/s11205-018-02053-0

Download citation

  • Accepted:

  • Published:

  • Issue Date:

  • DOI: https://doi.org/10.1007/s11205-018-02053-0

Keywords

Search

Navigation