Springer Nature is making SARS-CoV-2 and COVID-19 research free. View research | View latest news | Sign up for updates

Comparative Assessment of Methods for Measuring Consensual Poverty: Sort Card Versus CAPI

Abstract

Poverty means more than having a low income and includes exclusion from a minimally accepted way of life. It is now common practice in Europe to measure progress against poverty in terms of low income, material deprivation rates and some combination of both. This makes material deprivation indicators, and their selection, highly significant in its own right. The ‘consensual poverty’ approach is to identify deprivation items which a majority of the population agree constitute life’s basic necessities, accepting that these items will need revised over time to reflect social change. Traditionally, this has been carried out in the UK through specialised poverty surveys using a Sort Card (SC) technique. Based on analysis of a 2012 omnibus survey, and discussions with three interviewers, this article examines how perception of necessities is affected by mode of administration—SC and Computer Assisted Personal Interviewing (CAPI). More CAPI respondents scored deprivation items necessary. Greatest disparities are in material items where 25 out of 32 items were significantly higher via CAPI. Closer agreement is found in social participation with 3 out of 14 activities significantly different. Consensus is higher on children’s material deprivation. We consider influencing variables which could account for the disparities and believe that the SC method produces a more considered response. However, in light of technological advances, we question how long the SC method will remain socially acceptable. This paper concludes that the CAPI method can be easily modified without compromising the benefits of the SC method in capturing thoughtful responses.

This is a preview of subscription content, log in to check access.

Notes

  1. 1.

    A relative risk ratio of 2.0 means twice the risk, a score of 0.5 means half the risk, a score of 3.0 is three times the risk and a score of 0.33 is a third of the risk and so forth. A relative risk of 1 would indicate that there are no differences between the two groups (see Gordon 2012 for more information).

  2. 2.

    Confidence intervals provide information about the range in which the true value lies with a certain degree of probability. Thus a 95 % confidence interval means we can be sure 95 % of the time that the findings are not due to chance. If the 95 % confidence intervals of a relative risk ratio span 1.0 then we cannot be confident at the 5 % level that the difference is significant.

References

  1. Applegate, B., & Sanborn, J. (2011). Public opinion on the harshness of local courts: An experimental test of question working effects. Criminal Justice Review, 36(4), 487–497.

  2. Bajekal, M., Harries, T., Breman, R., & Woodfield, K. (2004). Review of disability estimates and definitions. In-house Report No. 128. London: Department of Work and Pensions.

  3. Berthoud, R., Blekesaune, M., Hancock, R. (2006). Arepoorpensionersdeprived’? Department for Work and Pensions Research Report No 364, Corporate Document Services.

  4. Blake, M., Clery, E., d’Ardenne, J. & Legard, R. (2009). Cognitive testing: British Social Attitudes child poverty questions. Report to Department for Work and Pensions, Norwich: Department for Work and Pensions.

  5. Bowling, A. (2005). Mode of questionnaire administration can have serious effects on data quality. Journal of Public Health, 27(3), 281–291.

  6. Bradbum, N. M. (1983). Response effects. In P. H. Rossi, J. D. Wright, & A. B. Anderson (Eds.), Handbook of survey research (pp. 289–328). New York: Academic Press.

  7. Cantril, H. (1944). Gauging public opinion. Princeton, NJ: Princeton University Press.

  8. Department for Work and Pensions. (2003). Measuring child poverty consultation: Final conclusions. London: DWP.

  9. Dewar, A. (2005). Improving survey quality in the measurement of social capital. London: Office for National Statistics.

  10. Dominy, N. & Kempson, E. (2006). Understanding older people’s experiences of poverty and material deprivation. Department for Work and Pensions Research Report No 363, Corporate Document Services.

  11. Eurostat. (2010). Income poverty and material deprivation in european countries. Eurostat Working Paper. Brussels: European Commission. ISSN: 1977-0375.

  12. Fahmy, E., Pemberton, S. & Sutton, E. (2012). Cognitive testing of the 2011 Poverty and Social Exclusion Survey, PSE study. Working Paper: Methods Series No 17. Poverty and Social Exclusion Study. http://poverty.ac.uk/methods-working-papers-poverty-poverty-measurement-social-exclusion-deprivation-attitudes/cognitive

  13. Finch, H. & Kemp, P. (2006). Which pensioners don’t spend their income and why? Department for Work and Pensions Research Report No 334, Corporate Document Services.

  14. Gordon, D. (2012). Why use relative risks? PSE: Statistical Briefing Note Number 1. Poverty and Social Exclusion Study. http://poverty.ac.uk/pse-research/pse-uk/statistical-notes

  15. Gordon, D., Adelman, L., Ashworth, K., Bradshaw, J., Levitas, R., Middleton, S., et al. (2000). Poverty and social exclusion in Britain. York: Joseph Rowntree Foundation.

  16. Gordon, D., Nandy, S., Patsios, D. (2012). Bias in the Northern Ireland Omnibus June 2012 Sort Card Module? PSE: Statistical Briefing Note No 2, Poverty and Social Exclusion Study. http://poverty.ac.uk/pse-research/pse-uk/statistical-notes

  17. Hillyard, P., Kelly, G., McLaughlin, E., Patsios, D., & Tomlinson, M. (2003). Bare necessities: Poverty and social exclusion in Northern Ireland. Belfast: Democratic Dialogue.

  18. Hillyard, P. & Patsios, D. (2013). Study of expenditure poverty in Northern Ireland. Office of the First Minister and Deputy First Minister. www.ofmdfm.gov.uk

  19. Kelly, G., Tomlinson, M., Daly, M., Hillyard, P., Nandy, S. & Patsios, D. (2012). The necessities of life in Northern Ireland, Poverty and Social Exclusion Study UK. Analysis Working Paper Number 1.

  20. Krosnick, J., & Smith, W. (1994). Attitude strength. In V. S. Ramachandran (Ed.), Encyclopedia of human behavior. San Diego, CA: Academic Press.

  21. Krumpal, I. (2013). Determinants of social desirability bias in sensitive surveys: A literature review. Quality & Quantity, 47(4), 2025–2047.

  22. Legard, R., Gray, M. & Blake, M. (2008). Cognitive testing: Older people and the FRS material deprivation questions. Working Paper No 55, London: Department for Work and Pensions.

  23. Mack, J., & Lansley, S. (1985). Poor Britain. London: Allen and Unwin.

  24. McAuley, C., Hillyard, P., McLaughlin, E., Tomlinson, M., Kelly, G. & Patsios, D. (2003). The necessities of life in Northern Ireland. Working Paper 1, Poverty and Social Exclusion Northern Ireland Project. Belfast: Queens University Belfast.

  25. McKay, S. (2008). Measuring material deprivation among older people: Methodological study to revise the Family Resources Survey questions. Working Paper No 54, London: Department for Work and Pensions.

  26. Nicolaas G. & Tipping S. (2004). ‘HSE social capital questions: comparison of interviewer-administered questions and self-administered questions’, Prepared for Socio Inequalities Branch, Office for National Statistics. Referenced in Raham, Z & Dewar, A. (2006) The Impact of Mode on the Comparability of Survey Data. Survey Methodology Bulletin, Special Edition No 58, August 2006, London: Office for National Statistics.

  27. Nicolaas, G. & Tipping, S. (2006). Mode effects in social capital surveys. In ONS (2006), Survey Methodology Bulletin (pp 56–74). Special Edition No 58, August 2006. London: Office for National Statistics.

  28. Office for National Statistics. (2006). Survey Methodology Bulletin. Special Edition No 58, August 2006, London: Office for National Statistics.

  29. Pantazis, C., Townsend, P. & Gordon, D. (1999). The necessities of life in Britain. Working Paper No 1, Poverty and Social Exclusion Survey of Britain, Bristol: University of Bristol.

  30. Presser, S., Rothgeb, J., Couper, M., et al. (2004). Methods for testing and evaluating survey questionnaires. New Jersey: Wiley.

  31. Rahman, Z. & Dewar, A. (2006). The impact of mode on the comparability of survey data. In ONS (2006) Survey Method Bulletin (pp. 3–10). Special Edition No 58, August 2006. London: Office for National Statistics.

  32. Ralph, K., Palmer, K., & Olney, J. (2011). Subjective well-being: A qualitative investigation of subjective well-being questions. London: Office for National Statistics.

  33. Schuldt, J., Konrath, S., & Schwarz, N. (2011). “Global Warming” or “Climate Change”? Whether the planet is warming depends on question wording. Public Opinion Quarterly, 75(1), 115–124.

  34. Schuman, H., & Presser, S. (1996). Questions and answers in attitude surveys: Experiments on question form, wording and context. London: Sage Publications.

  35. Simón, T., Suengas, A., Ruiz-Gallego-Largo, T., & Bandrés, J. (2013). Positive bias is a defining characteristic of aging to the same extent as declining performance. International Journal of Psychology, 48(4), 704–714.

  36. Singer, E., & Couper, M. (2014). The effect of question wording on attitudes toward prenatal testing and abortion. Public Opinion Quarterly, 78(3), 751–760.

  37. Smith, T. (1997). The impact of the presence of others on a respondent’s answers to questions. International Journal of Public Opinion Research, 9(1), 33–47.

  38. Smyth, J., Christian, L., & Dillman, D. (2008). Does ‘yes or no’ on the telephone mean the same as ‘check-all-that-apply’ on the web. Public Opinion Quarterly, 72(1), 103–113.

  39. Sudman, S., & Bradburn, N. (1974). Response effects in surveys: A review and synthesis. Chicago: Aldine Publishing Company.

  40. Tomlinson, M., Hillyard, P., & Kelly, G. (2014). Child poverty in Northern Ireland: Results from the poverty and social exclusion study. In Child Poverty Alliance (ed.) Beneath the Surface: Child Poverty in Northern Ireland (pp. 11–34). Belfast: Child Poverty Alliance. http://www.ci-ni.org.uk/child-poverty-alliance

  41. Townsend, P. (1979). Poverty in the United Kingdom. Harmondsworth: Penguin.

Download references

Acknowledgments

The research reported here was funded by the Economic and Social Research Council, Grant Number RES-060-25-0052

Author information

Correspondence to Grace P. Kelly.

Ethics declarations

Conflict of interest

The authors declare that they have no conflict of interest.

Appendices

Appendix 1

See Table 6.

Table 6 Adult items and activities

Appendix 2

See Table 7.

Table 7 Children’s items and activities

Appendix 3

See Table 8.

Table 8 Number of significant differences across social groups

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Kelly, G.P., Tomlinson, M.W. & Patsios, D. Comparative Assessment of Methods for Measuring Consensual Poverty: Sort Card Versus CAPI. Soc Indic Res 129, 677–698 (2016). https://doi.org/10.1007/s11205-015-1150-z

Download citation

Keywords

  • Deprivation
  • Socially perceived necessities
  • Poverty
  • Social exclusion
  • Sort Card
  • CAPI
  • Mode of administration