Skip to main content
SpringerLink
Log in

Diversity in Survey Items and the Comparability Problem

  • Chapter
  • First Online:
Book cover Diversity in Survey Questions on the Same Topic

Part of the book series: Social Indicators Research Series ((SINS,volume 68))

Abstract

This chapter starts with an introduction to the incomparability problem and an overview of the diversity in survey items. This is followed by a description of the problem of incomparability of the time series on life satisfaction in the USA, Japan and The Netherlands to illustrate the problem. Next two conventional methods for scale transformation are described: the Linear Stretch Method and the Semantic Judgment of Fixed Word Value Method. We explain why they fall short to solve the comparability problem and conclude that these shortcomings require further investigations and innovative solutions to solve them.

This is a preview of subscription content, log in via an institution to check access.

Access this chapter

Log in via an institution
Chapter
USD 29.95
Price excludes VAT (USA)
  • Available as PDF
  • Read on any device
  • Instant download
  • Own it forever
eBook
USD 79.99
Price excludes VAT (USA)
  • Available as EPUB and PDF
  • Read on any device
  • Instant download
  • Own it forever
Softcover Book
USD 99.99
Price excludes VAT (USA)
  • Compact, lightweight edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info
Hardcover Book
USD 99.99
Price excludes VAT (USA)
  • Durable hardcover edition
  • Dispatched in 3 to 5 business days
  • Free shipping worldwide - see info

Tax calculation will be finalised at checkout

Purchases are for personal use only

Institutional subscriptions

Notes

  1. 1.

    We use the term ‘item’ for a survey question and its corresponding response options.

  2. 2.

    We use the term ‘anchor points’ for the response options at both ends of a discrete scale. In the case of a continuous distribution, we use the term ‘extremes’ to refer to the boundaries of the continuum that bounds this distribution.

  3. 3.

    http://www.gallup.com/products/170987/gallup-analytics.aspx. Assessed 3 February 2016.

  4. 4.

    The Weighted Average Approach is a generalization of the Rank Method, but does not require that the numbers assigned to the response options are equal to the ranks of these response options to calculate a sample mean.

  5. 5.

    Ridit stands for Relative to an Identified Distribution Integral Transformation.

  6. 6.

    A VAS is a measurement instrument used to ask respondents to specify their level of agreement with a statement on a subjective topic by indicating a position along a continuous line between two end-points.

  7. 7.

    A network of international multidisciplinary researchers committed to the measurement of health-related quality of life, www.euroqol.org.

References

  • Andrews, F. M., & Withey, S. B. (1976). Social indicator of well-being, Americans’ perceptions of life quality. New York: Plenum Press.

    Book  Google Scholar 

  • Bălţătescu, S. (2002). Problems of transforming scales of life satisfaction. Euromodule workshop, Berlin. Retrieved January 29, 2014, http://worlddatabaseofhappiness.eur.nl-hap_bib-freetexts-baltatescu_sm_2002A.pdf

  • Batz, C., Parrigon, S. & Tay, L. (2015). The impact of scale transformations on National subjective well-being scores. Social Indicators Research, Published online: 04 Sep 2015. doi: 10.1007/s11205-015-1088-1

  • Bjørnskov, C. (2010). How comparable are the gallup world poll life satisfaction data? Journal of Happiness Studies, 11, 41–60. doi:10.1007/s10902-008-9121-6.

    Article  Google Scholar 

  • Bross, I. D. J. (1958). How to use ridit analysis. Biometrics, 14, 18–38.

    Article  Google Scholar 

  • Campbell, A., Converse, P. E., & Rodgers, W. L. (1976). The quality of American life. Perceptions, evaluations, and satisfactions. New York: Russel Sage Foundation. Available at: http://babel.hathitrust.org/cgi/pt?id=mdp.39015071884634;view=2up;seq=6.

    Google Scholar 

  • Cummins, R. A. (1997). The comprehensive quality of life scale – intellectual-cognitive disability (ComQol-I5) (5th ed.). Melbourne: School of Psychology, Deakin University.

    Google Scholar 

  • Cummins, R. A. & Gullone, E. (2000). Why we should not use 5-point Likert scales: The case for subjective quality of life measurement. Proceedings, second international conference on quality of life in cities (pp. 74–93). Singapore: National University of Singapore

    Google Scholar 

  • Cummins, R. A. (2003). Normative life satisfaction: Measurement issues and homeostatic model. Social Indicators Research, 64, 225–240.

    Article  Google Scholar 

  • DeJonge, T. (2009). The state of play in measuring SWB in the Netherlands. Paper for the OECD meeting Measuring subjective well-being: An opportunity for National Statistical Offices?, Florence, Italy. Retrieved January 29, 2016, Available at: http://worlddatabaseofhappiness.eur.nl/hap_bib/freetexts/dejonge_t_2009a.pdf

  • DeJonge, T., Veenhoven, R., & Arends, L. R. (2015). ‘Very Happy’ is not always equally happy on the meaning of verbal response options in survey questions. Journal of Happiness Studies, 16(1), 77–101. doi:10.1007-s10902-013-9497-9.

  • Diener, E., & Diener, C. (1996). Most people are happy. Psychological Science, 7, 181–185.

    Article  Google Scholar 

  • Donaldson, G. W. (1998). Ridit scores for analysis and interpretation of ordinal pain data. European Journal of Pain, 2, 221–227.

    Article  Google Scholar 

  • Frijters, P., Johnston, D. W., & Shields, M. A. (2008). Happiness dynamics with quarterly life event data (IZA Discussion Paper No. 3604).

    Google Scholar 

  • Gallup (2016). Understanding how gallup uses the Cantril scale. Retrieved January 29, 2016, http://www.gallup.com/poll/122453/understanding-gallup-uses-cantril-scale.aspx

  • Guven, C., Senik, C., & Stichnoth, H. (2011). You can’t be happier than your wife. Happiness gaps and divorce (Paris School of Economics, Working Paper No. 2011–01, halshs-00555427).

    Google Scholar 

  • Hull, C. L. (1922). The conversion of test scores into series which shall have any assigned mean and degree of dispersion. Journal of Applied Psychology, 6(3), 298–300.

    Article  Google Scholar 

  • Jones, L. V., & Thurstone, L. L. (1955). The psychophysics of semantics. An experimental investigation. The Journal of Applied Psychology, 39(1), 31–36.

    Article  Google Scholar 

  • Kalmijn, W. M. (2010). Quantification of happiness inequality. Dissertation, Erasmus University Rotterdam, The Netherlands. Enschede: Ipskamp Drukkers. Available at http://repub.eur.nl/pub/21777

  • Kalmijn, W. M., Arends, L. R., & Veenhoven, R. (2011). Happiness scale interval study, methodological considerations. Social Indicators Research, 102(3), 497–515. doi:10.1007/s11205-010-9688-2.

    Article  Google Scholar 

  • Kapteyn, A., Smith, J. P., & van Soest, A. (2007). Vignettes and self-reports of work disability in the United States and the Netherlands. American Economic Review, 97(1), 461–473.

    Article  Google Scholar 

  • Lee, H., Kim, K. D., & Shin, D. C. (1982). Perceptions of quality of life in an industrializing country: The case of the Republic of Korea. Social Indicators Research, 10, 297–317. doi:10.1007/BF00301097.

    Article  Google Scholar 

  • Lim, H. E. (2008). The use of different happiness rating scales: Bias and comparison problem? Social Indicators Research, 87, 259–267. doi:10.1007/s11205-007-9171-x.

    Article  Google Scholar 

  • Lodge, M. (1981). Magnitude scaling: Quantitative measurement of opinions. Beverly Hills: Sage Publications, Series Quantitative applications in the social sciences, ISSN 0149-192X; 07-025. doi: 10.4135/9781412984874

  • Mazaheri, M., & Theuns, P. (2009). Effects of varying response formats on self-ratings of life-satisfaction. Social Indicators Research, 90, 381–395. doi:10.1007/s11205-008-9263-2.

    Article  Google Scholar 

  • McDowell, I. (2006). Measuring health. A guide to rating scales and questionnaires (3rd ed.). Oxford: Oxford University Press. http://a4ebm.org/sites/default/files/Measuring%20Health.pdf.

    Book  Google Scholar 

  • Pudney, S. (2010). An experimental analysis of the impact of survey design on measures and models of subjective wellbeing (ISER Working Paper Series, 2010–20). Economic & Social Research Council. Retrieved February 3, 2016, www.iser.essex.ac.uk/research/publications/working-papers/iser/2010-20.pdf

  • Sangster, R. L., Willits, F. K., Saltiel, J., Lorenz, F. O., & Rockwood, T. H. (2001). The effect of numerical labels on response scales. Article presented at the Annual Meeting of the American Statistical Association, Atlanta. http://www.bls.gov-osmr-pdf-st010120.pdf

  • Saris, W. E. & Gallhofer, I. N. (2007). Design, evaluation, and analysis of questionnaires for survey research(Wiley series in survey methodology). New York: Wiley-Interscience. ISBN 978-0-470-11495-7, e-ISBN 978-0-470-16519-5

    Google Scholar 

  • Schwarz, N., Knauper, B., Hippler, H. J., Noelle-Neumann, E., & Clark, W. (1991). Rating scales: Numeric values may change the meaning of scale labels. The Public Opinion Quarterly, 55, 570–582. http://www.jstor.org-stable-2749407

  • Szende, A., Oppe, M., & Devlin, N. (Eds.). (2007). EQ-5D value sets. Inventory, comparative review and user guide. Dordrecht: Springer.

    Google Scholar 

  • Szende, A., Janssen, B., & Cabases, J. (Eds.). (2014). Self-reported population health: An international perspective based on EQ-5D. Dordrecht: Springer. doi:10.1007/978-94-007-7596-1.

    Google Scholar 

  • Uwawunkonye, E. G., & Anaene, O. I. C. (2013). A comparative study between ridit and modified ridit analysis. American Journal of Theoretical and Applied Statistics, 2(6), 248–254. doi:10.11648/j.ajtas.20130206.23.

    Article  Google Scholar 

  • Valentine, N., Verdes-Tennant, & Bonsel, G. (2015). Health systems’ responsiveness and reporting behaviour: Multilevel analysis of the influence of individual-level factors in 64 countries. Social Science & Medicine, 138, 152–160.

    Article  Google Scholar 

  • van Beuningen, J., van der Houwen, K., & Moonen, L. (2014). Measuring well-being, an analysis of different response scales (Discussion Paper, Statistics Netherlands). Retrieved July 20, 2015, http://www.cbs.nl-NR-rdonlyres-FF644A99-580C-4D7E-B214-BEE54A947D46-0-Measuringwellbeing.pdf

  • Veenhoven, R. (1984). Databook of happiness. Dordrecht: Reidel Publishing. ISBN 90-277-1793-1

    Google Scholar 

  • Veenhoven, R. (1993). Happiness in nations, subjective appreciation of life in 56 nations, 1946–1992. Studies in Social-Cultural Transformation, 2. Risbo, Erasmus University Rotterdam, Netherlands. Available at: https://personal.eur.nl/veenhoven/Pub1990s/93b-con.html

  • Veenhoven, R. (2011). World database of happiness, example of a focused ‘Findings Archive’ (RatSWD, Working Paper Series, Working Paper, 169). Retrieved January 29, 2016, http://www.ratswd.de/download/RatSWD_WP_2011/RatSWD_WP_169.pdf

  • Veenhoven, R. (2016a). Measures of happiness. World Database of happiness, Erasmus University Rotterdam. Available at: http://worlddatabaseofhappiness.eur.nl/hap_quer/hqi_fp.htm

  • Veenhoven, R. (2016b). Happiness in Nations. World Database of Happiness, Erasmus University Rotterdam. Available at: http://worlddatabaseofhappiness.eur.nl/hap_nat/nat_fp.php?mode=1

Download references

Author information

Authors and Affiliations

  1. Erasmus Happiness Economics Research Organisation, Erasmus University Rotterdam, Rotterdam, The Netherlands

    Tineke de Jonge, Ruut Veenhoven & Wim Kalmijn

Authors
  1. Tineke de Jonge
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Ruut Veenhoven
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Wim Kalmijn
    View author publications

    You can also search for this author in PubMed Google Scholar

Rights and permissions

Reprints and permissions

Copyright information

© 2017 Springer International Publishing AG

About this chapter

Cite this chapter

de Jonge, T., Veenhoven, R., Kalmijn, W. (2017). Diversity in Survey Items and the Comparability Problem. In: Diversity in Survey Questions on the Same Topic. Social Indicators Research Series, vol 68. Springer, Cham. https://doi.org/10.1007/978-3-319-53261-5_1

Download citation

  • DOI: https://doi.org/10.1007/978-3-319-53261-5_1

  • Published:

  • Publisher Name: Springer, Cham

  • Print ISBN: 978-3-319-53260-8

  • Online ISBN: 978-3-319-53261-5

  • eBook Packages: Social SciencesSocial Sciences (R0)

Publish with us

Policies and ethics

Search

Navigation