Costs and Errors in Fixed and Mobile Phone Surveys

Chapter

Abstract

Due to low costs, speed, simplicity, interviewer assistance, and easy monitoring telephone interviewing had been the preferred mode of many survey practitioners for decades. However, technological developments related to information society, in particular the increasing rate of mobile-only individuals and households, are rapidly changing the survey research environment. In the first part of the chapter methodological issues of data collection by phone are delineated. After a brief history of phone surveys recent telephone use trends and their implications on survey coverage and sampling are outlined. Next, nonresponse in phone surveys is discussed. The section ends with an illustration of challenges posed by incorporating mobile phone in survey research. In the second part, phone surveys are discussed in the context of mixed modes, in particular their potential to improve coverage and response rates. Furthermore, data quality and cost issues are treated. Finally, the section introduces dual frame sampling of fixed and mobile numbers, a special type of mixed mode surveys that was developed to resolve the phone coverage problem. In the third part, the chapter deals with the optimization of telephone surveys according to costs and errors. An analytical solution for dual frame surveys and a more general postsurvey evaluation of different modes are presented. In conclusion, we recapitulate key issues of phone surveys and indicate future trends with guidelines for further research.

Keywords

Dual frameDual frame Mobile phone surveys  Mobile-only  Mean squarre error  Mixed mode  Nonresponse  Optimization  Phone surveys  Phone Coverage  Survey Response  Survey error  Survey costs  Survey sampling  

References

  1. AAPOR Cell Phone Task Force (2010) New Considerations for Survey Researchers When Planning and Conducting RDD Telephone Surveys in the U.S. With Respondents Reached via Cell Phone Numbers. American Association for Public Opinion Research, from http://aapor.org/AM/Template.cfm?Section=Cell_Phone_Task_Force&Template=/CM/ContentDisplay.cfm&ContentID=2818.
  2. Belak, E. (2007). Vpliv uporabe mobilnih telefonov na anketno zbiranje podatkov. Magistrsko delo. Ljubljana: FDV.Google Scholar
  3. Berzelak, N.,Vehovar, V. & Lozar Manfreda, K. (2008). What can we achieve with 5 Euros? optimization of survey data quality using mixed-mode approaches. Paper presented at the Applied Statistics Conference, RibnoGoogle Scholar
  4. Biemer, P. P. (2010). Total survey error: Design implementation and evaluation. Public Opinion Quarterly, 74(5), 817–848.CrossRefGoogle Scholar
  5. Biemer, P. P., & Lyberg, L. E. (2003). Introduction to survey quality. Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  6. Blumberg, S. J., & Luke, J. V. (2009). Wireless substitution: Early release of estimates from the National Health Interview Survey, July-December 2008. National Center for Health Statistics, from http://www.cdc.gov/nchs/nhis.htm.
  7. Bosnjak, M., Neubarth, W., Couper, M. P., Bandilla, W., & Kaczmirek, L. (2008). Prenotification in web-based access panel surveys: The influence of mobile text messaging versus e-mail on response rates and sample composition. Social Science Computer Review, 26(2), 213–223.CrossRefGoogle Scholar
  8. Brick, J. M., Dipko, S., Presser, S., Tucker, C., & Yangyang, Y. (2006). Nonresponse bias in a dual frame sample of cell and landline numbers. Public opinion Quarterly, 70(5, Special Issue), 780–793.Google Scholar
  9. Brick, J. M., Edwards, W. S., & Sunghee, L. (2007). Sampling telephone numbers and adults, interview length, and weighting in the California health interview survey cell phone pilot study. Public Opinion Quarterly, 71(5), 793–813.CrossRefGoogle Scholar
  10. Brick, M. J., Cervantes, I. F., Lee, S., & Norman, G. (2011). Nonsampling Errors in Dual Frame Telephone Surveys. Survey Methodology 37 (1), 1–12. Google Scholar
  11. Callegaro, M., Steeh, C., Buskirk, T. D., Vehovar, V., Kuusela, V., & Piekarski, L. (2007). Fitting disposition codes to mobile phone surveys: Experiences from studies in Finland, Slovenia and the USA. Journal of the Royal Statistical Society: Series A (Statistics in Society), 170(3), 647–670.CrossRefGoogle Scholar
  12. Chang, L. C., & Krosnick, J. A. (2009). National surveys Via Rdd telephone interviewing versus the internet. Comparing sample representativeness and response quality. Public Opinion Quarterly, 73(4), 641–678.CrossRefGoogle Scholar
  13. Cochran, W. G. (1978). Sampling techniques (3rd ed.). New York, NY: Wiley.Google Scholar
  14. de Leeuw, E. D. (1992). Data quality in mail, telephone, and face-to-face surveys. Amsterdam: TT-Publicaties.Google Scholar
  15. de Leeuw, E. D. (2005). To mix or not to mix data collection modes in surveys. Journal of Official Statistics, 21(2), 233–255.Google Scholar
  16. Deming, W. E. (1950). Some theory of sampling. New York, NY: Wiley.Google Scholar
  17. Deming, W. E. (1953). On a probability mechanism to attain an economic balance between the resultant error of response and the bias of nonresponse. Journal of the American Statistical Association, 48(264), 743–772.CrossRefGoogle Scholar
  18. Dillman, D. A., Smyth, J. D., & Christian, C. M. (2009). Internet, mail, and mixed-mode surveys: The tailored design method (3rd ed.). Hoboken: Wiley.Google Scholar
  19. Dolnicar, S., Laesser, C., & Matus, K. (2009). Online versus paper: Format effects in tourism surveys. Journal of Travel Research, 47(3), 295–316.CrossRefGoogle Scholar
  20. Elliott, M., Little, R. J. A., & Lewitsky, S. (2000). Subsampling callbacks to improve survey efficiency. Journal of the American Statistical Association, 95, 730–738.CrossRefGoogle Scholar
  21. European Commission (2008a). Special Eurobarometer 293. E-communication Household Survey. TNS Opinion and social, from http://ec.europa.eu/public_opinion/archives/ebs/ebs_293_full_en.pdf
  22. European Commission (2008b). Flash Eurobarometer 251. Public attitudes and perceptions in the Euro area. Gallup Europe, Brussels [Producer]. GESIS, Cologne [Publisher]. ZA4743, data set version 2.0.0. Description: http://info1.gesis.org/dbksearch13/sdesc2.asp?no=4743&db=e&doi=10.4232/1.10007 (July 29 2010).
  23. Fuchs, M. (2007). Mobile web surveys: A preliminary discussion of methodological implications. In F. G. Conrad & M. F. Schober (Eds.), Envisioning the survey interview of the future (pp. 77–94). Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  24. Gabler, S., & Häder, S. (1999). Erfahrungen beim Aufbau eines Auswahlrahmens für Telefonstichproben in Deutschland. ZUMA-Nachrichten Heft, 44, 45–61.Google Scholar
  25. Gallup Europe (2009, February). Presentation Technical workshop. Paper presented at the Eurobarometer Technical Workshop, Budapest, Hungary.Google Scholar
  26. Groves, R. M. (1989). Survey errors and survey costs. Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  27. Groves, R. M. (2005, March). Past, present, and future. Washington, DC: Paper presented at the Total Survey Error Workshop.Google Scholar
  28. Groves, R. M. (2006, April). Survey budgets, cost models, and responsive survey designs. Washington, DC: Paper presented at the Survey Cost Workshop.Google Scholar
  29. Groves, R. M., & Kahn, R. L. (1979). Surveys by telephone: A national comparison with personal interviews. New York: Academic Press.Google Scholar
  30. Groves, R. M., & Lepkowski, J. M. (1982). Alternative dual frame mixed mode surveys designs. Proceedings of the Survey Research Methods Section, American Statistical Association.Google Scholar
  31. Groves, R. M., & Peytcheva, E. (2008). The impact of nonresponse rates on nonresponse bias: a meta-analysis. Public Opinion Quarterly, 72(2), 167–189.CrossRefGoogle Scholar
  32. Groves, R. M., Fowler, F. J., Couper, M. P., Lepkowski, J. M., Singer, E., & Tourangeau, R. (2004). Survey methodology. Hoboken, NJ: Wiley.Google Scholar
  33. Hansen, M. H., Hurwitz, W. N., & Madow, W. G. (1953). Sample survey methods and theory. New York, NY: Wiley.Google Scholar
  34. Harter, R., Mach, T., Wolken, J., & Chapline, J. (2007, June). Determining subsampling rates for nonrespondents. Paper presented at the Third International Conference on Establishment Surveys, Montréal, Canada.Google Scholar
  35. Hartley, H. O. (1962). Multiple frame surveys. Proceedings of the Social Statistics Section, American Statistical Association.Google Scholar
  36. Karr, A. F., & Last, M. (2006). Survey Costs: Workshop Report and White Paper.Google Scholar
  37. Keeter, S., Kennedy, C., Clark, A., Tompson, T., & Mokrzycki, M. (2007). What’s missing from national landline RDD surveys? Public Opinion Quarterly, 71(5), 772–792.CrossRefGoogle Scholar
  38. Kennedy, C. (2007). Evaluating the effects of screening for telephone service in dual frame RDD surveys. Public opinion Quarterly, 71(5), 750–771.Google Scholar
  39. Kish, L. (1965). Survey sampling. New York, NY: Wiley.Google Scholar
  40. Kuusela, V., & Simpanen, M. (2002). Effects of mobile phones on phone survey practices and results. Copenhagen: ICIS Conference.Google Scholar
  41. Kuusela, V., Callegaro, M., & Vehovar, V. (2007). The influence of mobile telephones on telephone surveys. In J. M. Lepkowski, C. Tucker, J. M. Brick, E. D. de Leeuw, L. Japec, P. J. Javrakas, M. W. Link, & R. L. Sangster (Eds.), Advances in telephone survey methodology (pp. 87–112). Hoboken, NJ: Wiley.CrossRefGoogle Scholar
  42. Lavrakas, P. J., Shuttles, C. D., Steeh, C., & Fienberg, H. (2007). The state of surveying cell phone numbers in the United States: 2007 and beyond. Public Opinion Quarterly, 71(5), 840–854.CrossRefGoogle Scholar
  43. Leah, C., Keeter, S., Purcell, K., & Smith, A. (2010). Assesing the cell phone challenge. PEW research centre, from http://pewresearch.org/pubs/1601/assessing-cell-phone-challenge-in-public-opinion-surveys
  44. Lepkowski, J. M. & Groves, R. M. (1984). The impact of bias on dual frame survey design. AMSE model. Proceedings of the Section on Survey Research, American Statistical Association: http://www.amstat.org/sections/SRMS/ Proceedings/papers/1984_051.pdf
  45. Lepkowski, J. M., & Groves, R. M. (1986). A mean squarred error model for dual frame, mixed mode survey design. Journal of the American Statistical Association, 396(81),  .Google Scholar
  46. Lessler, J. T., & Kalsbeek, W. D. (1992). Nonsampling error in surveys. New York, NY: Wiley.Google Scholar
  47. Linacre, S. J., & Trewin, D. J. (1993). Total survey design—application to a collection of the construction industry. Journal of Official Statistics, 9(3), 611–621.Google Scholar
  48. Link, M. W., Battaglia, M. P., Frankel, M. R., Osborn, L., & Mokdad, A. H. (2007). Reaching the U.S. cell phone generation: Comparison of cell phone survey results with an ongoing landline telephone survey. Public Opinion Quarterly, 71(5), 814–839.CrossRefGoogle Scholar
  49. Lyberg, L. E., Japec, L., & Biemer, P. P. (1998). Quality improvement in surveys—A process perspective. Paper presented at the Survey Research Methods Section.Google Scholar
  50. Lynn, P., & Elliot, D. (2000). The british crime survey: A review of methodology. London, UK: National Centre for Social Research.Google Scholar
  51. McCutcheon, A. L. (2009). Mode or Mensch? Respondent sensitivity to mode changes in data collection methods. The 1st Alec Gallup Future of Survey Research Forum: Incorporating Mobile Phones in Social and Policy-oriented Surveys. Brussels.Google Scholar
  52. Nicolaas, G., Lynn, P. & Lound, C. (2000). Random digit dialing in the UK: viability of the sampling method revisited. Paper presented at the International Conference on Social Science Methodology, Cologne.Google Scholar
  53. Pew Research Center (2008, December 18). Calling cell phones in ‘08 pre-election polls. News Release. Retrieved February 20, 2009, from http://people-press.org/reports/pdf/cell-phone-commentary.pdf
  54. Singer, E. (2006). Nonresponse bias in household surveys. Public Opinion Quarterly, 70(5), 637–645.CrossRefGoogle Scholar
  55. Statistical Office of the Republic of Slovenia. (2008). Community survey on ICT usage in households and by individuals—2008 [Survey data for Slovenia]. Ljubljana, Slovenia: Statistical Office of the Republic of Slovenia.Google Scholar
  56. Steeh, C., & Piekarski, L. (2008). “Accommodating new technologies: Mobile and VoIP communication.” In J. M. Lepkowski, C. Tucker, J. M. Brick, E. de Leeuw, L. Japec, P. J. Lavrakas, M. W. Link, & R. L. Sangster (Eds.), Advances in telephone survey methodology (pp. 423–448). New York, NY: Wiley.Google Scholar
  57. Steeh, C., Buskirk, T. D., & Callegaro, M. (2007). Using text messages in U.S. mobile phone surveys. Field Methods, 19(1), 59–75.CrossRefGoogle Scholar
  58. Tortora, R. D. (2004). Response trends in a National Random Digit Dial telephone survey. Advances in Methodology and Statistics, 1(1), 61–84.Google Scholar
  59. Traugot, M. V., Groves, R. M., & Lepkowski, J. M. (1987). Dual frames to reduce nonresponse. Public Opinion Quarterly, 51(4), 522–539.CrossRefGoogle Scholar
  60. Tucker, C., Brick, M. J., & Meekin, B. (2007). Household telephone service and usage patterns in the United States in 2004. Public Opinion Quarterly, 71(1), 3–22.CrossRefGoogle Scholar
  61. Vehovar, V., Belak, E., Batagelj, Z., & Čikić, S. (2004). Mobile phone surveys: The Slovenian case study. Metodološki zvezki (Advances in Methodology and Statistics), 1(1), 1–19.Google Scholar
  62. Vehovar, V., & Lozar Manfreda, K. (2008). Overview: Online surveys. In N. G. Fielding, R. M. Lee, & G. Blank (Eds.), The handbook of online research methods (pp. 177–194). Thousand Oaks, CA: SAGE Publications.Google Scholar
  63. Vehovar, V., & Slavec, A. (2011). Costs Error Optimization for Cell-Landline Dual Frame Surveys. Paper presented on the 66th conference of the American Association for Public Opinion Research.Google Scholar
  64. Vehovar, V., Lozar Manfreda, K., & Batagelj, Z. (2001). Sensitivity of electronic commerce measurement to the survey instrument. International Journal of Electronic Commerce, 6(1), 31–52.Google Scholar
  65. Vehovar, V., Berzelak, J., & Lozar Manfreda, K. (2010). Mobile phones in an environment of competing survey modes: Applying metric for evaluation of costs and errors. Social Sciences Computer Review, 28(3), 303–318.CrossRefGoogle Scholar
  66. Virtanen, V., Sirkiä, T., & Jokiranta, V. (2007). Reducing nonresponse by SMS reminders in mail surveys. Social Science Computer Review, 25(3), 384–395.CrossRefGoogle Scholar
  67. Waksberg, J. (1978). Sampling method for random digit dialing. Journal of American Statistical Association, 73(361), 40–46.CrossRefGoogle Scholar
  68. Weisberg, H. F. (2005). The total survey error approach: A guide to the new science of survey research. Chicago: University of Chicago Press.Google Scholar

Suggested Readings

  1. AAPOR Cell Phone Task Force (2010). New Considerations for Survey Researchers When Planning and Conducting RDD Telephone Surveys in the U.S. With Respondents Reached via Cell Phone Numbers. American Association for Public Opinion Research.Google Scholar
  2. Groves, R. M., Biemer, P. P., Lyberg, L. E., Massey, J. T., Nicholls, W. L, I. I., & Waksberg, J. (2001). Telephone survey methodology. Hoboken, NY: Wiley.Google Scholar
  3. Lepkowski, J. M., Tucker, C., Brick, J. M., de Leeuw, E. D., Japec, L., Lavrakas, P. J., et al. (2008). Advances in telephone survey methodology. Hoboken, NY: Wiley.Google Scholar

Copyright information

© Springer Science+Business Media New York 2012

Authors and Affiliations

  1. 1.University of Ljubljana Faculty of Social SciencesLjubljanaSlovenija

Personalised recommendations