Advertisement

Assessing the effects of a mixed-mode design in a longitudinal household travel survey

  • Christine EisenmannEmail author
  • Bastian Chlond
  • Clotilde Minster
  • Christian Jödden
  • Peter Vortisch
Article
  • 132 Downloads

Abstract

The German Mobility Panel (MOP) is a national household travel survey, which has been collecting data on travel behavior in Germany since 1994. One of the MOP’s central assets is its ability to provide time-series data on travel behavior. Thus, the comparability of survey results from different years is a major objective of the survey method used. Declining survey participation rates in the last decade in various socio-demographic groups resulted in the implementation of a mixed-mode design for the MOP in 2013, both for the sampling stage (landline and mobile phone recruitment) and the data collection stage (paper and web). In this study, we analyze whether the adaptations in the survey mode do indeed improve the results and, if so, why and to what degree. Ideally, the survey mode adaptions have increased the representativeness of the MOP. However, measurement biases due to the mixed-mode design are also conceivable. To decompose survey mode effects, we applied the propensity score weighting method. This method imputes the hypothetical responses participants would have given in different survey modes; disparities between actual responses and hypothetical responses under another mode are then traced back to the mixed-mode design. Our analysis indicates that trip-rate biases on shopping, leisure, and short trips are partly caused by the mixed-mode design; in contrast, quantities of time spent in the transportation system, trips made by car and public transportation, and commuting trips are hardly biased.

Keywords

Germany Mobility Panel Mixed-mode survey National household travel survey Propensity score weighting 

Notes

Acknowledgements

The authors would like to acknowledge the valuable comments provided by three anonymous referees and by the associate editor (Patricia L. Mokhtarian). This paper presents analyses of the German Mobility Panel funded by the German Federal Ministry of Transport and Digital Infrastructure. An earlier version of the paper was presented at the 96th Transportation Research Board Annual Meeting.

References

  1. Agrawal, A.W., Granger-Bevan, S., Newmark, G.L., Nixon, H.: Comparing data quality and cost from three modes of on-board transit surveys. Transp. Policy 54, 70–79 (2017).  https://doi.org/10.1016/j.tranpol.2016.06.010 CrossRefGoogle Scholar
  2. Anderson, M., Perrin, A.: Disabled Americans are less likely to use technology. http://www.pewresearch.org/fact-tank/2017/04/07/disabled-americans-are-less-likely-to-use-technology/ (2017). Accessed 15 May 2017
  3. Arn, B., Klug, S., Kołodziejski, J.: Evaluation of an adapted design in a multi-device online panel. Methods Data Anal. 9, 185–212 (2015).  https://doi.org/10.12758/mda.2015.011 Google Scholar
  4. Bayart, C., Bonnel, P.: The mixing of survey modes: application to Lyon web and face-to-face household travel survey. In: 12th World Conference in Transportation Research (2010)Google Scholar
  5. Bonnel, P., Bayart, C., Smith, B.: Workshop synthesis—comparing and combining survey modes. Transp. Res. Procedia 11, 108–117 (2015).  https://doi.org/10.1016/j.trpro.2015.12.010 CrossRefGoogle Scholar
  6. Brambilla, D.J., McKinlay, S.M.: A comparison of responses to mailed questionnaires and telephone interviews in a mixed mode health survey. Am. J. Epidemiol. 126(5), 962–971 (1987)CrossRefGoogle Scholar
  7. Cameron, A.C., Trivedi, P.K.: Microeconometrics: Methods and applications. Cambridge University Press, New York (2005)CrossRefGoogle Scholar
  8. Chlond, B., Streit, T., Abler, G., Vortisch, P.: Balancing innovation and Continuity – Experiences with survey design adaptations of the German Mobility Panel. Transp. Res. Procedia 11, 43–59 (2015).  https://doi.org/10.1016/j.trpro.2015.12.005 CrossRefGoogle Scholar
  9. Christensen, L.: Possible explanations for an increasing share of no-trip respondents. In: Stopher, P., Stecher, Ch. (eds.) Travel Survey Methods—Quality and Future Directions, pp. 303–316. Elsevier, Oxford (2006)CrossRefGoogle Scholar
  10. Christensen, L.: The role of web interviews as part of a national travel survey. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 115–153. Emerald Group Publishing (2013)Google Scholar
  11. De Leeuw, E.D.: To mix or not to mix data collection modes in surveys. J. Off. Stat. 21(2), 233–255 (2005)Google Scholar
  12. Dillman, D.A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Messer, B.L.: Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Soc. Sci. Res. 38, 1–18 (2009).  https://doi.org/10.1016/j.ssresearch.2008.03.007 CrossRefGoogle Scholar
  13. Franken, V., Lenz, B.: Influence of mobility information services on travel behavior. In: Barlow, M., Tietze, W., Claval, P., Gradus, Y., Park, S.O., van der Wusten, H., Miller, H.J. (eds.) Societies and Cities in the Age of Instant Access, vol. 88, pp. 167–178. Springer, Dordrecht (2007)CrossRefGoogle Scholar
  14. Gabler, S., Ayhan, Ö.: Gewichtungen bei Erhebungen im Festnetz und über Mobilfunkt: Ein Dual Frame Ansatz. In: Gabler, S. (ed.) Mobilfunktelefonie. Eine Herausforderung für die Umfrageforschung, pp. 39–46. ZUMA, Mannheim (2007)Google Scholar
  15. Hu, S.S., Balluz, L., Battaglia, M.P., Frankel, M.R.: Improving public health surveillance using a dual-frame survey of landline and cell phone numbers. Am. J. Epidemiol. 173(6), 703–711 (2011).  https://doi.org/10.1093/aje/kwq442 CrossRefGoogle Scholar
  16. Kagerbauer, M., Manz, W., Zumkeller, D.: Analysis of PAPI, CATI, and CAWI methods for a multiday household travel survey. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 289–304. Emerald Group Publishing (2013)Google Scholar
  17. Kemmick Pintor, J.B., McAlpine, D., Beebe, T.J., Johnson, P.J.: Propensity score matching to measure the effect of survey mode on reports of racial and ethnic discrimination in health care. Med. Care 53(5), 471–476 (2015).  https://doi.org/10.1097/MLR.0000000000000351 CrossRefGoogle Scholar
  18. Klausch, T.: Lecture 1: Decomposing total mode effects in mixed mode data. Lecture Notes of the GESIS Summer Course “Mixed Mode and Mixed Device Surveys”. Cologne (2015)Google Scholar
  19. Klausch, T., Hox, J., Schouten, B.: Selection error in single- and mixed mode surveys of the Dutch general population. J. R. Stat. Soc. A 178(4), 945–961 (2015).  https://doi.org/10.1111/rssa.12102 CrossRefGoogle Scholar
  20. Klausch, T., Hox, J.J., Schouten, B.: Measurement effects of survey mode on the equivalence of attitudinal rating scale questions. Sociol. Methods Res. 42(3), 227–263 (2013).  https://doi.org/10.1177/0049124113500480 CrossRefGoogle Scholar
  21. Kolenikov, S., Kennedy, C.: Evaluating three approaches to statistically adjust for mode effects. J. Surv. Stat. Methodol. 2(2), 126–158 (2014).  https://doi.org/10.1093/jssam/smu004 CrossRefGoogle Scholar
  22. Link, M.W., Battaglia, M.P., Frankel, M.R., Osborn, L., Mokdad, A.H.: Reaching the U.S. cell phone generation—comparison of cell phone survey results with an ongoing landline telephone survey. Public Opin. Q. 71(5), 814–839 (2007).  https://doi.org/10.1093/poq/nfm051 CrossRefGoogle Scholar
  23. Rosenbaum, P.R.: Model-based direct adjustment. J. Am. Stat. Assoc. 82(398), 387 (1987).  https://doi.org/10.2307/2289440 CrossRefGoogle Scholar
  24. Rosenbaum, P.R., Rubin, D.: The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), 41–55 (1983).  https://doi.org/10.1093/biomet/70.1.41 CrossRefGoogle Scholar
  25. Segert, A.: Informationspraktiken. Technikaffinität und Alltagsmobilität, Wien (2012)Google Scholar
  26. Streit, T., Chlond, B., Weiss, C., Vortisch, P.: Deutsches Mobilitätspanel (MOP)—Wissenschaftliche Begleitung und Auswertungen Bericht 2013/2014. Alltagsmobilität und Fahrleistung, Karlsruhe (2015)Google Scholar
  27. Thériault, M., Lee-Gosselin, M., Alexandre, L., Théberge, F., Dieumegarde, L.: Web versus pencil-and-paper surveys of weekly mobility: conviviality, technical and privacy issues. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 225–246. Emerald Group Publishing (2013)Google Scholar
  28. Vannieuwenhuyze, J.T.A., Loosveldt, G.: Evaluating relative mode effects in mixed-mode surveys. Three methods to disentangle selection and measurement effects. Sociol. Methods Res. 42(1), 82–104 (2013).  https://doi.org/10.1177/0049124112464868 CrossRefGoogle Scholar
  29. Weiss, C., Chlond, B., Hilgert, T., Vortisch, P.: Deutsches Mobilitätspanel (MOP)—Wissenschaftliche Begleitung und Auswertungen Bericht 2014/2015. Alltagsmobilität und Fahrleistung, Karlsruhe (2016)Google Scholar
  30. Wirtz, M., Streit, T., Chlond, B., Vortisch, P.: On new measures for detection of data quality risks in mobility panel surveys. Transp. Res. Rec. J. Transp. Res. Board 2354, 19–28 (2013).  https://doi.org/10.3141/2354-03 CrossRefGoogle Scholar
  31. Wolf, J., Wilhelm, J., Casa, J., Sen, S.: Case study: multiple data collection methods and the NY/NJ/CT regional travel survey. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 71–90. Emerald Group Publishing (2013)Google Scholar
  32. Xing, Y., Handy, S.L.: Online Versus phone surveys: comparison of results for bicycling survey. In: Transportation Research Board (ed.) TRB 91st Annual Meeting Compendium of Papers (2012)Google Scholar
  33. Zumkeller, D., Chlond, B.: Dynamics of change: fifteen-year german mobility panel. In: Transportation Research Board (ed.) TRB 88th Annual Meeting Compendium of Papers (2009)Google Scholar

Copyright information

© Springer Science+Business Media, LLC, part of Springer Nature 2018

Authors and Affiliations

  1. 1.Institute for Transport StudiesKarlsruhe Institute of Technology (KIT)KarlsruheGermany
  2. 2.KANTAR TNS GermanyMunichGermany

Personalised recommendations