Assessing the effects of a mixed-mode design in a longitudinal household travel survey

Abstract

The German Mobility Panel (MOP) is a national household travel survey, which has been collecting data on travel behavior in Germany since 1994. One of the MOP’s central assets is its ability to provide time-series data on travel behavior. Thus, the comparability of survey results from different years is a major objective of the survey method used. Declining survey participation rates in the last decade in various socio-demographic groups resulted in the implementation of a mixed-mode design for the MOP in 2013, both for the sampling stage (landline and mobile phone recruitment) and the data collection stage (paper and web). In this study, we analyze whether the adaptations in the survey mode do indeed improve the results and, if so, why and to what degree. Ideally, the survey mode adaptions have increased the representativeness of the MOP. However, measurement biases due to the mixed-mode design are also conceivable. To decompose survey mode effects, we applied the propensity score weighting method. This method imputes the hypothetical responses participants would have given in different survey modes; disparities between actual responses and hypothetical responses under another mode are then traced back to the mixed-mode design. Our analysis indicates that trip-rate biases on shopping, leisure, and short trips are partly caused by the mixed-mode design; in contrast, quantities of time spent in the transportation system, trips made by car and public transportation, and commuting trips are hardly biased.

This is a preview of subscription content, access via your institution.

Fig. 1

References

  1. Agrawal, A.W., Granger-Bevan, S., Newmark, G.L., Nixon, H.: Comparing data quality and cost from three modes of on-board transit surveys. Transp. Policy 54, 70–79 (2017). https://doi.org/10.1016/j.tranpol.2016.06.010

    Article  Google Scholar 

  2. Anderson, M., Perrin, A.: Disabled Americans are less likely to use technology. http://www.pewresearch.org/fact-tank/2017/04/07/disabled-americans-are-less-likely-to-use-technology/ (2017). Accessed 15 May 2017

  3. Arn, B., Klug, S., Kołodziejski, J.: Evaluation of an adapted design in a multi-device online panel. Methods Data Anal. 9, 185–212 (2015). https://doi.org/10.12758/mda.2015.011

    Article  Google Scholar 

  4. Bayart, C., Bonnel, P.: The mixing of survey modes: application to Lyon web and face-to-face household travel survey. In: 12th World Conference in Transportation Research (2010)

  5. Bonnel, P., Bayart, C., Smith, B.: Workshop synthesis—comparing and combining survey modes. Transp. Res. Procedia 11, 108–117 (2015). https://doi.org/10.1016/j.trpro.2015.12.010

    Article  Google Scholar 

  6. Brambilla, D.J., McKinlay, S.M.: A comparison of responses to mailed questionnaires and telephone interviews in a mixed mode health survey. Am. J. Epidemiol. 126(5), 962–971 (1987)

    Article  Google Scholar 

  7. Cameron, A.C., Trivedi, P.K.: Microeconometrics: Methods and applications. Cambridge University Press, New York (2005)

    Book  Google Scholar 

  8. Chlond, B., Streit, T., Abler, G., Vortisch, P.: Balancing innovation and Continuity – Experiences with survey design adaptations of the German Mobility Panel. Transp. Res. Procedia 11, 43–59 (2015). https://doi.org/10.1016/j.trpro.2015.12.005

    Article  Google Scholar 

  9. Christensen, L.: Possible explanations for an increasing share of no-trip respondents. In: Stopher, P., Stecher, Ch. (eds.) Travel Survey Methods—Quality and Future Directions, pp. 303–316. Elsevier, Oxford (2006)

    Google Scholar 

  10. Christensen, L.: The role of web interviews as part of a national travel survey. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 115–153. Emerald Group Publishing (2013)

  11. De Leeuw, E.D.: To mix or not to mix data collection modes in surveys. J. Off. Stat. 21(2), 233–255 (2005)

    Google Scholar 

  12. Dillman, D.A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Messer, B.L.: Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Soc. Sci. Res. 38, 1–18 (2009). https://doi.org/10.1016/j.ssresearch.2008.03.007

    Article  Google Scholar 

  13. Franken, V., Lenz, B.: Influence of mobility information services on travel behavior. In: Barlow, M., Tietze, W., Claval, P., Gradus, Y., Park, S.O., van der Wusten, H., Miller, H.J. (eds.) Societies and Cities in the Age of Instant Access, vol. 88, pp. 167–178. Springer, Dordrecht (2007)

    Google Scholar 

  14. Gabler, S., Ayhan, Ö.: Gewichtungen bei Erhebungen im Festnetz und über Mobilfunkt: Ein Dual Frame Ansatz. In: Gabler, S. (ed.) Mobilfunktelefonie. Eine Herausforderung für die Umfrageforschung, pp. 39–46. ZUMA, Mannheim (2007)

    Google Scholar 

  15. Hu, S.S., Balluz, L., Battaglia, M.P., Frankel, M.R.: Improving public health surveillance using a dual-frame survey of landline and cell phone numbers. Am. J. Epidemiol. 173(6), 703–711 (2011). https://doi.org/10.1093/aje/kwq442

    Article  Google Scholar 

  16. Kagerbauer, M., Manz, W., Zumkeller, D.: Analysis of PAPI, CATI, and CAWI methods for a multiday household travel survey. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 289–304. Emerald Group Publishing (2013)

  17. Kemmick Pintor, J.B., McAlpine, D., Beebe, T.J., Johnson, P.J.: Propensity score matching to measure the effect of survey mode on reports of racial and ethnic discrimination in health care. Med. Care 53(5), 471–476 (2015). https://doi.org/10.1097/MLR.0000000000000351

    Article  Google Scholar 

  18. Klausch, T.: Lecture 1: Decomposing total mode effects in mixed mode data. Lecture Notes of the GESIS Summer Course “Mixed Mode and Mixed Device Surveys”. Cologne (2015)

  19. Klausch, T., Hox, J., Schouten, B.: Selection error in single- and mixed mode surveys of the Dutch general population. J. R. Stat. Soc. A 178(4), 945–961 (2015). https://doi.org/10.1111/rssa.12102

    Article  Google Scholar 

  20. Klausch, T., Hox, J.J., Schouten, B.: Measurement effects of survey mode on the equivalence of attitudinal rating scale questions. Sociol. Methods Res. 42(3), 227–263 (2013). https://doi.org/10.1177/0049124113500480

    Article  Google Scholar 

  21. Kolenikov, S., Kennedy, C.: Evaluating three approaches to statistically adjust for mode effects. J. Surv. Stat. Methodol. 2(2), 126–158 (2014). https://doi.org/10.1093/jssam/smu004

    Article  Google Scholar 

  22. Link, M.W., Battaglia, M.P., Frankel, M.R., Osborn, L., Mokdad, A.H.: Reaching the U.S. cell phone generation—comparison of cell phone survey results with an ongoing landline telephone survey. Public Opin. Q. 71(5), 814–839 (2007). https://doi.org/10.1093/poq/nfm051

    Article  Google Scholar 

  23. Rosenbaum, P.R.: Model-based direct adjustment. J. Am. Stat. Assoc. 82(398), 387 (1987). https://doi.org/10.2307/2289440

    Article  Google Scholar 

  24. Rosenbaum, P.R., Rubin, D.: The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), 41–55 (1983). https://doi.org/10.1093/biomet/70.1.41

    Article  Google Scholar 

  25. Segert, A.: Informationspraktiken. Technikaffinität und Alltagsmobilität, Wien (2012)

    Google Scholar 

  26. Streit, T., Chlond, B., Weiss, C., Vortisch, P.: Deutsches Mobilitätspanel (MOP)—Wissenschaftliche Begleitung und Auswertungen Bericht 2013/2014. Alltagsmobilität und Fahrleistung, Karlsruhe (2015)

    Google Scholar 

  27. Thériault, M., Lee-Gosselin, M., Alexandre, L., Théberge, F., Dieumegarde, L.: Web versus pencil-and-paper surveys of weekly mobility: conviviality, technical and privacy issues. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 225–246. Emerald Group Publishing (2013)

  28. Vannieuwenhuyze, J.T.A., Loosveldt, G.: Evaluating relative mode effects in mixed-mode surveys. Three methods to disentangle selection and measurement effects. Sociol. Methods Res. 42(1), 82–104 (2013). https://doi.org/10.1177/0049124112464868

    Article  Google Scholar 

  29. Weiss, C., Chlond, B., Hilgert, T., Vortisch, P.: Deutsches Mobilitätspanel (MOP)—Wissenschaftliche Begleitung und Auswertungen Bericht 2014/2015. Alltagsmobilität und Fahrleistung, Karlsruhe (2016)

    Google Scholar 

  30. Wirtz, M., Streit, T., Chlond, B., Vortisch, P.: On new measures for detection of data quality risks in mobility panel surveys. Transp. Res. Rec. J. Transp. Res. Board 2354, 19–28 (2013). https://doi.org/10.3141/2354-03

    Article  Google Scholar 

  31. Wolf, J., Wilhelm, J., Casa, J., Sen, S.: Case study: multiple data collection methods and the NY/NJ/CT regional travel survey. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 71–90. Emerald Group Publishing (2013)

  32. Xing, Y., Handy, S.L.: Online Versus phone surveys: comparison of results for bicycling survey. In: Transportation Research Board (ed.) TRB 91st Annual Meeting Compendium of Papers (2012)

  33. Zumkeller, D., Chlond, B.: Dynamics of change: fifteen-year german mobility panel. In: Transportation Research Board (ed.) TRB 88th Annual Meeting Compendium of Papers (2009)

Download references

Acknowledgements

The authors would like to acknowledge the valuable comments provided by three anonymous referees and by the associate editor (Patricia L. Mokhtarian). This paper presents analyses of the German Mobility Panel funded by the German Federal Ministry of Transport and Digital Infrastructure. An earlier version of the paper was presented at the 96th Transportation Research Board Annual Meeting.

Author information

Affiliations

Authors

Corresponding author

Correspondence to Christine Eisenmann.

Rights and permissions

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Eisenmann, C., Chlond, B., Minster, C. et al. Assessing the effects of a mixed-mode design in a longitudinal household travel survey. Transportation 46, 1737–1753 (2019). https://doi.org/10.1007/s11116-018-9879-2

Download citation

Keywords

  • Germany Mobility Panel
  • Mixed-mode survey
  • National household travel survey
  • Propensity score weighting