Abstract
The German Mobility Panel (MOP) is a national household travel survey, which has been collecting data on travel behavior in Germany since 1994. One of the MOP’s central assets is its ability to provide time-series data on travel behavior. Thus, the comparability of survey results from different years is a major objective of the survey method used. Declining survey participation rates in the last decade in various socio-demographic groups resulted in the implementation of a mixed-mode design for the MOP in 2013, both for the sampling stage (landline and mobile phone recruitment) and the data collection stage (paper and web). In this study, we analyze whether the adaptations in the survey mode do indeed improve the results and, if so, why and to what degree. Ideally, the survey mode adaptions have increased the representativeness of the MOP. However, measurement biases due to the mixed-mode design are also conceivable. To decompose survey mode effects, we applied the propensity score weighting method. This method imputes the hypothetical responses participants would have given in different survey modes; disparities between actual responses and hypothetical responses under another mode are then traced back to the mixed-mode design. Our analysis indicates that trip-rate biases on shopping, leisure, and short trips are partly caused by the mixed-mode design; in contrast, quantities of time spent in the transportation system, trips made by car and public transportation, and commuting trips are hardly biased.
Similar content being viewed by others
References
Agrawal, A.W., Granger-Bevan, S., Newmark, G.L., Nixon, H.: Comparing data quality and cost from three modes of on-board transit surveys. Transp. Policy 54, 70–79 (2017). https://doi.org/10.1016/j.tranpol.2016.06.010
Anderson, M., Perrin, A.: Disabled Americans are less likely to use technology. http://www.pewresearch.org/fact-tank/2017/04/07/disabled-americans-are-less-likely-to-use-technology/ (2017). Accessed 15 May 2017
Arn, B., Klug, S., Kołodziejski, J.: Evaluation of an adapted design in a multi-device online panel. Methods Data Anal. 9, 185–212 (2015). https://doi.org/10.12758/mda.2015.011
Bayart, C., Bonnel, P.: The mixing of survey modes: application to Lyon web and face-to-face household travel survey. In: 12th World Conference in Transportation Research (2010)
Bonnel, P., Bayart, C., Smith, B.: Workshop synthesis—comparing and combining survey modes. Transp. Res. Procedia 11, 108–117 (2015). https://doi.org/10.1016/j.trpro.2015.12.010
Brambilla, D.J., McKinlay, S.M.: A comparison of responses to mailed questionnaires and telephone interviews in a mixed mode health survey. Am. J. Epidemiol. 126(5), 962–971 (1987)
Cameron, A.C., Trivedi, P.K.: Microeconometrics: Methods and applications. Cambridge University Press, New York (2005)
Chlond, B., Streit, T., Abler, G., Vortisch, P.: Balancing innovation and Continuity – Experiences with survey design adaptations of the German Mobility Panel. Transp. Res. Procedia 11, 43–59 (2015). https://doi.org/10.1016/j.trpro.2015.12.005
Christensen, L.: Possible explanations for an increasing share of no-trip respondents. In: Stopher, P., Stecher, Ch. (eds.) Travel Survey Methods—Quality and Future Directions, pp. 303–316. Elsevier, Oxford (2006)
Christensen, L.: The role of web interviews as part of a national travel survey. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 115–153. Emerald Group Publishing (2013)
De Leeuw, E.D.: To mix or not to mix data collection modes in surveys. J. Off. Stat. 21(2), 233–255 (2005)
Dillman, D.A., Phelps, G., Tortora, R., Swift, K., Kohrell, J., Berck, J., Messer, B.L.: Response rate and measurement differences in mixed-mode surveys using mail, telephone, interactive voice response (IVR) and the Internet. Soc. Sci. Res. 38, 1–18 (2009). https://doi.org/10.1016/j.ssresearch.2008.03.007
Franken, V., Lenz, B.: Influence of mobility information services on travel behavior. In: Barlow, M., Tietze, W., Claval, P., Gradus, Y., Park, S.O., van der Wusten, H., Miller, H.J. (eds.) Societies and Cities in the Age of Instant Access, vol. 88, pp. 167–178. Springer, Dordrecht (2007)
Gabler, S., Ayhan, Ö.: Gewichtungen bei Erhebungen im Festnetz und über Mobilfunkt: Ein Dual Frame Ansatz. In: Gabler, S. (ed.) Mobilfunktelefonie. Eine Herausforderung für die Umfrageforschung, pp. 39–46. ZUMA, Mannheim (2007)
Hu, S.S., Balluz, L., Battaglia, M.P., Frankel, M.R.: Improving public health surveillance using a dual-frame survey of landline and cell phone numbers. Am. J. Epidemiol. 173(6), 703–711 (2011). https://doi.org/10.1093/aje/kwq442
Kagerbauer, M., Manz, W., Zumkeller, D.: Analysis of PAPI, CATI, and CAWI methods for a multiday household travel survey. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 289–304. Emerald Group Publishing (2013)
Kemmick Pintor, J.B., McAlpine, D., Beebe, T.J., Johnson, P.J.: Propensity score matching to measure the effect of survey mode on reports of racial and ethnic discrimination in health care. Med. Care 53(5), 471–476 (2015). https://doi.org/10.1097/MLR.0000000000000351
Klausch, T.: Lecture 1: Decomposing total mode effects in mixed mode data. Lecture Notes of the GESIS Summer Course “Mixed Mode and Mixed Device Surveys”. Cologne (2015)
Klausch, T., Hox, J., Schouten, B.: Selection error in single- and mixed mode surveys of the Dutch general population. J. R. Stat. Soc. A 178(4), 945–961 (2015). https://doi.org/10.1111/rssa.12102
Klausch, T., Hox, J.J., Schouten, B.: Measurement effects of survey mode on the equivalence of attitudinal rating scale questions. Sociol. Methods Res. 42(3), 227–263 (2013). https://doi.org/10.1177/0049124113500480
Kolenikov, S., Kennedy, C.: Evaluating three approaches to statistically adjust for mode effects. J. Surv. Stat. Methodol. 2(2), 126–158 (2014). https://doi.org/10.1093/jssam/smu004
Link, M.W., Battaglia, M.P., Frankel, M.R., Osborn, L., Mokdad, A.H.: Reaching the U.S. cell phone generation—comparison of cell phone survey results with an ongoing landline telephone survey. Public Opin. Q. 71(5), 814–839 (2007). https://doi.org/10.1093/poq/nfm051
Rosenbaum, P.R.: Model-based direct adjustment. J. Am. Stat. Assoc. 82(398), 387 (1987). https://doi.org/10.2307/2289440
Rosenbaum, P.R., Rubin, D.: The central role of the propensity score in observational studies for causal effects. Biometrika 70(1), 41–55 (1983). https://doi.org/10.1093/biomet/70.1.41
Segert, A.: Informationspraktiken. Technikaffinität und Alltagsmobilität, Wien (2012)
Streit, T., Chlond, B., Weiss, C., Vortisch, P.: Deutsches Mobilitätspanel (MOP)—Wissenschaftliche Begleitung und Auswertungen Bericht 2013/2014. Alltagsmobilität und Fahrleistung, Karlsruhe (2015)
Thériault, M., Lee-Gosselin, M., Alexandre, L., Théberge, F., Dieumegarde, L.: Web versus pencil-and-paper surveys of weekly mobility: conviviality, technical and privacy issues. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 225–246. Emerald Group Publishing (2013)
Vannieuwenhuyze, J.T.A., Loosveldt, G.: Evaluating relative mode effects in mixed-mode surveys. Three methods to disentangle selection and measurement effects. Sociol. Methods Res. 42(1), 82–104 (2013). https://doi.org/10.1177/0049124112464868
Weiss, C., Chlond, B., Hilgert, T., Vortisch, P.: Deutsches Mobilitätspanel (MOP)—Wissenschaftliche Begleitung und Auswertungen Bericht 2014/2015. Alltagsmobilität und Fahrleistung, Karlsruhe (2016)
Wirtz, M., Streit, T., Chlond, B., Vortisch, P.: On new measures for detection of data quality risks in mobility panel surveys. Transp. Res. Rec. J. Transp. Res. Board 2354, 19–28 (2013). https://doi.org/10.3141/2354-03
Wolf, J., Wilhelm, J., Casa, J., Sen, S.: Case study: multiple data collection methods and the NY/NJ/CT regional travel survey. In: Zmud, J., Lee-Gosselin, M., Munizaga, M., Carrasco, J.A. (eds.) Transport Survey Methods - Best Practice for Decision Making, pp. 71–90. Emerald Group Publishing (2013)
Xing, Y., Handy, S.L.: Online Versus phone surveys: comparison of results for bicycling survey. In: Transportation Research Board (ed.) TRB 91st Annual Meeting Compendium of Papers (2012)
Zumkeller, D., Chlond, B.: Dynamics of change: fifteen-year german mobility panel. In: Transportation Research Board (ed.) TRB 88th Annual Meeting Compendium of Papers (2009)
Acknowledgements
The authors would like to acknowledge the valuable comments provided by three anonymous referees and by the associate editor (Patricia L. Mokhtarian). This paper presents analyses of the German Mobility Panel funded by the German Federal Ministry of Transport and Digital Infrastructure. An earlier version of the paper was presented at the 96th Transportation Research Board Annual Meeting.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Eisenmann, C., Chlond, B., Minster, C. et al. Assessing the effects of a mixed-mode design in a longitudinal household travel survey. Transportation 46, 1737–1753 (2019). https://doi.org/10.1007/s11116-018-9879-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11116-018-9879-2