, Volume 83, Issue 1, pp 255–271 | Cite as

Multilevel Multivariate Meta-analysis with Application to Choice Overload

  • Blakeley B. McShaneEmail author
  • Ulf Böckenholt


We introduce multilevel multivariate meta-analysis methodology designed to account for the complexity of contemporary psychological research data. Our methodology directly models the observations from a set of studies in a manner that accounts for the variation and covariation induced by the facts that observations differ in their dependent measures and moderators and are nested within, for example, papers, studies, groups of subjects, and study conditions. Our methodology is motivated by data from papers and studies of the choice overload hypothesis. It more fully accounts for the complexity of choice overload data relative to two prior meta-analyses and thus provides richer insight. In particular, it shows that choice overload varies substantially as a function of the six dependent measures and four moderators examined in the domain and that there are potentially interesting and theoretically important interactions among them. It also shows that the various dependent measures have differing levels of variation and that levels up to and including the highest (i.e., the fifth, or paper, level) are necessary to capture the variation and covariation induced by the nesting structure. Our results have substantial implications for future studies of choice overload.


multilevel multivariate meta-analysis choice overload 


  1. Akaike, H. (1974). A new look at the statistical model identification. IEEE Transactions on Automatic Control, 19(6), 716–723.CrossRefGoogle Scholar
  2. Becker, B. J. (2000). Multivariate meta-analysis. In H. E. A. Tinsley & S. D. Brown (Eds.), Handbook of applied multivariate statistics and mathematical modeling. San Diego: Academic Press.Google Scholar
  3. Becker, B. J., Hedges, L. V., & Pigott, T. D. (2004). Statistical analysis policy brief. Oslo: The Campbell Collaboration.Google Scholar
  4. Berkey, C . S., Hoaglin, D . C., Antczak-Bouckoms, A., Mosteller, F., & Colditz, G . A. (1998). Meta-analysis of multiple outcomes by regression with random effects. Statistics in Medicine, 17(22), 2537–2550. ISSN 1097-0258.CrossRefPubMedGoogle Scholar
  5. Chernev, A. (2003a). When more is less and less is more: The role of ideal point availability and assortment in consumer choice. Journal of Consumer Research, 30(2), 170–183.CrossRefGoogle Scholar
  6. Chernev, A. (2003b). Product assortment and individual decision processes. Journal of Personality and Social Psychology, 85, 151–162.CrossRefPubMedGoogle Scholar
  7. Chernev, A. (2005). Feature complementarity and assortment in choice. Journal of Consumer Research, 31(4), 748–759.CrossRefGoogle Scholar
  8. Chernev, A. (2006). Decision focus and consumer choice among assortments. Journal of Consumer Research, 33(1), 50–59.CrossRefGoogle Scholar
  9. Chernev, A., & Hamilton, R. (2009). Assortment size and option attractiveness in consumer choice among retailers. Journal of Marketing Research, 46(3), 410–420.CrossRefGoogle Scholar
  10. Chernev, A., Böckenholt, U., & Goodman, J. (2015). Choice overload: A conceptual review and meta-analysis. Journal of Consumer Psychology, 25(2), 333–358.CrossRefGoogle Scholar
  11. Cohen, J. (1992). A power primer. Psychological Bulletin, 112, 155–159.CrossRefPubMedGoogle Scholar
  12. Diehl, K., & Poynor, C. (2010). Great expectations?! Assortment size, expectations and satisfaction. Journal of Marketing Research, 47(2), 312–322.CrossRefGoogle Scholar
  13. Fasolo, B., Carmeci, F. A., & Misuraca, R. (2009). The effect of choice complexity on perception of time spent choosing: When choice takes longer but feels shorter. Psychology and Marketing, 26(3), 213–228.CrossRefGoogle Scholar
  14. Goodman, J. K., & Malkoc, S. A. (2012). Choosing here and now versus there and later: The moderating role of psychological distance on assortment size preferences. Journal of Consumer Research, 39(4), 751–768.CrossRefGoogle Scholar
  15. Gourville, J . T., & Soman, D. (2005). Overchoice and assortment type: When and why variety backfires. Marketing Science, 24(3), 382–395.CrossRefGoogle Scholar
  16. Greifeneder, R., Scheibehenne, B., & Kleber, N. (2010). Less may be more when choosing is difficult: Choice complexity and too much choice. Acta Psychologica, 133(1), 45–50.CrossRefPubMedGoogle Scholar
  17. Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation and to related problems. Journal of the American Statistical Association, 72(358), 320–338.CrossRefGoogle Scholar
  18. Haynes, G. A. (2009). Testing the boundaries of the choice overload phenomenon: The effect of number of options and time pressure on decision difficulty and satisfaction. Psychology & Marketing, 26(3), 204–212.CrossRefGoogle Scholar
  19. Inbar, Y., Botti, S., & Hanko, K. (2011). Decision speed and choice regret: When haste feels like waste. Journal of Experimental Social Psychology, 47(3), 533–540.CrossRefGoogle Scholar
  20. Ishak, K. J., Platt, R. W., Joseph, L., & Hanley, J. A. (2008). Impact of approximating or ignoring within-study covariances in multivariate meta-analyses. Statistics in Medicine, 27, 670–686.CrossRefPubMedGoogle Scholar
  21. Iyengar, S. S., & Lepper, M. R. (2000). When choice is demotivating: Can one desire too much of a good thing? Journal of Personality and Social Psychology, 79(6), 996–1006.CrossRefGoogle Scholar
  22. Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis. Psychological Methods, 1(3), 227–235.CrossRefGoogle Scholar
  23. Lin, C.-H., & Wu, P.-H. (2006). The effect of variety on consumer preferences: The role of need for cognition and recommended alternatives. Social Behavior and Personality: An International Journal, 34(7), 865–875.CrossRefGoogle Scholar
  24. McShane, B. B., & Böckenholt, U. (2014). You cannot step into the same river twice: When power analyses are optimistic. Perspectives on Psychological Science, 9(6), 612–625.CrossRefPubMedGoogle Scholar
  25. McShane, B. B., & Böckenholt, U. (2017). Single paper meta-analysis: Benefits for study summary, theory-testing, and replicability. Journal of Consumer Research, 43, 1048–1063.Google Scholar
  26. McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016). Adjusting for publication bias in meta-analysis: An evaluation of selection methods and some cautionary notes. Perspectives on Psychological Science, 11(5), 730–749.CrossRefPubMedGoogle Scholar
  27. Mogilner, C., Rudnick, T., & Iyengar, S. S. (2008). The mere categorization effect: How the presence of categories increases choosersí perceptions of assortment variety and outcome satisfaction. Journal of Consumer Research, 35(2), 202–215.CrossRefGoogle Scholar
  28. Morrin, M., Broniarczyk, S . M., & Inman, J . J. (2012). Plan format and participation in 401 (k) plans: The moderating role of investor knowledge. Journal of Public Policy & Marketing, 31(2), 254–268. Fall.CrossRefGoogle Scholar
  29. Oppewal, H., & Koelemeijer, K. (2005). More choice is better: Effects of assortment size and composition on assortment evaluation. International Journal of Research in Marketing, 22(1), 45–60.CrossRefGoogle Scholar
  30. Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993). The adaptive decision maker. Cambridge: Cambridge University Press.CrossRefGoogle Scholar
  31. Riley, R. D. (2009). Multivariate meta-analysis: The effect of ignoring within-study correlation. Journal of the Royal Statistical Society, A, 172, 789–811.CrossRefGoogle Scholar
  32. Robinson, G. K. (1991). That blup is a good thing: The estimation of random effects. Statistical Science, 6(1), 15–32.CrossRefGoogle Scholar
  33. Scheibehenne, B., Greifeneder, R., & Todd, P. M. (2009). What moderates the too-much-choice effect? Psychology and Marketing, 26(3), 229–253.CrossRefGoogle Scholar
  34. Scheibehenne, B., Greifeneder, R., & Todd, P. M. (2010). Can there ever be too many options? A meta-analytic review of choice overload. Journal of Consumer Research, 37(3), 409–425.CrossRefGoogle Scholar
  35. Schwartz, B. (2004). The paradox of choice: Why more is less. New York: Ecco.Google Scholar
  36. Sela, A., Berger, J., & Liu, W. (2009). Variety, vice, and virtue: How assortment size influences option choice. Journal of Consumer Research, 35(6), 941–951.CrossRefGoogle Scholar
  37. Shah, A. M., & Wolford, G. (2007). Buying behavior as a function of parametric variation of number of choices. Psychological Science, 18(5), 369–370.CrossRefPubMedGoogle Scholar
  38. Townsend, C., & Kahn, B. E. (2014). The ‘visual preference heuristic’: The influence of visual versus verbal depiction on assortment processing, perceived variety, and choice overload. Journal of Consumer Research, 40(5), 993–1015.CrossRefGoogle Scholar

Copyright information

© The Psychometric Society 2017

Authors and Affiliations

  1. 1.Kellogg School of ManagementNorthwestern UniversityEvanstonUSA

Personalised recommendations