# Multilevel Multivariate Meta-analysis with Application to Choice Overload

- 818 Downloads
- 4 Citations

## Abstract

We introduce multilevel multivariate meta-analysis methodology designed to account for the complexity of contemporary psychological research data. Our methodology directly models the observations from a set of studies in a manner that accounts for the variation and covariation induced by the facts that observations differ in their dependent measures and moderators and are nested within, for example, papers, studies, groups of subjects, and study conditions. Our methodology is motivated by data from papers and studies of the choice overload hypothesis. It more fully accounts for the complexity of choice overload data relative to two prior meta-analyses and thus provides richer insight. In particular, it shows that choice overload varies substantially as a function of the six dependent measures and four moderators examined in the domain and that there are potentially interesting and theoretically important interactions among them. It also shows that the various dependent measures have differing levels of variation and that levels up to and including the highest (i.e., the fifth, or paper, level) are necessary to capture the variation and covariation induced by the nesting structure. Our results have substantial implications for future studies of choice overload.

## Keywords

multilevel multivariate meta-analysis choice overload## References

- Akaike, H. (1974). A new look at the statistical model identification.
*IEEE Transactions on Automatic Control*,*19*(6), 716–723.CrossRefGoogle Scholar - Becker, B. J. (2000). Multivariate meta-analysis. In H. E. A. Tinsley & S. D. Brown (Eds.),
*Handbook of applied multivariate statistics and mathematical modeling*. San Diego: Academic Press.Google Scholar - Becker, B. J., Hedges, L. V., & Pigott, T. D. (2004).
*Statistical analysis policy brief*. Oslo: The Campbell Collaboration.Google Scholar - Berkey, C . S., Hoaglin, D . C., Antczak-Bouckoms, A., Mosteller, F., & Colditz, G . A. (1998). Meta-analysis of multiple outcomes by regression with random effects.
*Statistics in Medicine*,*17*(22), 2537–2550. ISSN 1097-0258.CrossRefPubMedGoogle Scholar - Chernev, A. (2003a). When more is less and less is more: The role of ideal point availability and assortment in consumer choice.
*Journal of Consumer Research*,*30*(2), 170–183.CrossRefGoogle Scholar - Chernev, A. (2003b). Product assortment and individual decision processes.
*Journal of Personality and Social Psychology*,*85*, 151–162.CrossRefPubMedGoogle Scholar - Chernev, A. (2005). Feature complementarity and assortment in choice.
*Journal of Consumer Research*,*31*(4), 748–759.CrossRefGoogle Scholar - Chernev, A. (2006). Decision focus and consumer choice among assortments.
*Journal of Consumer Research*,*33*(1), 50–59.CrossRefGoogle Scholar - Chernev, A., & Hamilton, R. (2009). Assortment size and option attractiveness in consumer choice among retailers.
*Journal of Marketing Research*,*46*(3), 410–420.CrossRefGoogle Scholar - Chernev, A., Böckenholt, U., & Goodman, J. (2015). Choice overload: A conceptual review and meta-analysis.
*Journal of Consumer Psychology*,*25*(2), 333–358.CrossRefGoogle Scholar - Diehl, K., & Poynor, C. (2010). Great expectations?! Assortment size, expectations and satisfaction.
*Journal of Marketing Research*,*47*(2), 312–322.CrossRefGoogle Scholar - Fasolo, B., Carmeci, F. A., & Misuraca, R. (2009). The effect of choice complexity on perception of time spent choosing: When choice takes longer but feels shorter.
*Psychology and Marketing*,*26*(3), 213–228.CrossRefGoogle Scholar - Goodman, J. K., & Malkoc, S. A. (2012). Choosing here and now versus there and later: The moderating role of psychological distance on assortment size preferences.
*Journal of Consumer Research*,*39*(4), 751–768.CrossRefGoogle Scholar - Gourville, J . T., & Soman, D. (2005). Overchoice and assortment type: When and why variety backfires.
*Marketing Science*,*24*(3), 382–395.CrossRefGoogle Scholar - Greifeneder, R., Scheibehenne, B., & Kleber, N. (2010). Less may be more when choosing is difficult: Choice complexity and too much choice.
*Acta Psychologica*,*133*(1), 45–50.CrossRefPubMedGoogle Scholar - Harville, D. A. (1977). Maximum likelihood approaches to variance component estimation and to related problems.
*Journal of the American Statistical Association*,*72*(358), 320–338.CrossRefGoogle Scholar - Haynes, G. A. (2009). Testing the boundaries of the choice overload phenomenon: The effect of number of options and time pressure on decision difficulty and satisfaction.
*Psychology & Marketing*,*26*(3), 204–212.CrossRefGoogle Scholar - Inbar, Y., Botti, S., & Hanko, K. (2011). Decision speed and choice regret: When haste feels like waste.
*Journal of Experimental Social Psychology*,*47*(3), 533–540.CrossRefGoogle Scholar - Ishak, K. J., Platt, R. W., Joseph, L., & Hanley, J. A. (2008). Impact of approximating or ignoring within-study covariances in multivariate meta-analyses.
*Statistics in Medicine*,*27*, 670–686.CrossRefPubMedGoogle Scholar - Iyengar, S. S., & Lepper, M. R. (2000). When choice is demotivating: Can one desire too much of a good thing?
*Journal of Personality and Social Psychology*,*79*(6), 996–1006.CrossRefGoogle Scholar - Kalaian, H. A., & Raudenbush, S. W. (1996). A multivariate mixed linear model for meta-analysis.
*Psychological Methods*,*1*(3), 227–235.CrossRefGoogle Scholar - Lin, C.-H., & Wu, P.-H. (2006). The effect of variety on consumer preferences: The role of need for cognition and recommended alternatives.
*Social Behavior and Personality: An International Journal*,*34*(7), 865–875.CrossRefGoogle Scholar - McShane, B. B., & Böckenholt, U. (2014). You cannot step into the same river twice: When power analyses are optimistic.
*Perspectives on Psychological Science*,*9*(6), 612–625.CrossRefPubMedGoogle Scholar - McShane, B. B., & Böckenholt, U. (2017). Single paper meta-analysis: Benefits for study summary, theory-testing, and replicability.
*Journal of Consumer Research*,*43*, 1048–1063.Google Scholar - McShane, B. B., Böckenholt, U., & Hansen, K. T. (2016). Adjusting for publication bias in meta-analysis: An evaluation of selection methods and some cautionary notes.
*Perspectives on Psychological Science*,*11*(5), 730–749.CrossRefPubMedGoogle Scholar - Mogilner, C., Rudnick, T., & Iyengar, S. S. (2008). The mere categorization effect: How the presence of categories increases choosersí perceptions of assortment variety and outcome satisfaction.
*Journal of Consumer Research*,*35*(2), 202–215.CrossRefGoogle Scholar - Morrin, M., Broniarczyk, S . M., & Inman, J . J. (2012). Plan format and participation in 401 (k) plans: The moderating role of investor knowledge.
*Journal of Public Policy & Marketing*,*31*(2), 254–268. Fall.CrossRefGoogle Scholar - Oppewal, H., & Koelemeijer, K. (2005). More choice is better: Effects of assortment size and composition on assortment evaluation.
*International Journal of Research in Marketing*,*22*(1), 45–60.CrossRefGoogle Scholar - Payne, J. W., Bettman, J. R., & Johnson, E. J. (1993).
*The adaptive decision maker*. Cambridge: Cambridge University Press.CrossRefGoogle Scholar - Riley, R. D. (2009). Multivariate meta-analysis: The effect of ignoring within-study correlation.
*Journal of the Royal Statistical Society, A*,*172*, 789–811.CrossRefGoogle Scholar - Robinson, G. K. (1991). That blup is a good thing: The estimation of random effects.
*Statistical Science*,*6*(1), 15–32.CrossRefGoogle Scholar - Scheibehenne, B., Greifeneder, R., & Todd, P. M. (2009). What moderates the too-much-choice effect?
*Psychology and Marketing*,*26*(3), 229–253.CrossRefGoogle Scholar - Scheibehenne, B., Greifeneder, R., & Todd, P. M. (2010). Can there ever be too many options? A meta-analytic review of choice overload.
*Journal of Consumer Research*,*37*(3), 409–425.CrossRefGoogle Scholar - Schwartz, B. (2004).
*The paradox of choice: Why more is less*. New York: Ecco.Google Scholar - Sela, A., Berger, J., & Liu, W. (2009). Variety, vice, and virtue: How assortment size influences option choice.
*Journal of Consumer Research*,*35*(6), 941–951.CrossRefGoogle Scholar - Shah, A. M., & Wolford, G. (2007). Buying behavior as a function of parametric variation of number of choices.
*Psychological Science*,*18*(5), 369–370.CrossRefPubMedGoogle Scholar - Townsend, C., & Kahn, B. E. (2014). The ‘visual preference heuristic’: The influence of visual versus verbal depiction on assortment processing, perceived variety, and choice overload.
*Journal of Consumer Research*,*40*(5), 993–1015.CrossRefGoogle Scholar