Abstract
Within higher education research, both hierarchical linear models (HLM) and econometric panel models are commonly employed in studies examining multilevel data. These two statistical traditions are interesting to compare, because despite a number of underlying similarities, they differ in complementary but often confusing ways. The confusion arises from varying terminology and model presentation, which makes almost identical models appear different. Econometrics textbooks focus on how the multilevel structure can be exploited to advance overall causal inference, while HLM texts primarily highlight opportunities to examine heterogeneity across groups. This chapter highlights the core similarities between these two traditions so that HLM-trained researchers can use their existing knowledge base to read econometric-based articles and vice versa. By contrasting these approaches, this chapter helps applied higher education researchers learn the full range of benefits allowed by the advanced analysis of multilevel data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
To our knowledge, only one previous paper has spent considerable space directly comparing statistical models from these two specific traditions (HLM and econometrics), and this paper was prepared for a conference and not disseminated into the education research community (Chaplin, 2003).
- 2.
Of course, knowledge is only one of the challenges faced by higher education researchers when choosing the methodological approach in which to specialize. Students within higher education programs often face difficulty gaining access to graduate methodological classes in particular fields, and some programs, such as economics, teach in a manner that requires high levels of prior mathematical training. For these and several other reasons, higher education programs have historically funneled their students into methodology courses taught within educational psychology.
- 3.
Pooled OLS simply means an OLS regression that examines a data set that combines data from a number of different groups.
- 4.
This prediction clearly assumes that parental income is likely to be correlated with the propensity of students to be on the margin of attendance. Because price responsiveness and the number of submitted applications vary by a student’s parental income, such an assumption is realistic.
- 5.
To understand what we mean by a “small number,” consider the sample size requirements for statistical analysis in general. Now apply those requirements to each group in the analysis.
- 6.
The fourth benefit connects to multiple elements of Shadish, Cook, and Campbell’s validity typology. Because the upcoming discussion focuses on the trade-offs between validity types, we will focus solely on the first three benefits.
- 7.
For these points, our focus on simplicity is obscuring some important technical details. The HLM framework does not include direct analysis of the variation in β 0j and β 1j using equations (1)–(3). Instead, this variation is measured by estimating a different set of equations which do not contain W j in equations (2) and (3). Raudenbush and Bryk (2002, p. 77–80) call this restricted version the “random coefficients” model. Using this model, one can examine the variation in β 0j by estimating the variance of u 0j . The variation in β 1j can be examined by estimating the variance of u 1j .
- 8.
- 9.
One can also think of the fixed-effects model as adding a dummy variable for each group. From this perspective, we are employing a fixed-effects model whenever we add dummy variables for any classification in which each observation is in one, but no more than one, category. Older versions of the Carnegie classification would be a good example from higher education research. For a multilevel data set with a large number of groups, the addition of dummy variables for each group creates computational challenges, which is why equation (10) is used instead.
- 10.
The coverage of other multilevel data structures is slightly larger in more advanced econometric textbooks, such as Wooldridge (2002). These books, however, still place a much larger emphasis on panel data. Individual journal articles, such as Moulton (1990) and Wooldridge (2003), focus on cluster samples in much more depth. These articles as well as advanced econometrics textbooks assume a strong existing knowledge base in econometrics and mathematics, so their usefulness will vary considerably across researchers.
- 11.
There are some prominent econometric papers, however, that focus on our third benefit of multilevel data. For example, Rivkin et al. (2005) used a data set containing students nested within teachers to estimate the variation in student test scores across teachers.
- 12.
Some papers that use alternative multilevel structures may also possess more realistic assumptions than the three papers we reviewed. As noted earlier in the chapter, difference-in-differences models can produce compelling results in certain contexts (Dynarski, 2000; Cornwell, Mustard, & Sridhar, 2006). Analysis of within-family differences can also be convincing, because siblings possess a number of shared traits and experiences (Ashenfelter & Rouse, 1998). Both difference-in-differences models and sibling studies are very common in economics.
References
Archibald, R., & Feldman, D. (2006). State higher education spending and the tax revolt. Journal of Higher Education, 77, 618–644.
Ashenfelter, O., & Krueger, A. (1994). Estimates of the economic return to schooling from a new sample of twins. American Economic Review, 84(5), 1157–1173.
Ashenfelter, O., & Rouse, C. (1998). Income, schooling, and ability: Evidence from a new sample of identical twins. Quarterly Journal of Economics, 113(1), 253–284.
Baltagi, B. H. (2008). Econometric analysis of panel data (4th ed.). Chichester: Wiley.
Becher, T., & Trowler, P. (2001). Academic tribes and territories: Intellectual enquiry and the cultures of disciplines. Buckingham: Society for Research in Higher Education/Open University Press.
Berliner, C. (2002). Comment: Educational research: The hardest science of all. Educational Researcher, 31(8), 18–20.
Breneman, D. (1994). Liberal arts colleges: Thriving, surviving, or endangered? Washington, DC: Brookings Institution.
Bryk, A., & Raudenbush, S. (1992). Hierarchical linear models. Newbury Park, CA: Sage.
Cameron, A., & Trivedi, P. (2005). Microeconometrics: Methods and applications. Cambridge: Cambridge University Press.
Chamberlain, G. (1982). Multivariate regression models for panel data. Journal of Econometrics, 18, 5–46.
Chamberlain, G. (1984). Panel data. In Z. Griliches & M. Intriligator (Eds.), Handbook of econometrics (Vol. 2, pp. 1247–1318). Amsterdam: North-Holland.
Chaplin, D. (2003). Hierarchical linear models: Strengths and weaknesses. Paper prepared for the November 2003 meetings of the Association for Public Policy and Management.
Cheslock, J., & Rios-Aguilar, C. (2008, November). Reaping (or not) the benefits of hierarchical data. Paper presented at the Association for the Study of Higher Education Meeting, Jacksonville, FL.
Cornwell, C., Mustard, D., & Sridhar, D. (2006). The enrollment effects of merit-based financial aid: Evidence from Georgia’s hope program. Journal of Labor Economics, 24(4), 761–786.
Cronbach, L. J. (1976). Research on classrooms and schools: Formulation of questions, design, and analysis, Occasional paper. Stanford, CA: Stanford Evaluation Consortium, Stanford University.
Dale, S., & Krueger, A. (2002). Estimating the payoff to attending a more selective college: An application of selection on observables and unobservables. Quarterly Journal of Economics, 107(4), 1491–1527.
Dedrick, R., et al. (2009). Multilevel modeling: A review of methodological issues and applications. Review of Educational Research, 79(1), 69–102.
Dynarski, S. (2000). Hope for whom? Financial aid for the middle class and its impact on college attendance. National Tax Journal, 53(3), 629–662.
Ethington, C. (1997). A hierarchical linear modeling approach to studying college effects. In J. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 12, pp. 165–194). New York: Agathon Press.
Gelman, A., & Hill, J. (2007). Data analysis using regression and multilevel/hierarchical models. Cambridge: Cambridge University Press.
Gelman, A., & Little, T. (1997). Poststratification into many categories using hierarchical logistic regression. Survey Methodology, 23, 127–135.
Gelman, A., Shor, B., Bafumi, J., & Park, D. (2007). Rich state, poor state, red state, blue state: What’s the matter with Connecticut. Quarterly Journal of Political Science, 2, 345–367.
Goldhaber, D., & Brewer, D. (1997). Why don’t schools and teachers seem to matter? Assessing the impact of unobservables on educational productivity. Journal of Human Resources, 32(3), 505–523.
Griliches, Z. (1979). Sibling models and data in economics: Beginnings of a survey. Journal of Political Economy, 87(5), S37–S64.
Heck, R., & Thomas, S. (2009). An introduction to multilevel modeling techniques (2nd ed.). New York: Routledge/Taylor and Francis.
Huber, P. (1967). The behavior of maximum-likelihood estimates under non-standard conditions. Paper presented at the Proceedings of the Fifth Berkeley Symposium on Mathematical Statistics and Probability, Berkeley, CA.
Kenny, D., Mannetti, L., Pierro, A., Livi, S., & Kashy, D. (2002). The statistical analysis of data from small groups. Journal of Personality and Social Psychology, 83, 126–137.
Labaree, D. (2003). The peculiar problems of preparing educational researchers. Educational Researcher, 32(4), 13–22.
Langenkamp, A. (2010). Academic vulnerability and resilience during transition to high school: The role of social relationships and district context. Sociology of Education, 83(1), 1–19.
McPherson, M., & Schapiro, M. (1998). The student aid game: Meeting need and rewarding talent in American higher education. Princeton, NJ: Princeton University Press.
Meyer, B. D. (1995). Natural and quasi-experiments in economics. Journal of Business and Economic Statistics, 13(2), 151–161.
Moulton, B. (1990). An illustration of a pitfall in estimating the effects of aggregate variables on micro units. Review of Economics and Statistics, 72(2), 334–338.
Mundlack, Y. (1978a). On the pooling of time series and cross section data. Econometrica, 46, 69–85.
Mundlack, Y. (1978b). Models with variable coefficients: Integration and extension. Annales de l’INNSEE, 30–31, 483–509.
Raudenbush, S., & Bryk, A. (1986). A hierarchical model for examining school effects. Sociology of Education, 59(1), 1–17.
Raudenbush, S. W., & Bryk, A. S. (2002). Hierarchical linear models: Applications and data analysis methods (2nd ed.). Newbury Park, CA: Sage.
Raudenbush, S., Bryk, A., Cheong, Y., Congdon, R., & du Toit, M. (2004). HLM 6: Linear and nonlinear modeling. Lincolnwood, IL: Scientific Software International Inc.
Rivkin, S., Hanushek, E., & Kain, J. (2005). Teachers, schools, and academic achievement. Econometrica, 73(2), 417–458.
Rogers, W. (1993). Regression standard errors in clustered samples. Stata Technical Bulletin, 13, 19–23.
Shadish, W., Cook, T., & Campbell, D. (2002). Experimental & quasi-experimental designs for generalized causal inference. Boston: Houghton Mifflin.
Smart, J. (2005). Attributes of exemplary research manuscripts employing quantitative analyses. Research in Higher Education, 46(4), 461–477.
Snijders, T. (2005). Power and sample size in multilevel linear models. In B. Everitt & D. Howell (Eds.), Encyclopedia of statistics in behavioral science (Vol. 3, pp. 1570–1573). Chichester: Wiley.
Snijders, T., & Bosker, R. (1999). Multilevel analysis: An introduction to basic and advanced multilevel modeling. Thousand Oaks, CA: Sage.
Stock, J., & Watson, M. (2007). Introduction to econometrics, Addison-Wesley Series in Economics (2nd ed.). Boston: Pearson.
Umbach, P. (2007). How effective are they? Exploring the impact of contingent faculty on undergraduate education. The Review of Higher Education, 30(2), 91–123.
White, H. (1980). A heteroskedasticity-consistent covariance matrix estimator and a direct test for heteroskedasticity. Econometrica, 48, 817–830.
Wooldridge, J. (2002). Econometric analysis of cross section and panel data. Cambridge, MA: MIT Press.
Wooldridge, J. M. (2003). Cluster-sample methods in applied econometrics. American Economic Review, 93(2), 133–138.
Wooldridge, J. (2009). Introductory econometrics: A modern approach (4th ed.). Mason, OH: South-Western Cengage Learning.
Zhang, L. (2010). The use of panel data models in higher education policy studies. In J. Smart (Ed.), Higher education: Handbook of theory and research (Vol. 25, pp. 307–349). New York: Agathon Press.
Author information
Authors and Affiliations
Corresponding author
Editor information
Editors and Affiliations
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media B.V.
About this chapter
Cite this chapter
Cheslock, J.J., Rios-Aguilar, C. (2011). Multilevel Analysis in Higher Education Research: A Multidisciplinary Approach. In: Smart, J., Paulsen, M. (eds) Higher Education: Handbook of Theory and Research. Higher Education: Handbook of Theory and Research, vol 26. Springer, Dordrecht. https://doi.org/10.1007/978-94-007-0702-3_3
Download citation
DOI: https://doi.org/10.1007/978-94-007-0702-3_3
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-94-007-0701-6
Online ISBN: 978-94-007-0702-3
eBook Packages: Humanities, Social Sciences and LawEducation (R0)