Abstract
In an attempt to forge tighter social relations, small school reformers advocate school designs intended to create smaller, more trusting, and more collaborative settings. These efforts to enhance students’ social capital in the form of social closure are ultimately tied to improving academic outcomes. Using data derived from ELS: 2002, this study employs propensity scores in the context of multilevel models to estimate the effects of a specific school-level variant of social closure, referred to as a norm-enforcing school, on students’ mathematics achievement. Results estimate that attending a norm-enforcing school has no effect on 12th-grade mathematics achievement. This result questions the presumed benefits of social capital and its emphasis on norm-enforcement and social control. Policy implications are discussed in light of contemporary urban school reform initiatives that focus on reductions in school size.
This is a preview of subscription content, log in via an institution to check access.
Similar content being viewed by others
Notes
This cutpoint was determined by examining a histogram of the distribution of mean social closure around school scores (M = 1.97, SD = .38). The top quarter was approximately 65% of a SD above the mean. Specifically, this treatment group had scores that ranged from 2.22 to 3.00. Admittedly, the boundaries of the treatment and control groups are not discreet having been derived from a continuous composite measure. As noted by Winship and Morgan (1999), any two states to which an individual could be assigned or could choose to enter can be considered treatment and control, hence the use of traditional experimental language. To address the arbitrariness of treatment assignment, sensitivity analyses were performed for using different cutpoints in the distribution.
In addition, there is a more practical reason for focusing solely on mathematics achievement. Unlike its predecessor NELS, ELS only included a mathematics assessment on its first follow-up (2004). This is likely due to costs and the fact that mathematics achievement, unlike other subject areas, is so strongly related to secondary and post-secondary success.
The goal of propensity score matching is to obtain accurate predictions of the propensity to attend a norm-enforcing school by using every appropriate variable in the dataset that might potentially be directly related to the effect, which is why these 23 have been selected. Other applications of propensity scores in educational research have used over 150 covariates in the matching (see e.g., Hong and Raudenbush, 2005), though it is more common in other research areas to use fewer in order to better balance the covariates within strata. As noted by Cochran (1968), as the number of covariates increases, the number of strata increases exponentially.
This weight is needed in order to make inferences about the study’s target population (10th grade students attending any U.S. high school in 2002). It has been calculated by first normalizing the appropriate weight variable (f1pnlw). Then, by dividing the normalized weight by the design effect of the dependent variable, a design-effect adjusted weight is created. The design effect is described in the ELS: 2002 technical documentation (Ingels, et al., 2005).
It should be noted that a variety of matching schemes are possible (Winship and Morgan, 1999). The result of the subclassification approach employed in this study was confirmed by also matching multiple control cases to each treatment case (referred to as the many-to-one approach).
An alternative way to verify balance uses F ratios by subjecting each of the 23 covariates to two-way [2 groups (treatment and control) *5 (strata)] analysis of variance. This is the method advocated by Rosenbaum and Rubin (1983).
References
Bill and Melinda Gates Foundation. (2003). $51 Million grant from the Bill & Melinda Gates Foundation to support small dynamic high schools to boost student achievement. Seattle, WA: Author.
Bourdieu, P., & Wacquant, L. (1992). An invitation to reflexive sociology. Chicago: University of Chicago Press.
Burt, R. S. (1992). Structural holes: The social structure of competition. Cambridge, MA: Harvard University Press.
Burt, R. S. (2001). The social capital of structural holes. In M. F. Guillen, R. Collins, P. England, & M. Meyer (Eds.), The new economic sociology: Developments in an emerging field (pp. 148–192). New York: Russell Sage Foundation.
Burt, R. S. (2004). Structural holes and good ideas. American Journal of Sociology, 110, 349–399.
Carbonaro, W. (1998). A little help from my parent’s friends: Intergenerational closure and effects on educational outcomes. Sociology of Education, 71, 295–313.
Chicago Annenberg Challenge. (1994). Smart schools/smart kids: A proposal to the Annenberg Foundation to create the Chicago school reform collaboratives. Chicago: Author.
Cochran, W. G. (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics, 24, 295–313.
Coleman, J. S. (1961). The adolescent society: The social life of the teenager and its impact on education. New York: Free Press.
Coleman, J. S. (1988). Social capital in the creation of human capital. The American Journal of Sociology, 94, S95–S120.
Coleman, J. S. (1990). Foundations of social theory. Cambridge, MA: Harvard University Press.
Connors, A. F., Speroff, T., Dawson, N. V., Thomas, C., Harell, F. E., & Wagner, D. (1996). The effectiveness of right heart catheterization in the initial care of critically ill patients. Journal of the American Medical Association, 276, 889–918.
Cotton, K. (2002). New small learning communities: Findings from recent literature. Portland, OR: Northwest Regional Educational Laboratory.
Croninger, R., & Lee, V. E. (2001). Social capital and dropping out of high school: Benefits to at-risk students of teachers’ support and guidance. Teachers College Record, 103, 548–581.
Dehejia, R., & Wahba, S. (2002). Propensity score-matching methods for nonexperimental causal studies. Review of Economics and Statistics, 84, 151–161.
Dika, S. L., & Singh, K. (2002). Applications of social capital in educational literature: A critical synthesis. Review of Educational Research, 72, 31–60.
Earle, C. C., Tsai, J. S., Gelber, R. D., Weinstein, M. C., Neumann, P. J., & Weeks, J. C. (2001). Effectiveness of chemotherapy for advanced lung cancer in the elderly: Instrumental variable and propensity analysis. Journal of Clinical Oncology, 19, 1064–1070.
Frisco, M., Muller, C., & Frank, K. A. (2007). Using propensity scores to study changing family structure and academic achievement. Journal of Marriage and Family, 69, 721–741.
Gargiulo, M., & Benassi, M. (1999). The dark side of social capital. In S. Gabbay & R. Leenders (Eds.), Social capital and liability (pp. 298–322). Norwell, MA: Kluwer.
Goodnough, A. (2002). Klein taking time to make his mark as schools chancellor. The New York Times, November 2. Retrieved August 1, 2008, from http://query.nytimes.com/gst/fullpage.html?res=9A03EFDF1139F937A15752C1A9649C8B63.
Gootman, E. (2006). City’s new small schools are focus of a bias inquiry. The New York Times, June 16. Retrieved August 1, 2008, from http://www.nytimes.com/2006/06/16/nyregion/16schools.html.
Granovetter, M. (1973). The strength of weak ties. American Journal of Sociology, 78, 1360–1380.
Hahs-Vaughn, D. L., & Onwuegbuzie, A. J. (2006). Estimating and using propensity score analysis with complex samples. The Journal of Experimental Education, 75, 31–65.
Harris, K. M., Florey, F., Tabor, J., Bearman, P. S., Jones, J., & Udry, J. R. (2003). The National longitudinal study of adolescent health: Research design. Chapel Hill, NC: Carolina Population Center at the University of North Carolina.
Hartley, H. (1958). Maximum likelihood estimation from incomplete data. Biometrics, 14, 174–194.
Herszenhorn, D. (2005). Study raises issues on small high schools. The New York Times, November 4. Retrieved August 1, 2008, from http://www.nytimes.com/2005/11/04/nyregion/04small.html.
Herszenhorn, D. (2006). In push to open small schools, a big obstacle: Limited space. The New York Times, August 3. Retrieved August 1, 2008, from http://www.nytimes.com/2006/08/03/education/03schools.html.
Herszenhorn, D. (2007). Small schools to be added by September. The New York Times, January 31. Retrieved August 1, 2008, from http://www.nytimes.com/2007/01/31/nyregion/31schools.html#.
Hong, G., & Raudenbush, S. W. (2005). Effects of kindergarten retention policy on children’s cognitive growth in reading and mathematics. Educational Evaluation and Policy Analysis, 27, 205–224.
Howley, C. B. (2002). Small schools. In A. Molnar (Ed.), School reform proposals: The research evidence (pp. 49–77). Greenwich, CT: Information Age Publishing.
Ingels, S. J., Pratt, D. J., Rogers, J., Siegel, P. H., & Stutts, E. (2005). ELS: 2002 base year to first follow-up data file documentation. Washington, D. C.: National Center for Educational Statistics.
Ingersoll, R. M. (2003). Who controls teachers’ work? Power and accountability in America’s schools. Cambridge, MA: Harvard University Press.
Johnson, J., Duffett, A., Farkas, S., & Collins, K. (2002). Sizing things up: What parents, teachers and students think about large and small high schools. New York: Public Agenda Foundation for the Bill and Melinda Gates Foundation.
Kadushin, C. (2004). Too much investment in social capital. Social Networks, 26, 75–90.
Kafka, J. (2008). Thinking big about getting small: An ideological genealogy of small-school reform. Teachers College Record, 110, 1802–1836.
Kahne, J. E., Sporte, S., de la Torre, M., & Easton, J. Q. (2006). Small high schools on a larger scale: The fiirst three years of the Chicago high school redesign initiative. Chicago: Consortium on Chicago School Research at the University of Chicago.
Kahne, J. E., Sporte, S. E., de la Torre, M., & Easton, J. Q. (2008). Small high schools on a larger scale: The impact of school conversions in Chicago. Educational Evaluation and Policy Analysis, 30, 281–315.
Lee, V. E., & Smith, J. B. (1997). High school size: Which works best and for whom? Educational Evaluation and Policy Analysis, 19, 205–227.
Maroulis, S., & Gomez, L. M. (2008). Does connectedness matter? Evidence from a social network analysis within a small-school reform. Teachers College Record, 110, 1901–1929.
McNeal, R. B., Jr. (1999). Parent involvement as social capital: Differential effectiveness on Science achievement, truancy and dropping-out. Social Forces, 78(1), 117–144.
McRobbie, J. (2001). Are small schools better. San Francisco: WestEd.
Miner, B. (2005). The Gates Foundation and small schools. Milwaukee, WI: Rethinking Schools.
Morgan, S. L. (2001). Counterfactuals, causal effect heterogeneity, and the Catholic school effect on learning. Sociology of Education, 74, 341–374.
Morgan, S. L., & Sørensen, A. B. (1999). Parental networks, social closure, and mathematics learning: A test of Coleman’s social capital explanation of school effects. American Sociological Review, 64, 661–681.
National Association of Secondary School Principals. (1996). Breaking ranks: Changing an American institution. Reston, VA: Author.
Noguera, P. A. (2002). Beyond size: The challenge of high school reform. Educational Leadership, 59, 60–63.
Porter, A. C. (1989). A curriculum out of balance: The case of elementary school mathematics. Educational Researcher, 18, 9–15.
Portes, A., & Landolt, P. (1996). The downside of social capital. The American Prospect, 26, 18–21.
Rosenbaum, P. R. (2002). Observational studies. New York: Springer.
Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70, 41–55.
Rosenbaum, P. R., & Rubin, D. B. (1984). Reducing bias in observational studies using subclassification on the propensity score. Journal of the American Statistical Association, 79, 516–524.
Rubin, D. B. (1979). Using multivariate matched sampling and regression adjustment to control bias in observational studies. Journal of the American Statistical Association, 73, 318–328.
Rubin, D. B., & Thomas, N. (2000). Combining propensity score matching with additional adjustments for prognostic covariates. Journal of the American Statistical Association, 95, 573–585.
Schneider, B., Carnoy, M., Kilpatrick, J., Schmidt, W. H., & Shavelson, R. J. (2007a). Estimating causal effects: Using experimental and observational designs. Washington D.C.: American Educational Research Association.
Schneider, B., Wyse, A. E., & Keesler, V. (2007b). Is small really better? Testing assumptions about high school size. In T. Loveless & F. Hess (Eds.), Brookings papers on education policy: 2006–2007 (pp. 15–47). Washington D.C: The Brookings Institution.
Sizer, T. R. (1992). Horace’s school: Redesigning the American high school. Boston: Houghton Mifflin.
Smith-Maddox, R. (1999). The social networks and resources of African American eighth graders: Evidence from the National Educational Longitudinal Study of 1988. Adolescence, 34, 169–183.
SRI/AIR. (2002). Targeted literature review of major constructs and their components: Evaluating the national school district and network grants program. Palo Alto, CA; Menlo Park, CA: Authors.
U.S. Department of Education. (2001). Small learning communities program. Washington D.C.: Author.
Wasley, P. A., Fine, M., Gladden, M., Holland, N. E., King, S. P., Mosak, E., et al. (2000). Small schools: Great strides—A study of new small schools in Chicago. New York: Bank Street College of Education.
Wells, A. S., & Crain, R. (1997). Stepping over the color line: African-American students in white suburban schools. New Haven, CT: Yale University Press.
Wilson, W. J. (1996). When work disappears: The world of the new urban poor. New York: Knopf.
Winship, C., & Morgan, S. L. (1999). The estimation of causal effects from observational data. Annual Review of Sociology, 25, 659–706.
Acknowledgments
The author would like to thank Spiro Maroulis for his comments on an earlier version of this manuscript. This manuscript also benefited from the comments of two anonymous reviewers. Support was provided by the City University of New York’s Faculty Fellowship Publication Program.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
About this article
Cite this article
Carolan, B.V. Estimating the Effects of Students’ Social Networks: Does Attending a Norm-Enforcing School Pay Off?. Urban Rev 42, 422–440 (2010). https://doi.org/10.1007/s11256-009-0141-2
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11256-009-0141-2