Abstract
Probing the causes of effects, the contextual analyses of the previous four chapters exemplified Coleman’s (1964, 116, 189–240) emphasis on quantifying the implied causal effects of several predetermining variables on a response. Probing the effects of a cause (Holland 1986, 945; Morgan and Winship 2007, 280–282), these chapters on evaluative research exemplify Rubin’s (1974) emphasis on studying the effects of a manipulated cause—here, some consequences of comprehensive school reform (CSR) on measures of achievement. Because these chapters study repeated measures on the same schools, they are similar to studies that analyze panel data. But, because of the students’ mobility out of and into these schools, and other changes in the compositions of the schools, these chapters are best viewed as analyzing repeated quasi-panel data.
This is a preview of subscription content, log in via an institution to check access.
Access this chapter
Tax calculation will be finalised at checkout
Purchases are for personal use only
Notes
- 1.
Etzioni (1968, 1983, 87–88) defines malleability as the extent to which a policy variable is movable or changeable. Policy researchers rank factors according to their malleability and focus their research on the more malleable factors and on the constraints that impede the malleability of the less changeable factors.
- 2.
- 3.
Coleman’s many educational policy studies express his pragmatic activism; Smith (2006, i–xi) lists and briefly discusses his educational studies.
- 4.
Ravitch (2010), believing that both schools and students are changeable, advocates the creation of a widely shared core curriculum that would engender multiple benefits. Hirsch (2010, 18) summarizes some of her thoughts about core curriculums as follows:
It would assure the cumulative organization of knowledge by all students, would help overcome the notorious achievement gaps between racial and ethnic groups. It would make the creation of an effective teaching force much more feasible, because it would become possible to educate American teachers in the well-defined, wide-ranging subjects they would be expected to teach—thus educating students and teachers simultaneously.
It would also foster the creation of much better teaching materials, with more substance; and it would solve the neglected problem of students (mostly low-income ones) who move from one school to another, often in the middle of the school year. It would, in short, offer American education the advantages enjoyed by high-performing school systems in the rest of the world, which far outshine us in the quality and fairness of their results.
- 5.
Similarly, reading across the two columns, for time = 0, the difference d'(0) between the target group (1) and the comparison group (0) = the coefficient for the target group. For time = 1, the difference between the target group (1) and the comparison group (0) = d'(1) = the coefficient for the target group and the interaction effect, target × post. The difference between these two differences is again the program effect coefficient:
$$ \begin{array}{lll} \delta { = }d^\prime (1) - d^\prime (0) = {\hbox{Target (1) + Target}} \times {\hbox{Post Period(1)}} - {\hbox{Target (1)}} \hfill \\& = {\hbox{Target}} \times {\hbox{Post}}\;{\hbox{Period(1) = Target}} \times {\hbox{Post}} \hfill \\\end{array} $$ - 6.
Jill Tao of the SAS Institute has graciously provided the following SAS program that will create an illustrative SAS data set and will then call Proc Mixed to calculate the treatment effect as an interaction and as a difference-in-differences. Chapter 12 explains in some detail the logic of similar SAS programs for calculating program effects as differences-in-differences, also see endnote 11 of this chapter for an illustrative application of Proc Glimmix and logistic regression from a DID perspective.
*Example of SAS code for a Difference-in-Difference Design for a Continuous Response Variable;
*This code creates the SAS data set;
data test;
do i = 1 to 20;
trt = round(ranuni(2345), 1);
period = round(ranuni(1234), 1);
y = trt + period + rannor(2687);
p = exp(-y)/(1 + exp(-y));
if ranuni(2356) > p then y2 = 1; else y2 = 0;
output;
end;
run;
proc print data = test;
run;
*This code calls Proc Mixed to estimate the treatment effect
on the continuous response variable;
proc mixed data = test;
class trt period;
model y = trt period trt*period/s;
*This statement will calculate the difference between
the means in the target group;
estimate ‘1,1 - 1,0’ period -1 1 trt*period 0 0 -1 1;
*This statement will calculate the difference between
the means in the comparison group;
estimate ‘0,1 - 0,0’ period -1 1 trt*period -1 1;
*This statement will calculate the difference-in-differences;
estimate ‘(1,1–1,0) versus (0,1–0,0)’ trt*period 1 -1 -1 1;
run;
- 7.
The acronym PATE refers to the population average treatment effect. The PATE estimand is the population expectation of the unit-level causal effect, Y i (1) − Y i (0), which is δPATE = E[Y i (1) − Y i (0)]. CATE refers to “conditional on the covariates in the sample” average treatment effect. The CATE estimand is δCATE = 1/N \( \sum\limits_{i = 1}^N E \)[Y i (1) − Y i (0) | X i ] (Imbens and Wooldridge 2009, 15–17; Rubin 1974, 689–693).
- 8.
Willard Manning (circa 1990, personal communication) has pointed out that when the data are pooled, the intercept is based on all of the cases, even though the various cells in the design may actually have different numbers of cases. Thus, there is this counterfactual aspect to this procedure: At time 0, all of the cases are placed in the (0, 0) cell and experience the null treatment. Then, all of the cases are placed in the treatment group cell for that time period (0, 1) and they experience the baseline difference between the two groups. Then, at time 1 all of the cases are placed in the comparison group and are exposed to the effect of time (1, 0). Then, all of the cases are placed in the target group at that time (1, 1) and experience the effect of time, the target program, and their interaction; the latter is the program effect coefficient.
- 9.
At a meeting of the Boston Chapter of the American Statistical Association (May 4, 2010), Til Stürmer (an epidemiologist teaching at the University of North Carolina, Chapel Hill) reported the results of a computer simulation study that clarifies criteria for the selection of potential covariates in the calculation of propensity scores: Antecedent variables that influence both treatment and outcome should be included because they reduce spurious associations. Instrumental variables that influence only the treatment should be excluded because they lead to separation. Variables that influence only the outcome can be included because they improve the efficiency of the estimation.
- 10.
Imbens and Wooldridge (2009, 33) note several limitations to the use of propensity scores as covariates in regression models: the propensity score does not have substantive meaning; units with propensity scores of 0.45 and 0.50 are more similar than units with scores of 0.01 and 0.06; logit and probit models will produce similar scores in the middle range of the data but different scores for extreme observations; and propensity scores close to 0 or 1 are outliers that may have too much impact in weighting schemes.
- 11.
The SAS code below by Jill Tao of SAS implements the DID design for a binomial response variable. Chapter 12 explicates similar models in depth.
*Example of SAS code for a Difference-in-Difference Design for a
Dichotomous Response Variable;
*This code creates the SAS data set;
data test;
do i = 1 to 20;
trt = round(ranuni(2345), 1);
period = round(ranuni(1234), 1);
y = trt + period + rannor(2687);
p = exp(-y)/(1 + exp(-y));
if ranuni(2356) > p then y2 = 1; else y2 = 0;
output;
end;
run;
proc print data = test;
run;
*This code calls Proc Glimmix to estimate the treatment effect
on the dichotomous response variable. It calculates a logistic
regression model and converts the logit scale means to odds ratios and proportions;
proc glimmix data = test;
class trt period;
model y2 (event = “1”) = trt period trt*period/s dist = binary link = logit;
*This statement will calculate the difference between
the means in the target group on the logit scale etc.;
estimate ‘1,1 - 1,0’ period -1 1 trt*period 0 0 -1 1/or ilink cl;
*This statement will calculate the difference between
the means in the comparison group on the logit scale, etc.;
estimate ‘0,1 - 0,0’ period -1 1 trt*period -1 1/or ilink cl;
*This statement will calculate the difference-in-differences on the logit scale, etc.;
estimate ‘(1,1–1,0) versus (0,1–0,0)’ trt*period 1 -1 -1 1/or ilink cl;
run;
References
Achen, Christopher H. 1986. The statistical analysis of quasi-experiments. Berkeley: University of California Press.
Aladjem, Daniel K., Kerstin Carlson LeFloch, Yu Zhang, Anja Kurki, Andrea Boyle, James E. Taylor, Suzannah Herrmann, Kazuaki Uekawa, Kerri Thomsen, and Olatokunbo Fashola. September 2006. Models matter—the final report of the national longitudinal evaluation of comprehensive school reform. Washington, DC: American Institutes of Research.
Berends, Mark, Susan Bodily, and Sheila Nataraj Kirby. 2002. Facing the challenges of whole school reform: New American schools after a decade. Santa Monica, CA: RAND.
Borman, Geoffrey D., Gina M. Hewes, L.T. Overman, and S. Brown. 2003. Comprehensive school reform and achievement: A meta-analysis. Review of Educational Research 73: 125–230.
Burd-Sharps, Sarah, Kristen Lewis, and Eduardo Borges Martins. 2008. The measure of America: American human development report 2008–2009. New York: Columbia University Press.
Coleman, James S. 1964. Introduction to mathematical sociology. New York: Free Press.
Coleman, James S. 1972. Policy research in the social sciences. Morristown, NJ: General Learning Press.
Comprehensive School Reform Quality Center. 2006. CSQR Center report on elementary school comprehensive school reform models. Washington, DC: American Institutes for Research.
Etzioni, Amitai. 1968. The active society: A theory of societal and political process. New York: Free Press.
Firth, David. 1993. Bias reduction of maximum likelihood estimates. Biometrika 80: 27–38.
Fraser, Steven (ed.). 1995. The bell curve wars: Race, intelligence, and the future of America. New York: Basic Books.
Gardner, Howard. 1983. Frames of mind: The theory of multiple intelligences. New York: Basic Books.
Goldberg, Bruce, and John Richards. 1996. The Co-NECT design for school change. In Bold plans for school restructuring: The New American Schools designs, ed. Samuel C. Stringfield, Steven M. Ross, and Lana Smith, 75–108. Mahwah, NJ: Lawrence Erlbaum.
Herrnstein, Richard J., and Charles Murray. 1994. Bell curve: Intelligence and class structure in American Life. New York: Free Press.
Hirsch Jr., E.D. 2010. How to save the schools. The New York Review of Books 67(8, May 13): 16–19.
Holland, Paul. 1986. Statistics and causal inference (with comments). Journal of the American Statistical Association 81: 945–970.
Imbens, Guido I., and Jeffrey M. Wooldridge. 2009. Recent developments in the econometrics of program evaluation. Journal of Economic Literature 47(1): 5–86.
Jacoby, Russel, and Naomi Glauberman (eds.). 1995. The bell curve debate: History, documents, opinions. New York: Times Books.
Murray, Charles. 2008. Real education: Four simple truths for bringing America’s schools back to reality. New York: Crown Forum.
Myrdal, Gunnar. [1944] 1964. An American dilemma. New York, NY: McGraw-Hill.
Ravitch, Diane. 2010. The death and life of the great American school system: How testing and choice are undermining education. New York, NY: Basic Books.
Rosenbaum, Paul R., and Donald B. Rubin. [1983] 2006. The central role of the propensity score in observational studies for causal effects. In Matched sampling for causal effects, ed. Donald B. Rubin, 170–184. New York: Cambridge University Press.
Rubin, Donald B. 1974. Estimating causal effects of treatments in randomized and non randomized studies. Journal of Educational Psychology 66: 688–701.
Smith, Robert B. 2006. Introduction to James S. Coleman, the mathematics of ollective action. New Brunswick, NJ: AldineTransaction. vii–li.
Weber, Max. 1947. The theory of social and economic organization, translated by A. M. Henderson and Talcott Parsons. Glencoe, IL: The Free Press.
Wooldridge, Jeffrey M. 2006. Introductory econometrics: A modern approach, 3rd ed. Mason, OH: Thomson/South-Western.
Author information
Authors and Affiliations
Corresponding author
Rights and permissions
Copyright information
© 2011 Springer Science+Business Media B.V.
About this chapter
Cite this chapter
Smith, R.B. (2011). Cause and Consequences. In: Multilevel Modeling of Social Problems. Springer, Dordrecht. https://doi.org/10.1007/978-90-481-9855-9_10
Download citation
DOI: https://doi.org/10.1007/978-90-481-9855-9_10
Published:
Publisher Name: Springer, Dordrecht
Print ISBN: 978-90-481-9854-2
Online ISBN: 978-90-481-9855-9
eBook Packages: Humanities, Social Sciences and LawSocial Sciences (R0)