Accounting for Taste: Using Propensity Score Methods to Evaluate the Documentary Film, Waiting for “Superman”

  • Johanna Blakley
  • Sheena Nahm


Evaluating the impact of the arts can be difficult for a variety of reasons, including the cost of survey administration and the issue of selection bias. However, recent media impact studies offer models for economically evaluating programming that appeals to particular personal tastes, supplying methodologies that can be applied to measure the impact of the arts. This chapter explores a study of the documentary film Waiting for “Superman,” in which propensity score matching (PSM) was used to identify covariates for propensity to be exposed to the film. Survey respondents are matched based on scores attached to degree of propensity to be exposed; matched propensity scores that differ according to exposure allow for evaluation of impact while controlling for selection bias.


Propensity score matching Media impact Social networks 


  1. Bandura, A. (1977). Social learning theory. Englewood Cliffs, NJ: Prentice Hall.Google Scholar
  2. Bandura, A., Ross, D., & Ross, S. A. (1963). Imitation of film-mediated aggressive models. Journal of Abnormal and Social Psychology, 66, 3–11.CrossRefGoogle Scholar
  3. Belfiore, E., & Bennett, O. (2010). The social impact of the arts: An intellectual history. New York: Palgrave Macmillan.Google Scholar
  4. Bertrand, J. T., O’Reilly, K., Denison, J., Anhang, R., & Sweat, M. (2006). Systematic review of the effectiveness of mass communication programs to change HIV/AIDS-related behaviors in developing countries. Health Education Research, 21(4), 567–597.CrossRefGoogle Scholar
  5. Brown, A. S., & Novak, J. L. (2007). Assessing the intrinsic impacts of a live performance. San Francisco, CA: WolfBrown.Google Scholar
  6. Centre for AIDS Development, Research and Evaluation. (2005). Tsha Tsha: Key findings of the evaluation of episodes 1–26. Retrieved January 18, 2011, from
  7. Cochran, W. G. (1968). The effectiveness of adjustment by subclassification in removing bias in observational studies. Biometrics, 24, 295–313.CrossRefGoogle Scholar
  8. Conquergood, D. (1988). Health theater in a Hmong refugee camp: Performance, communication and culture. TDR: Journal of Performance Studies, 32, 174–208.CrossRefGoogle Scholar
  9. Danielsson, S. (2004). The propensity score and estimation in nonrandom surveys: An overview. Retrieved January 18, 2011, from
  10. Dehejia, R. H., & Wahba, S. (2002). Propensity score-matching methods for nonexperimental causal studies. The Review of Economics and Statistics, 84(1), 151–161.CrossRefGoogle Scholar
  11. Do, M. P., & Kincaid, D. L. (2006). Impact of an entertainment-education television drama on health knowledge and behavior in Bangladesh: An application of propensity score matching. Journal of Health Communication, 11, 301–325.CrossRefGoogle Scholar
  12. Galloway, S. (2009). Theory-based evaluation and the social impact of the arts. Cultural Trends, 18(2), 125–148.CrossRefGoogle Scholar
  13. Landrum, M. B., & Ayanian, J. Z. (2001). Causal effect on ambulatory specialty care on mortality following myocardial infarction: A comparison of propensity score and instrumental variable analyses. Health Services & Outcomes Research Methodology, 2, 221–245.CrossRefGoogle Scholar
  14. Marcus, P. M., Huang, G. C., Beck, V., & Miller, M. J. (2010). The impact of a primetime cancer storyline: From individual knowledge and behavioral intentions to policy-level changes. Journal of Cancer Education, 25(4), 484–489.CrossRefGoogle Scholar
  15. Morgan, S. E., Movius, L., & Cody, M. J. (2009). The power of narratives: The effect of entertainment television organ donation storylines on the attitudes, knowledge, and behaviors of donors and nondonors. Journal of Communication, 59(1), 135–151.CrossRefGoogle Scholar
  16. Murphy, S. T., Hether, H. J., Felt, L. J., & Buffington, S. C. (2012). Public diplomacy in prime time: Exploring the potential of entertainment education in international diplomacy. American Journal of Media Psychology, 5, 5–32.Google Scholar
  17. Nahm, S., Le, K., Buffington, S., Schiman, N., Raider, S., & Resko, S. (2010). Engaging youth through entertainment education through partnership and collaboration. Cases in Public Health Communication & Marketing, 4(2), 57–78.Google Scholar
  18. National Endowment for the Arts and Arts & Humanities Research Council. (2014). Measuring cultural engagement: A quest for new terms, tools, and techniques.Google Scholar
  19. Posner, M. A., Ash, A. S., Freund, K. M., Moskowitz, M. A., & Shwartz, M. (2001). Comparing standard regression, propensity score matching, and instrumental variables methods for determining the influence of mammography on stage of diagnosis. Health Services & Outcomes Research Methodology, 2, 279–290.CrossRefGoogle Scholar
  20. Rosenbaum, P. R., & Rubin, D. B. (1983). The central role of the propensity score in observational studies for causal effects. Biometrika, 70(1), 41–55.CrossRefGoogle Scholar
  21. Rubin, D. B. (2001). Using propensity scores to help design observational studies: Application to the tobacco litigation. Health Services & Outcomes Research Methodology, 2, 169–188.CrossRefGoogle Scholar
  22. Singhal, A. (2004). Empowering the oppressed through participatory theater. Investigacion y Desarrollo, 12(1), 138–163.Google Scholar
  23. Singhal, A., & Rogers, E. M. (1989). Prosocial television for development in India. In R. E. Rice & C. K. Atkin (Eds.), Public communication campaigns (2nd ed., p. 331). Newbury Park, CA: Sage.Google Scholar
  24. Singhal, A., & Rogers, E. M. (1999). Entertainment-education: A communication strategy for social change. Mahwah, NJ: Lawrence Erlbaum Associates.Google Scholar
  25. TakePart. (2010). Waiting for Superman impact. Retrieved September 12, 2016, from
  26. Winkelmayer, W. C., & Kurth, T. (2004). Propensity scores: Help or hype? Nephrology, Dialysis, Transplantation, 19, 1671–1673.CrossRefGoogle Scholar

Copyright information

© The Author(s) 2018

Authors and Affiliations

  • Johanna Blakley
    • 1
  • Sheena Nahm
    • 2
  1. 1.Annenberg School for Communication and JournalismUniversity of Southern CaliforniaLos AngelesUSA
  2. 2.Anthropology & SociologyThe New School for Public EngagementNew YorkUSA

Personalised recommendations