Abstract
Sociologists of (e)valuation have devoted considerable attention to understanding differences in evaluative practices across a number of fields. Yet, little is understood about how individuals learn about and navigate multivalent valid group styles within a single setting. As a social phenomenon, many accept how central processes of evaluation are to everyday life. Accordingly, scholars have attempted to link research on evaluation to processes of inequality. Nevertheless, the sociology of evaluation only has tenuous, often implicit connections to literature on inequality and disadvantage. This article addresses these two gaps. Drawing on over two-hundred hours of ethnographic fieldwork in an urban high school debate league, twenty-seven semi-structured interviews with league judges, and archival data, we illustrate how high school policy debate judges employ evaluative frames and link them to the implementation of evaluative practices in a disadvantaged setting. We show that the cultural meanings that emerge within the evaluation process—in this case, urban uplift and competition—stem from the conflicted context in which evaluation is occurring. We also make a first step toward applying the conceptual tools within the sociology of evaluation to a disadvantaged setting, and more broadly, suggest that micro-processes of evaluation are important to the study of urban inequality.
This is a preview of subscription content, access via your institution.
Notes
To some extent, Lamont (2009) recognizes evaluative frames without labeling them as such: “[D]efinitions of excellence that panelists employ…are influenced by their individual proclivities, and by various facets of their identity and of their intellectual and social trajectories” (58). However, Lamont continues that the “epistemological criteria that panelists value most… resonate with the definition of excellence that prevails in their specific discipline” (58). We demonstrate that, under certain circumstances, awareness of a discipline’s dominant criteria for excellence is less salient.
Here, we refer to Swidler’s (1986) insight that ideologies more directly drive action in “unsettled cultural periods” or “periods of social transformation” (278).
This concept helps to illuminate why studying evaluative frames is important. If the evaluative culture and the set of evaluative practices within urban debate were more settled, we would not expect culture to drive action. Evaluative practice would be dictated primarily by long-established tradition, as perhaps is true in national debate. Yet, since the league’s evaluative culture is multivalent, judges’ framing of the activity matters much more for understanding how judges make decisions about rounds and send messages to students about valued forms of engagement with the activity.
Sampson’s work on neighborhood selection most closely invokes the concept of evaluation.
Because teams are large, training resources are distributed disproportionately toward newer debaters. The average technical skill level among the debaters is thus lower than one might have observed in Fine’s world, and likely in other, more selective urban debate leagues (field notes). Further, debaters being inconsistently evaluated on technical versus life skills may contribute to their lower average technical acumen.
These numbers are based on the number of students who attended at least one tournament in the season. Although we believe these numbers hold true, a smaller group of students consistently participates in the activity. In addition, the racial numbers obscure important diversity: For example, the top debaters from the 2011–2012 season included three black women, all of immigrant descent from either Africa or the Caribbean; two African-American males; two white males; one Asian-American male; one Arab-American male; one white female; and two Latina females (one of whom is also African-American). Several students who would be classified as “white” for numerical purposes are recent immigrants or of Middle Eastern descent.
Of course, adoption of a frame does not always lead to a specific set of evaluations; we show a strong association, but make no causal claim, between frames and practices. Furthermore, one set of evaluative practices does not necessarily lead to a certain outcome. Technical judges often disagree with each other, and life-skills judges frequently disagree with one another as well.
These distinctions are important because many respondents reported that they evaluated debaters differently depending on the division and the stage of the tournament. Also, the process of judging elimination rounds is different from preliminary rounds, as we show later.
Lincoln-Douglas, or “LD,” is a debate event founded in 1979 as an alternative to policy debate. In LD, individual students, rather than two-person teams, debate balancing competing values (e.g., liberty and equality) instead of policy.
One example of interpretive consensus comes from our findings on mutual evaluation among league judges, presented below. Both researchers observed some judges’ highly presentational nature, but could not determine how to make sense of this behavior independently. Were these judges actually performing? If so, for whom? After triangulating our observations with interview data and additional participant observation, we agreed that some judges engage in a kind of performance to bolster their legitimacy in the face of other, more technical judges.
Distinguishing between 20 years ago and less than 20 years ago is important because policy debate evolved from emphasizing traditional communication skills to emphasizing technical argumentation, often at the cost of persuasive speaking. See Fine (2001b).
Matching respondents by interviewer racial/ethnic background was beneficial in many respects, most notably in facilitating respondents’ comfort with speaking openly. Lee, a white lawyer, felt secure enough with Asad that he openly used racial slurs when recounting stories about his interactions with some of the debaters. Likewise, Davin, an African-American former league debater, told Bell that he “hated” whites before getting involved with urban debate.
Although we recognize the complexity of each respondent’s perspective, we believe the differences between these two evaluative frames and how they link to evaluative practices is more analytically salient than the fine distinctions and complexities between individual judges. The broader pattern suggests that this typology is useful for thinking about evaluation in this context (see Barton 1955; Lazarsfeld 1937; Menger 1883/1996; Weber 1922/1978).
Although these statements are laced with racial meaning, it is worth noting that one of the students whose “aggression” sparked the most controversy during interviews is white.
We are confident that evaluative frames and evaluative practices are two separate phenomena because, while there are consistent links between both, there are important divergences. Some uplift-frame judges employ technical standards, while some competition judges utilize life-skills standards. All of our competition-life-skills respondents were inexperienced with policy debate. For example, Annaliese, a competition-life-skills judge, decided after a few rounds that she had been judging “wrong” and attempted to modify her evaluative practices to more closely match the technical model. If evaluative frames and practices were the same phenomenon, we would expect them to correlate perfectly; we find variation and conclude that they exist separately.
Most judges are unpaid volunteers, but some league alumni are paid $100 per tournament to remain present for all rounds. Some coaches and debaters criticize this practice because the alumni may view themselves as more technically skilled than they actually were or bring personal feelings about their former opponents into their judgments. Others point to benefits: it keeps alumni connected to the league, ensures that judges familiar with technical aspects of debate will be available to evaluate elimination rounds, and adds diversity to the judging corps.
Omitted from our analysis are non-volunteers, those who might not only criticize the urban uplift frame but also the league’s emphasis on middle–class (“white”) values. However, some judges—even those who adopt an urban uplift frame—are still concerned that other volunteers have a patronizing or condescending view toward students.
The Tournament of Champions (TOC) is an elite, highly competitive policy debate championship. Debaters must qualify by achieving a certain level of success at national-level debate tournaments in order to earn a “bid” to the TOC. Two bids are required to qualify.
Fine explains that one of debaters’ favorite pastimes is complaining about judges. Although most judges on the national debate circuit are well-paid and fairly expert in policy debate’s subcultural rules, there are usually a few who are less well-versed in the technical style of judging. The TOC touts the technical skill of its judging pool as one reason for the tournament’s superiority: it “strives to offer the national high school debate community the highest quality judging, impartial tournament officials, and a friendly, congenial atmosphere” (archival materials). Of course, debaters sometimes complain when a technically skilled judge votes for the other team despite having good reasons to do so. Part of community building in any debate division often includes sharing stories about particularly egregious judges.
References
Anderson, Susannah, and Briana Mezuk. 2012. Participating in a policy debate program and academic achievement among at-risk adolescents in an urban public school district: 1997–2007. Journal of Adolescence 35: 1225–1235.
Barton, Allen H. 1955. The concept of property-space in social research. In The language of social research, ed. Paul F. Lazarsfeld and Morris Rosenberg, 40–53. New York: Free Press.
Becker, Howard S. 1982. Art worlds. Berkeley: University of California Press.
Beljean, Stefan. 2012. Sociological research on evaluation: A review and synthesis of existing literature. Unpublished Manuscript, Department of Sociology, Harvard University.
Benford, Robert D., and David A. Snow. 2000. Framing processes and social movements: an overview and assessment. Annual Review of Sociology 26: 611–639.
Benjamin, Lehn M. 2008. Evaluator’s role in accountability relationships: measurement technician, capacity builder or risk manager? Evaluation 14: 323–343.
Blank, Grant. 2007. Critics, ratings, and society: The sociology of reviews. Lanham: Rowman & Littlefield.
Boltanski, Luc, and Laurent Thévenot. 1983. Finding one’s way in social space: a study based on games. Social Science Information 22: 631–680.
Boltanski, Luc and Laurent Thévenot. 2006. On justification: Economies of worth (trans: Porter, C.). Princeton: Princeton University Press.
Bourdieu, Pierre. 1993. The field of cultural production. New York: Columbia University Press.
Couch, Carl J. 1992. Toward a formal theory of social processes. Symbolic Interaction 15: 117–134.
Creed, W.E. Douglas, Jeffrey A. Langstraat, and Maureen A. Scully. 2002. A picture of the frame: frame analysis as technique and as politics. Organizational Research Methods 5: 34–55.
Desrosières, Alain. 1998. The politics of large numbers: A history of statistical reasoning. Cambridge: Harvard University Press.
Edin, Kathryn and Maria Kefalas. 2005. Promises I can keep: Why poor women put motherhood before marriage. Berkeley: University of California Press.
Eliasoph, Nina, and Paul Lichterman. 2003. Culture in interaction. American Journal of Sociology 108: 735–794.
Espeland, Wendy, and Michael Sauder. 2007. Rankings and reactivity: how public measures recreate social worlds. American Journal of Sociology 113: 1–40.
Espeland, Wendy, and Mitchell Stevens. 1998. Commensuration as a social process. Annual Review of Sociology 24: 313–343.
Fine, Gary Alan. 2001a. Difficult reputations: Collective memories of the evil, inept, and controversial. Chicago: University of Chicago Press.
Fine, Gary Alan. 2001b. Gifted tongues: High school debate and adolescent culture. Princeton: Princeton University Press.
Fligstein, Neil, and Doug McAdam. 2012. A theory of fields. Oxford: Oxford University Press.
Fourcade, Marion. 2011. Cents and sensibility: economic valuation and the nature of “nature. American Journal of Sociology 116: 1721–1777.
Friedland, Roger, and Robert R. Alford. 1991. Bringing society back in: Symbols, practices, and institutional contradictions. In The new institutionalism in organizational analysis, ed. Walter W. Powell and Paul J. DiMaggio, 232–263. Chicago: University of Chicago Press.
Ghaziani, Amin, and Marc J. Ventresca. 2005. Keywords and cultural change: frame analysis of business model public talk, 1975–2000. Sociological Forum 20: 523–559.
Goffman, Erving. 1959. The presentation of self in everyday life. New York: Doubleday.
Goffman, Erving. 1974. Frame analysis: An essay on the organization of experience. Cambridge: Harvard University Press.
Harding, David J. 2009. Violence, older peers, and the socialization of adolescent boys in disadvantaged neighborhoods. American Sociological Review 74: 445–464.
Jack, Anthony Abraham. . Culture shock revisited: The social and cultural contingencies to class marginality. Sociological Forum 29(2) Forthcoming.
Jr, Young, and A. Alford. 2004. 13 experiences in ethnographic interviewing about race. In Researching race and racism, ed. Martin Blumer and John Solomos, 187–202. London: Routledge.
Koretz, Daniel M. 2008. Measuring up: What educational testing really tells us. Cambridge: Harvard University Press.
Lamont, Michèle. 2000. The dignity of working men: Morality and the boundaries of race, class, and immigration. New York: Russell Sage Foundation.
Lamont, Michèle. 1992. Money, morals and manners: The culture of the French and the American upper-middle class. Chicago: University of Chicago Press.
Lamont, Michèle. 2009. How professors think: Inside the curious world of academic judgment. Cambridge: Harvard University Press.
Lamont, Michèle. 2012. Toward a comparative sociology of valuation and evaluation. Annual Review of Sociology 38: 201–221.
Lamont, Michèle, and Katri Huutoniemi. 2011. Comparing customary rules of fairness: Evaluative practices in various types of peer review panels. In Social knowledge in the making, ed. Charles Camic, Neil Gross, and Michèle Lamont, 202–232. Chicago: University of Chicago Press.
Lamont, Michèle, and Mario Luis Small. 2008. How culture matters: Enriching our understandings of poverty. In The colors of poverty: Why racial and ethnic disparities persist, ed. Ann Chih Lin and David R. Harris, 76–102. New York: Russell Sage Foundation.
Lamont, Michèle, Stefan Beljean, and Matthew Clair. 2014. What is missing? Culture and causal pathways to inequality. Socioeconomic Review Forthcoming.
Lareau, Annette. 2003. Unequal childhoods: Class, race, and family life. Berkeley: University of California Press.
Lazarsfeld, Paul F. 1937. Some remarks on the typological procedures in social research. Zeitschrift für Sozialforschung 6: 119–139.
Lens, Vicki. 2009. Confronting government after welfare reform: moralists, reformers, and narratives of (ir)responsibility at administrative fair hearings. Law & Society Review 43: 563–592.
Lom, Stacy. 2010. Evaluative cultures in two art worlds. Paper presented at the Annual Meeting of the American Sociological Association. Atlanta, GA: Hilton Atlanta and Atlanta Marriott Marquis.
Lowry, Kem. 1993. Evaluation of community justice programs. In The possibility of popular justice: A case study of community mediation, ed. Sally Engle Merry and Neil Milner, 89–123. Ann Arbor: University of Michigan Press.
MacKenzie, Donald. 2011. The credit crisis as a problem in the sociology of knowledge. American Journal of Sociology 116: 1778–1841.
MacKenzie, Donald. 2011a. Evaluation cultures? On invoking “culture” in the analysis of behaviour in financial markets. Working Paper. http://www.sps.ed.ac.uk/__data/assets/pdf_file/0007/64564/EvalCults11.pdf. Accessed 21 November 2013.
Mallard, Grégoire, Michèle Lamont, and Joshua Guetzkow. 2009. Fairness as appropriateness: negotiating epistemological differences in peer review. Science, Technology & Human Values 34: 573–606.
May, Buford, A. Reuben, and Mary Pattillo-McCoy. 2000. Do you see what I see: examining a collaborative ethnography. Qualitative Inquiry 6: 65–87.
McLean, Paul D. 1998. A frame analysis of favor-seeking in the renaissance: agency, networks, and political culture. American Journal of Sociology 104: 51–91.
Menger, Carl. 1883/1996. Investigations into the method of the social sciences. Auburn: Ludwig von Mises Institute.
Mezuk, Briana, Irina Bondarenko, Suzanne Smith, and Eric Tucker. 2011. Impact of participating in a policy debate program on academic achievement: evidence from the Chicago urban debate league. Educational Research and Reviews 6: 622–635.
Pager, Devah. 2007. Marked: Race, crime, and finding work in an era of mass incarceration. Chicago: University of Chicago Press.
Polikoff, Morgan S. 2012. Instructional alignment under no child left behind. American Journal of Education 118: 341–368.
Porter, Theodore M. 1995. Trust in numbers: The pursuit of objectivity in science and public life. Princeton: Princeton University Press.
Power, Michael. 1997. The audit society: Rituals of verification. Oxford: Oxford University Press.
Rivera, Lauren. 2011. Ivies, extracurriculars, and exclusion: elite employers’ use of educational credentials. Research in Social Stratification and Mobility 29: 71–90.
Roberts, Dorothy. 2002. Shattered bonds: The color of child welfare. New York: Basic Civitas Books.
Sampson, Robert J. 2012. Great American city: Chicago and the enduring neighborhood effect. Chicago: University of Chicago Press.
Small, Mario Luis. 2002. Culture, cohorts, and social organization theory: understanding local participation in a Latino housing project. American Journal of Sociology 108: 1–54.
Small, Mario Luis, David J. Harding, and Michèle Lamont. 2010. Reconsidering culture and poverty. Annals of the American Academy of Political and Social Science 629: 6–27.
Smith, Sandra Susan. 2005. “Don’t put my name on it”: social capital activation and job-finding assistance among the black urban poor. American Journal of Sociology 111: 1–57.
Smith, Sandra Susan. 2007. Lone pursuit: Distrust and defensive individualism among the black poor. New York: Russell Sage Foundation.
Snow, David A., E. Burke Rochford Jr., Worden Steven K, and Benford Robert D. 1986. Frame alignment processes, micromobilization, and movement participation. American Sociological Review 51: 464–481.
Stevens, Mitchell. 2007. Creating a class: College admissions and the education of elites. Cambridge: Harvard University Press.
Swidler, Ann. 1986. Culture in action: symbols and strategies. American Sociological Review 51: 273–286.
Tach, Laura M. 2009. More than bricks and mortar: neighborhood frames, social processes, and the mixed-income redevelopment of a public housing project. City & Community 8: 269–299.
Thorton, Patricia H., William Ocasio, and Michael Lounsbury. 2012. The institutional logics perspective: A new approach to culture, structure, and process. Oxford: Oxford University Press.
Tilly, Charles. 2008. Explaining social processes. Boulder: Paradigm Publishers.
Waldinger, Roger. 1996. Still the promised city? African Americans and immigrants in postindustrial New York. Cambridge: Harvard University Press.
Waters, Mary C. 1999. Black identities: West Indian immigrant dreams and American realities. New York: Russell Sage Foundation.
Watkins-Hayes, Celeste. 2009. The new welfare bureaucrats: entanglements of race, class, and policy reform. Chicago: University of Chicago Press.
Weber, Max. 1922/1978. Economy and society: An outline of interpretive sociology. Berkeley: University of California Press.
Williams, Joseph J., and David R. Mandel. 2007. Do evaluation frames improve the quality of conditional probability judgment? In Proceedings of the 29th Annual Meeting of the Cognitive Science Society, ed. D.S. McNamara and J.G. Trafton, 1653–1658. Mahwah: Erlbaum.
Zelizer, Viviana. 2005. The purchase of intimacy. Princeton: Princeton University Press.
Acknowledgments
We thank Stefan Beljean, Rachel Bradshaw, Matthew Clair, Caitlin Daniel, Nicole Deterding, Anthony Jack, Michèle Lamont, Michael Sauder, and the anonymous reviewers for their helpful comments and suggestions on previous versions of this paper.
Author information
Authors and Affiliations
Corresponding author
Additional information
Asad L. Asad and Monica C. Bell contributed equally to the preparation of this article.
Electronic supplementary material
Below is the link to the electronic supplementary material.
ESM 1
(PDF 67 kb)
Rights and permissions
About this article
Cite this article
Asad, A.L., Bell, M.C. Winning to Learn, Learning to Win: Evaluative Frames and Practices in Urban Debate. Qual Sociol 37, 1–26 (2014). https://doi.org/10.1007/s11133-013-9269-1
Published:
Issue Date:
DOI: https://doi.org/10.1007/s11133-013-9269-1
Keywords
- Evaluation
- Education
- Inequality
- Urban sociology
- Culture
- Policy debate