Is Plea Bargaining in the “Shadow of the Trial” a Mirage?
- First Online:
- Cite this article as:
- Bushway, S.D. & Redlich, A.D. J Quant Criminol (2012) 28: 437. doi:10.1007/s10940-011-9147-5
- 526 Views
It has been well established that a “plea discount” or “trial penalty” exists, such that defendants who plead guilty receive significant sentencing discounts relative to what they would receive if convicted at trial. Theorists argue that the exact value of this plea discount is determined by bargaining “in the shadow of a trial,” meaning that plea decision-making is premised on the perceived probable outcome of a trial. In trials, the strength of the evidence against defendants greatly impacts the probability of conviction. In the present study, we estimate the probability of conviction at the individual level for those who pled guilty. We find that, contrary to the shadow of the trial model, evidentiary factors either do not impact or negatively impact the probability of conviction, which stands in stark contrast to the impact evidence has at trials. These findings suggest that plea bargain decision-making may not occur in the shadow of the trial.
KeywordsPlea bargainingStrength of evidenceSentencingProsecutorial discretionCounterfactual
Recent empirical work by Ulmer and colleagues has focused criminological attention on the substantial observed variation in the plea discounts offered to defendants who plead guilty (Ulmer and Bradley 2006; Ulmer et al. 2010). This interest in understanding differences in sentences among people who plead guilty is a natural outgrowth of a long tradition of sentencing research in criminology which focuses on identifying and explaining disparate outcomes for individuals who look similar from a legal perspective (for a review, see Spohn 2000). Criminological theorists have offered explanations for why there might be variation in the plea discount at the court level (Eisenstein and Jacob 1977; Eisenstein et al. 1987; Nardulli et al. 1988; Ulmer 1997), but little theoretical attention has been paid to why there might be variation at the defendant level within courts. In contrast, legal and economic theorists have developed a simple model that can predict variation in the size of the plea discount. This model is known as bargaining in the “shadow of the trial” (Landes 1971; Mnookin and Kornhauser 1979; Nagel and Neef 1979; Bibas 2004). In this paper, we formally introduce this idea to a criminological audience, and then present a direct test of this theory using individual case-level data.
A rational, risk-neutral defendant would never accept a plea deal that was more than what he expected if he went to trial, which implies that the plea will be less than or equal to the discounted value of the trial outcome (X ≤ P(Y)). But, on the other hand, a rational district attorney would never offer a person a plea deal that was less than the discounted trial sentence, since she knows that the defendant will accept the discounted trial sentence, P(Y). The result, which is the driving result of the “shadow of the trial” model, is that X = P(Y), or, that the sentence at plea (X) will be equal to the sentence at trial (Y) discounted by probability of conviction at trial (P).3 The value of this simple model is that it creates a very strong and clear prediction about the size of the plea discount (or alternatively, the trial penalty) for each defendant. For example, the theory says that the plea discount/trial penalty will be large when the probability of conviction is low, and small when the probability of conviction is high. This is consistent with existing quantitative and qualitative evidence from Kramer and Ulmer (2002) about the behavior of prosecutors who offer substantial plea bargains to defendants when they do not have adequate evidence to convict at trial (e.g., p. 919).
And, overwhelmingly, legal scholars view the shadow of trial as an accurate description of the plea bargain decision-making process at the individual case level (for a review, see Bibas 2004, p. 2466). However, in a prominent article in the Harvard Law Review, Professor Bibas (2004) raised serious doubts about the prominence of the paradigm and provides a compelling argument for why the shadow model is much too simplistic. Specifically, Bibas observes that structural impediments (e.g., ineffective lawyering, agency costs, bail and pretrial detention) and psychological biases (e.g., overconfidence, risk preferences, anchoring heuristics, and discount rates) are ignored in this theory. Despite these problems, he observes, somewhat incredulously, that there has been no test of even the simple version of this theory.
The idea of the shadow of the trial, although common in legal and economics research, is not a common concept in criminology.4 A more common description of the plea bargain framework in criminology involves a recognition of a court community or organization which comes together around a shared set of norms and expectations (Eisenstein and Jacob 1977; Eisenstein et al. 1987; Nardulli et al. 1988; Ulmer 1997). The institutional focus is then used to explain why plea outcomes vary across courts (e.g. Brereton and Casper 1982; King et al. 2005; Ostrom et al. 2007; Ulmer and Bradley 2006, Ulmer et al. 2010).
Because of the different explanatory focus (courts vs. individual court decision-makers), there is no explicit conflict between the shadow of the trial theory and the institutional theory in criminology. The institutional perspective says that different courts that have different norms about the outcome at trial, will also have different outcomes for pleas. In terms of the shadow of the trial model, this is the same as saying that plea sentences X will be more severe in places with more severe trial sentences Y, because X = P(Y). And, the shadow of the trial model predicts that if courts have institutional factors like caseload size that affect the probability of conviction (P), the plea discount will also vary systematically across courts. For example, jurisdictions with higher caseloads or fewer trials are more likely to have lower probabilities of conviction, therefore, the plea discount must be larger to encourage a plea.5 A conflict between institutional theories arises only if the norms of the courts explicitly create plea discounts that are independent of the probability of conviction and the sentence at trial. This is certainly a possibility, but it is not required by the institutional theories.
Because of the sociological/institutional focus in criminological studies of sentencing (Engen 2009), there is less well developed theory and few tests in criminology for why trial discounts may vary across individual cases in a given court. Two papers by Ulmer with two different sets of colleagues are an important exception (Ulmer and Bradley 2006; Ulmer et al. 2010). Each paper uses a conviction dataset to measure the difference between the sentence outcome for those who pled versus those who went to trial. This provides an unbiased estimate of the average sentence difference between those who pled guilty to a particular charge and those who were convicted at trial of the same charges, under the assumption that those who plea and those who are convicted at trial are the same after controlling for observable variables. The most important omitted variable is the probability of conviction, which, according to the theory of the shadow of the trial, can perfectly predict the size of the plea discount.
While the focal concerns theory predicts that the discount will vary with the probability of conviction, the nature of the data prevents testing this idea more formally (Ulmer and Bradley 2006; Ulmer et al. 2010). In both papers, the importance of doing so is emphasized in the conclusion. For example, Ulmer and Bradley state that “(i)n particular, research should assess trial penalties while accounting for possible selection bias stemming from the likelihood of conviction, and balancing plea–trial sentencing differences against the odds of acquittal” (Ulmer and Bradley 2006, p. 663).
The central problem with actually doing this is empirical. For any given individual case, we only see a sentence at plea or a sentence at trial. We simply do not know the counterfactual of what would have happened if the defendant had made a different decision. In this paper, we present what we believe is the first attempt to create an estimate of the counterfactual for people who pled guilty. Our approach will capture both charge bargaining and sentence bargaining by allowing for the charge to change from arraignment to conviction, and by allowing for the sentence for plea bargains to be different than the sentence for the same case at trial. We will test this simple model with a well-known plea-bargaining dataset, Plea Bargaining in the United States, 1978 by Miller et al. (1980). This dataset has been used for some of the most oft-cited criminology articles on plea bargaining, including articles by Smith (1986) and LaFree (1985), and it thus the main dataset on which the current state of evidence about the drivers of plea bargaining is based. The dataset contains case files from 3,397 people who were charged with felonies in six jurisdictions in 1976 and 1977. Of importance to our goals here, this well-known–albeit dated– dataset contains key information on (1) charge at indictment, (2) charge at conviction, and (3) evidence. Although there are other datasets with these three pieces of information, to our knowledge, these other datasets are even older (i.e., PROMIS 1973 and 1974; Rhodes 1979) than the Miller et al. one used here.
After first creating an estimate of the defendant’s probability of conviction at trial for those who pled, we examine the distribution of this estimate in the data and compare it to the distribution observed for those who actually go to trial. Finally, we compare our ability to predict conviction at trial with the strength of evidence for those who went to trial with our ability to predict the conviction at trial/plea discount with strength of evidence for those who pled guilty. If our estimates are valid, and the “shadow of the trial” model is correct, we should see roughly equivalent coefficients of evidence for those who went to trial, and those who pled guilty. We find that our estimates of the probability of conviction at trial are true in aggregate, but find little support for the claim that strength of evidence predicts the plea discount for those who pled guilty. In the conclusion, we discuss the possible implications of our findings for the dominant “shadow of the trial” model.
We also believe it is important to acknowledge up front that this exercise has serious limitations. The tested theory focuses on explaining variation in individual plea bargains. It is prominent in legal circles, but does not capture many of the institutional features of plea bargaining that criminologists feel are important. The dataset is old and limited, and the assumptions needed to create reasonable counterfactuals are strong. However, we believe it is important for the progress of research on the question of individual variation in plea bargains to begin to formalize, and then test, some of the ideas in the plea bargaining literature. There is growing consensus that much of the discretion during the sentencing process is executed during plea negotiations (Forst 2002), yet this process is dramatically understudied relative to its importance in the field (Bushway and Forst 2011). Our hope is that this attempt to formalize the shadow of the trial paradigm, which is the simplest of all the available plea bargain theories, will lead to the formalization of other theories and the collection of better data for testing. Therefore, while we are humbly aware that this effort has serious shortcomings, we believe that it is an important step for research in the field.
Over the last 20 years, the field of criminology has produced numerous prominent studies of sentencing and sentencing disparity in single jurisdictions (state or federal government) using data on criminal convictions.6 In most cases, researchers were interested in identifying and then explaining the discretion of various actors in the system, including judges, prosecutors, bail administrators and defense attorneys. Yet the simple conviction outcome used as the dependent variable in most quantitative sentencing research is the result of the actions of all the actors in the system combined. Research that focuses solely on the sentenced outcome is unable to comment on the discretion of any given actor. Discretion can be identified methodologically, however, if the researcher can identify the counterfactual punishment that would be meted out if a given actor did not in fact exercise discretion. Researchers have begun to use the institutional features of the sentencing process, most notably sentencing guidelines, to identify the counterfactual.
In the structure of sentencing guidelines, a natural reference point is the recommended sentence length, operationalized as the midpoint or minimum of the guideline range. Absent additional action by the judge or prosecutor, the convicted offender should receive this presumptive sentence. Any departure from this presumptive sentence can then be viewed as an exercise of discretion that can be placed directly in the hands of the actor ultimately responsible for assigning sentencing lengths (Bushway and Piehl 2001; Engen and Gainey 2000). Numerous studies have attempted to study the causal elements that drive whether or not someone gets a departure (e.g. Kramer and Ulmer 1996, 2002; Johnson 2003; Miethe and Moore 1986; Mustard 2001). At least three studies have attempted to study the distance or value of this departure by studying the difference between the minimum or maximum guideline recommendation and the actual sentence (Mustard 2001; Albonetti 1997, 1998).
Another example is the study of mandatory minimums (Kessler and Piehl 1998; Bjerk 2005; Farrell 2003; Bynum 1982; Loftin et al. 1983; Hofer 2000; Ulmer et al. 2007). The mandatory minimum is perhaps the easiest type of prosecutorial discretion to study because the counterfactual penalty in question is fixed. But, typically, the counterfactual penalty is not fixed. In charge bargaining more generally, the value of the initial charge in terms of a penalty has a wide variance (Bibas 2004), so we are unable to discern immediately what the penalty would have been in the absence of the plea bargain. This has the potential disadvantage of forcing the researcher to focus on the existence of a charge bargain, rather than on the value of the bargain. But the predominant theory of plea bargaining at the individual level (i.e., shadow of the trial) explicitly assumes the existence of a perceived sentence at trial. In this framework, the substantive value of interest is the difference between sentences the persons would have received if they had not pled (and were convicted) and sentences they received as the result of the plea bargain. Moreover, the correlates of these two facts may in fact be different. Mustard (2001) found that the factors that determine which cases received departures are different than the factors that explain the size or value of the departures. In the same manner, the factors that determine who gets a charge bargain may be different than the factors that determine the value of the charge bargain. What is needed is a methodology to measure the consequences of the decision to charge bargain.
Piehl and Bushway (2007) developed such a method, based on Smith (1986), which measures the distance covered by a charge bargain. The method uses data from people convicted at trial to estimate the sentences persons who pled guilty would have received if they were convicted at trial of the charges included in the indictment. This predicted amount is then compared with the sentence the person actually received through the plea bargain. The difference between these two sentences is the value of the plea. Piehl and Bushway found a correlation between the value of the plea and race, but were largely unable to explain the value of the plea discount. We think this is because they had no information on the amount and type of evidence. In this paper, we focus on understanding if and how evidence explains the value of the plea, as predicted by the shadow of the trial theory.
Strength of Evidence in Trials and Plea Negotiations
The primary factor believed to drive convictions (either via guilty pleas or jury verdicts) is strength of the evidence (SOE). Indeed, the terms “probability of conviction” and SOE have been used interchangeably (Kramer et al. 2007; McAllister and Bregman 1986). However, SOE appears to be a label that is subjectively and/or arbitrarily applied. Evidence types may hold varying strengths and may differentially impact the probability of conviction, a difference which we posit should manifest in the value of the plea bargain.
Indeed, research has clearly indicated that jurors value certain kinds of evidence over others. Heller (2006) distinguishes between direct and circumstantial evidence. Direct evidence is evidence that “proves a fact without an inference or presumption and which in itself, if true, establishes that fact” (p. 248). Confessions and eyewitness identification are the quintessential examples of direct evidence. In contrast, circumstantial evidence is evidence “from which the fact-finder can infer whether the facts in dispute existed or did not exist” (p. 250). All forensic evidence, including DNA, blood, and fingerprints, and some non-forensic evidence are considered to be of this kind. Although probabilistically circumstantial evidence is much more likely to be reliable than direct evidence, research shows that jurors consistently overvalue direct evidence (Heller 2006; Niedermeier et al. 1999; Wells 1992).
Confession evidence, a form of direct evidence, is arguably the most potent form of evidence, leading one legal scholar to claim that confessions make the introduction of other types of evidence superfluous (McCormick 1983). Kassin and Neumann (1997) tested this notion by manipulating whether mock jurors considered confession, eyewitness, or character evidence. Across three separate studies and several crime types, they found that confessions significantly increased guilty verdicts in comparison to the other two evidence types. Thus, within direct evidence, jurors appear to weight confessions more heavily than eyewitness testimony.
If plea bargaining occurs in the “shadow of a trial,” then the same factors that affect trial decisions should also affect the value of the plea discount. We test whether the presence or absence of certain kinds of evidence drives plea discounts. On the one hand, if prosecuting and defense attorneys are basing their plea offers/acceptances on what jurors would do at trial, rather than structural or extra-legal factors (such as race of the defendant), we would expect to see the presence of confession, for example, to influence the plea discount. On the other hand, if plea arrangements are not conducted in the “shadow of a trial” (Bibas 2004), we would expect to see factors other than those related to evidence to be influential. There is research indicating that extra-legal factors influence the acceptance of a plea (Albonetti 1990; Elder 1989; Frenzel and Ball 2007; LaFree 1985) and whether the prosecutor dismisses charges (Kellough and Wortley 2002). However, we know of no research that looks at whether extra legal factors influence the value of the plea discount with one exception. Ball (2006) found that defendant characteristics did not influence prosecutor’s discretionary use of count bargaining. However, Ball specifically states that “(i)t is important for future research to include strength of evidence that may have been considered in the charging phase and the plea negotiation phase” (p. 257).
To test these alternate hypotheses, we use the Plea Bargaining in the United States 1978 (ICPSR #7775) data. Using this same dataset, Smith (1986) showed that the basic shadow of the trial model held in these data. Specifically, as shown in Fig. 1, the shadow of the trial model indicates that the expected value of a plea must equal the expected value of going to the trial. The plea penalty is received with certainty.7 The trial penalty, however, is not, given the chance of an acquittal. In the shadow of the trial model, the expected value of a trial should equal the probability of conviction times the penalty of a conviction (the sentence) plus the probability of an acquittal times the penalty of an acquittal (zero punishment), or more simply put, that the expected value of a plea must be equal to the probability of a conviction times the sentence at trial. Smith focused on the probability of a prison sentence as his main form of punishment. And, consistent with the theory, he showed that the average probability of a prison sentence for those who pled guilty is indeed equal to the average estimated probability of a prison sentence at trial times the average probability of conviction if the defendant went to trial. In these data, the plea discount was roughly 70%, which is also the estimated probability of conviction in this dataset. Smith’s estimated trial penalty was roughly 40%, which approaches the estimated trial penalty in conventional datasets (Ulmer and Bradley 2006; Ulmer et al. 2010).
While this relationship was true on average, Smith was able to identify groups of people, particularly those with the most severe cases and most serious criminal histories, where this basic relationship did not hold.8 Rather than try to explain the residual or leftover variation, we want to explain the relationship itself. We do this by focusing on a key ratio from the shadow of the trial model explained above. The numerator is the predicted probability of prison given a plea bargain using the standard set of legal factors found to predict prison in sentencing research (Spohn 2000). This is the expected value of the plea. The denominator is the predicted probability of prison sentence given a trial. In the shadow of the trial model, this ratio is, on average, equal to the estimated probability of conviction if the people who pled had gone to trial.
In this step, the goal is to create the best prediction possible, not build a causal model. We use a probit model. We estimate this model first only for those people who were convicted at trial. We then use these estimated coefficients to form a predicted sentence length for each person who pled guilty based on the charges at arraignment. In other words, we create an estimate of a person’s probability of incarceration at arraignment, using the estimated coefficients from Eq. 1. We then estimate a standard prison equation for people who pled guilty. Using this equation, we create an estimate for the systematic or predictable component of sentence at plea bargain. This estimate will be based on whatever charge to which the person pled guilty, and will be based on the data from the people who pled guilty. Therefore, it will capture both charge bargaining and sentence bargaining, and not simply charge bargaining as in Bushway and Piehl (2007). This is important because research has shown that plea bargaining occurs both ways (Ulmer and Bradley 2006; Ulmer et al. 2010). Each person who pled guilty will have two estimated probabilities—the estimated probability of incarceration if they go to trial and the estimated probability of incarceration given that they pled guilty.
We then calculate the ratio of each person’s predicted actual probability of a prison sentence divided by the expected probability of prison at arraignment. In the shadow of the trial model, this ratio should be equal to the probability of conviction at trial. And, as we will show, this is roughly true on average. However, as we will also show, this is not true for every individual. In fact, we have a number of people in our data for whom this ratio is greater than 1. In the next section, we attempt to explain this ratio using case evidence. We will then compare these results with the same model estimated for those who go to trial. If plea bargaining truly occurs in the shadow of the trial, evidence should play a similar role predicting conviction for both those who pled and those who go to trial.
Following Smith (1986), we chose a subsample of the data from the Miller et al. (1980) data. First, we selected only those people who pled guilty or went to trial. Second, we restricted the analysis to focus only on males charged with robbery or burglary felony offenses, which are the bulk of cases in these jurisdictions. Data from El Paso were eliminated due to missing cases. These restrictions, together with listwise deletion of missing cases, resulted in a sample of 1,593 plea cases and 305 tried cases.
Descriptive statistics for plea sample (N = 1,593)
Incarceration (maximum sentence 6 months or more)
Number of prior felony arrests
Full time employed prior to arresta
Under CJ supervision prior to arresta
Number of witnesses (10 = 10 or more)
Robbery charge (vs. burglary)a
Lost or damaged more than $1,000a
Harm to victima
Weapon used in the crimea
Married at time of arresta
In regard to evidence, 72.5% of the sample had at least one eyewitness. Although this seems high, it is worth remembering that these are robbery and burglary defendants who pled guilty. In fact, 86% of the robbery defendants had an eyewitness, and only 66% of the burglary defendants had an eyewitness. 89% of the sample had some type of physical evidence against them, and 52% of the sample confessed.
Plea Bargaining in the Aggregate
Actual and expected values of conviction and incarceration (sample averages)
1. Probability of conviction at trial
2. Probability of incarceration for defendants convicted at trial
3. Probability of incarceration for defendants pleading guilty
4. Estimated probability of incarceration if convicted at trial for defendants pleading guilty
5. Ratio of the probability of incarceration if pleading guilty to the estimated probability of incarceration if convicted at trial for defendants pleading guilty
Probit estimates of the determinants of incarceration given trial conviction
Incarceration given conviction (N = 266)
Full time employment
Number felony arrests
Supervision at time of arrest
Over $1,000 damage/lost
Harm to victim
Weapon used in crime
Of course, not everyone is convicted at trial. The shadow of the trial model says that the penalty that the defendant will accept at plea must be equal to the penalty at trial discounted by the probability of being acquitted. As a result, the ratio of the penalty at trial and the penalty at plea for those who pled must be equal to the probability of conviction. If we divide Table 2 row 3 by row 4, we get row 5 (55.7/72.2 = 77.1%). In other words, the model predicts that the probability of conviction at trial for those who pled guilty would be 77.1%. Those who actually went to trial were found guilty 76.7% of the time. At least in the aggregate, the “shadow of the trial” appears to be quite accurate (in that these rates of 76.7 and 77.1% are quite close). This analysis is very similar to that originally done by Smith (1986), with comparable results. On average, the plea discount appears explainable by the shadow of the trial model.
Plea Bargaining at the Individual Level
The next more stringent test of the shadow of the trial model is to check if the same factors that explain the “probability of conviction” for those who go to trial can also explain our estimated probability of conviction at trial for those who pled. Recall that we do not know the probability of conviction for those who pled, but we can generate an estimate of this ratio at the individual level. We can then try to explain this probability with evidence and compare the role of evidence for those convicted at trial and those convicted by plea.
To ground this exercise, it is helpful to refer back to Fig. 1. What we propose to do is create an estimate of Y for those who pled guilty. We already have X for those who pled guilty. The ratio of X/Y should be equal to the probability of conviction, if the theory holds.
In/out incarceration probit model for those who pled guilty
Full time employment
Number felony arrests
Supervision at time of arrest
Over $1,000 damage/lost
Harm to victim
Weapon used in crime
Seattle Washington (relative to Norfolk VA)
New Orleans Louisiana
Delaware County Pennsylvania
Finally, using the estimated probability of incarceration at trial for those who pled (Y) and the actual probability of incarceration for those who pled (X), we created a ratio for every individual (P). This ratio is the estimated plea discount for every person. X and Y are highly correlated within our sample (.72), and in general the ratio is less than 1. The average of the individual ratios across the sample is .77, matching what we found on the last row of Table 2. The average predicted probability of incarceration using this estimated ratio is equal to the average probability of incarceration found in this dataset. Again, this appears to support the idea that people are bargaining in the shadow of the trial.
But, the interesting (and striking) fact is that the individual variation in this ratio is quite large. The ratios range from a value of .02 (the predicted probability of incarceration for a plea is only 2% of the estimated probability of incarceration if the person goes to trial) to values over 2 (the predicted probability of incarceration at plea is twice as high the estimated probability of incarceration if the person went to trial). 16% of the cases have ratios over 1, which in the context of the model does not make sense. The probability of conviction cannot be higher than 1.10 More to the point, these range of values do not appear to be credible estimates of the probability of conviction for these individuals, despite the fact that the estimate appeared to have validity at the aggregate level.
A simple response would be to conclude that the models used to generate the estimates are poor. But, if the model is poor, we would see little variation in X and Y, and therefore little variation in the ratio. In each case, the models used to generate the estimates (Tables 3, 4) are standard models used in the sentencing literature, and the models perform reasonably well. However, we concede that these models are not as powerful as the standard conviction models reported using data from sentencing guidelines states, which often have R2 close to .6 for the sentence length data. Nonetheless, the estimated probabilities from each model have excellent ranges, going from a minimum of .01 to a maximum of .99. Poor models generate predictions with small ranges (a model with only a constant will produce no range). The models generate good variation in the expected penalties at trial and plea bargains.
OLS regression explaining the probability of conviction
Estimated probability of conviction for those who pled guilty (incarceration ratio)
Probability of conviction at trial
Probability of conviction at trial
Number of witnesses
Full time employment
Number felony arrests
Supervision at time of arrest
Over $1000 damage/lost
Harm to victim
Weapon used in crime
Seattle Washington (Relative to Norfolk VA)
New Orleans Louisiana
Delaware County Pennsylvania
Ideally, we would include all the variables in the model that are used to predict conviction in the standard model. But, we generated the estimate of the probability of conviction using observable legal and extra-legal factors. Essentially, our dependent variables are non-linear combinations of the variables that we used to estimate the incarceration models in Tables 3 and 4. Therefore, by definition, these variables will be strongly correlated with a dependent variable that was estimated using these variables. And, in fact a model with all of the variables from Table 3 included has an R2 of .68. This is artificially high. However, we excluded evidence from the incarceration models (Tables 3, 4) on the grounds of existing research which claims that evidence does not drive sentencing outcomes. Therefore, any correlation between evidence and the estimated probability of conviction will be real and not an artifact of the model. We only included the four available measures of evidence to explain the estimated probability of conviction for those who pled guilty.
Our prediction was that evidence would be positively correlated with the probability of conviction. However, in Column 1, of Table 5, the evidence is either not significant or it is negative and significant. The effects are large, with a confession leading to a nearly 12 percentage point reduction in the probability of conviction at trial for those who pled guilty. The R2 are also quite low (.045). In general, the model performs poorly.
In contrast, Column 2, which reports a standard linear probability model of conviction for those who go to trial, is more along the lines of what we expect for a trial conviction model. Three of the four evidence factors are positive and significant in the model for those who went to trial. The presence of a confession, for example, leads to a 13 percentage point increase in the probability of conviction for those who go to trial, which is consistent with previous research and theory (Heller 2006; Kassin and Neumann 1997). Simply put, the model in Column 2 performs as expected for those who go to trial. Evidence plays an important role in predicting conviction.
The one caveat is that our model fit is still relatively low. Although the fit improves, evidence still matters as expected in Column 3 when all legal and extralegal factors are included to explain trial conviction for those who go to trial. If the shadow of the trial model is valid, and the data are valid, we should be able to generate a legitimate estimate of the probability of conviction for those who pled. In that case, we would expect the results of column 1 and column 2 to be similar. They clearly are not. Although the “shadow of the trial” model allowed us to predict accurate aggregate estimates of the plea discount, the individual estimates appear to be deeply flawed.11 Bibas (2004) argues convincingly that the evaluation of the shadow model must take place at the individual level. He posited that the plea discount would be both too high and too low for some people relative to what one would expect if bargaining occurred in the shadow of the trial. He also argued that this variation would not be explained by factors such as evidence. His claims appear to be true in these data.
In this paper, we present the first (to our knowledge) explicit empirical test of the “shadow of the trial” model. We generated an important counterfactual—an estimate of the probability of conviction at trial for those who pled guilty—and showed that the estimates make sense in the aggregate and have meaningful variation. However, we then showed that although evidence does an excellent and predictable job of explaining the probability of conviction for those who go to trial, evidence cannot similarly explain variation in our estimates of the probability of conviction for those who pled guilty.
There are two primary possible explanations for the results of our model. The first explanation is that our data cannot accurately estimate the required empirical models. In this view, for example, the data prevent us from generating good/valid estimates of the probability of conviction for those who actually pled guilty. The largest threats here are the lack of more explanatory variables, which leave us with poor model fit, and the failure to control for selection bias when estimating the probability of incarceration given conviction. Although we observe cases from arraignment forward, our plea models are estimated only on the cases that pled. What about the cases where the charges were dismissed? The fact that these cases were pursued (and others were not) might affect our estimates of the sentencing outcomes that we used to generate our estimated probability of trial conviction for those who pled guilty.
We recognize the limitations of the data and therefore strongly encourage future research that replicates this approach with better, newer datasets or develops ways to test this idea with alternative approaches, such as experimental surveys with court practitioners. Surveys would both be less costly, and potentially more accurate, particularly if the perceived counterfactual used in the decision process can be directly obtained from the respondents rather than estimated from a model.
The second possible explanation is that the shadow of the trial model is simply wrong at the individual level. Bibas (2004) reviewed multiple structural and psychological reasons for why plea bargaining does not occur in the shadow of the trial, or alternatively why strength of the evidence does not influence plea discounts and decisions in the same manner as trial verdict decisions. While the plea discount “looks” rational, the actual process by which decision makers reach plea deals is different than the process by which juries and judges reach conviction decisions in trials.
Indeed, there are numerous reasons supporting the differences between trial and plea decisions. For one, the extensive research on jury decision-making has consistently indicated that factors such as juror demographics and attitudes, attractiveness of the defendant, police credibility, attorney experience, etc. (see Devine et al. 2001 for a review; Garvey et al. 2004) make appreciable differences in verdicts; factors which can only be supposed, if considered at all, in plea negotiations.
Second, jury and plea decision-making may diverge because of the processes themselves. Trials allow for constitutionally afforded safeguards not present in guilty pleas (e.g., proof beyond a reasonable doubt, cross-examination of witnesses). The evidence presented in trials undergoes more intense scrutiny and has an opportunity to be challenged by both sides. Thus, the evidence that a jury hears–whether it be weak or strong–often will not mimic the evidence used in plea decision-making. Evidence in pleas and trials may be considered qualitatively different and used qualitatively differently.
Finally, at each stage of the plea decision process, from indictment to acceptance of an offer, decision makers consider both the seriousness of the case and the probability of conviction. The acceptance of the plea (and the decision to offer the plea) will be a mix of the probability of conviction and the seriousness of the case. This potential convolution of conviction probability and case seriousness distinguishes the plea process from the trial situation because generally the seriousness of the crime does not directly affect the decision to convict but rather only sentence severity after conviction decisions have been determined. In plea decision-making, the conviction and sentencing decision are combined; this is not the case for trials and this difference may fundamentally alter the nature of the decision.
Although, to our knowledge, this is the first empirical test of the shadow of trial model, there have been other formal theoretical models that cast doubt on the shadow of the trial model. For example, economist Bjerk (2005) argued that the structure of the plea bargain cannot accommodate both the desire to punish severely and the desire not to convict innocent people. He shows empirically that the existence of harsh penalties will “force” rational people to plead guilty in cases when they are innocent. By inference, if this is true, it must be the case that the process that determines conviction is necessarily different for those who plead versus those who go to trial.
Our finding that evidence does not influence pleas in the same manner as it does trials is potentially quite important, though preliminary and limited by the age and quality of the dataset. If these results are correct, the leading paradigm for explaining variation in the trial penalty/plea discount at the level of the individual defendant rather than the institutional or court level appears to be seriously flawed. And, despite the fact that 90% or more of all felony convictions result from plea bargains, we appear to have no formal theoretical alternative to the “shadow of the trial” paradigm at the individual level These models might profitably be developed from an explicit comparison of the individual, legalistic/rational approach discussed in this paper with the institutional approach more common in criminology (Eisenstein and Jacob 1977; Eisenstein et al. 1987; Nardulli et al. 1988; Ulmer 1997). At this point, these approaches have not been used to explain why the “shadow of the trial” model might be flawed. Perhaps such a comparison might help generate simple, testable and competing models for plea bargain decision-making.
Expected values are the weighted outcome from an uncertain game. For example, imagine a game where a person is given $1 half the time, and 0 half the time. The expected value of the game is $.50. In the plea situation, the person gets X with certainty, so the expected value of a plea is X.
In behavior decision theory, risk neutral has a specific meaning. A person is risk neutral if they are indifferent between receiving the expected value with certainty or as a result of an uncertain game. A person is risk seeking if they would rather receive a given expected value with uncertainty rather than with certainty. A person is risk averse if they would rather receive the same value with certainty than with uncertainty. The risk neutrality assumption is one of the potential problems identified by Bibas (2004).
In criminology, the plea discount is sometimes called the trial penalty. The trial penalty will be (1/P). So for example, if the probability of conviction is .75, the trial penalty will be 1.33 or 33%, meaning that the trial sentence will be 33% bigger than the plea sentence.
A search of Criminal Justice Abstracts with full text on 7-22-11 found no criminology articles with the phrase “shadow of trial” or “shadow of the trial”. In contrast, a Lexis Nexis search on the same date of law review articles found the phrase in 233 articles. Smith (1986) builds a model that is identical to the shadow of the trial model, but does not use the term. The idea is also implicit in Elder (1989) and Lafree (1985).
This expected correlation between high caseloads and trials and plea discounts is found by Ulmer et al. (2010) in federal conviction data.
Although we acknowledge that the judge, and not the prosecutor, is the final arbiter of sentencing, our understanding is that judges rarely stray from the sentence recommendations of prosecutors (e.g., Heumann 1981). Further, the main issue is punishment versus no punishment, not degree of punishment.
The coefficient on a private lawyer showing that those with a private lawyer are more likely to get incarcerated is unusual, but could reflect selection bias, simply showing that those facing a greater threat of incarceration are more likely to hire their own lawyer.
We delete 7 cases who are over 4 to deal with skew—these cases are situations where both probabilities are close to zero. Only 3 of these cases would have been included in the models because of missing data on evidence. The models do not appear to be sensitive to the exclusion of these 3 cases—The results in Table 5 column 1 are virtually identical with the excluded cases.
The results from the full model for those who pled guilty have, as noted, a much better fit. The coefficients for evidence are also substantially different than the estimates from Column 1. Three of the four variables are now 0, and one, confession, is positive and significant instead of negative and significant. However, its magnitude is much smaller than the estimate from Column 3. Our conclusion remains the same—evidence does not appear to explain our estimated conviction probability.