Correcting for Scale Perception Bias in Measuring Corruption: an Application to Chile and Spain

Abstract

Corruption perceptions are commonly utilized as essential information to measure corruption across countries and regions. In this paper we consider the hypothesis that respondents to corruption perception questions utilize different response scales in their answers, i.e. for identical levels of corruption practices, subjects from one country could answer a different level of corruption perception than subjects from another country, based on socioeconomic characteristics. This hypothesis is investigated by utilizing the technique of anchoring vignettes with a sample of citizens in Spain and Chile. The results show that response scales are used differently across individuals in both countries, suggesting that the use of uncorrected measures of corruption perceptions could mislead some conclusions about the comparisons of the corruption levels between countries.

Appendices

Appendix 1: The Vignette Equivalence Assumption

The methodology of anchoring vignettes relies on the assumption that all survey respondents interpret the levels of the categories described in the vignettes in the same way, i.e. the vignette equivalence assumption. Following Rice et al. (2010), a minimal condition for vignette equivalence is that the individual responses are consistent with the global ordering of the vignettes. Thus, for each domain and for each country we evaluate the proportion of respondents that gave an ordering of vignettes consistent with the global ordering. The latter can be derived by pooling all vignette responses across subsamples and considering the average categorical response for each vignette.

Table 7 presents the results of the proportions of consistent orderings by vignette and country. For each domain and for each vignette there was no substantial variation across both countries. The values of the percentages are all above 0.72, with a maximum rate of 0.98. On average, the percentages of consistent orderings are slightly larger for the domain of tax effectiveness than for the other two domains of corruption perception (public employment and bribing practices). Thus, these results give support to the maintained assumption of vignette equivalence.

Table 7 Proportions of consistent orderings by vignettes and country

Appendix 2: The Non-parametric Model

In order to explore whether or not data from different countries present scale perception bias, we follow King et al. (2004) recommendations and apply a non-parametric approach as a first step in the data analysis strategy. The main idea of this model is to “recode the categorical self-assessment relative to the set of vignettes” (King et al. 2004). If all the respondents order the vignettes in a similar way, individual responses can be recoded as a number that indicates the position of the self-assessment compared with the proposed vignettes.

More concisely, if V1, V2, V3 y V4 are the four vignettes proposed, and all the respondents rate V4 higher than V3, V3 higher than V2 and V2 higher than V1, then the individual response can be recoded in the following way.¹² The recoded variable would take a value of 1, if self-assessment value is lower than V1; a value of 2, if self-assessment value is the same than V1; a value of 3, if self-assessment value is higher than V1 but lower than V2; a value of 4, if self-assessment value is the same than V2; a value of 5, if self-assessment value is higher than V2 but lower than V3; a value of 6, if self-assessment value is the same than V3; a value of 7, if self-assessment value is higher than V3 but lower than V4; a value of 8, if self-assessment value is the same than V4; and a value of 9, if self-assessment value is higher than V4. That is, the original self-assessment variable for individual i (y _i ), can be recoded as the variable C _i that can be interpreted as a DIF-free variable. The recoded C _i variable takes the following values:

$$ C_{i} = \left\{ {\begin{array}{*{20}c} 1 \hfill & {{\text{if}}\,y_{i} < V1} \hfill \\ 2 \hfill & {{\text{if}}\,y_{i} < V1} \hfill \\ 3 \hfill & {{\text{if}}\,V1 < y_{i} < V2} \hfill \\ 4 \hfill & {{\text{if}}\,y_{i} < V2} \hfill \\ 5 \hfill & {{\text{if}}\,V2 < y_{i} < V3} \hfill \\ 6 \hfill & {{\text{if}}\,y_{i} < V3} \hfill \\ 7 \hfill & {{\text{if}}\,V3 < y_{i} < V4} \hfill \\ 8 \hfill & {{\text{if}}\,y_{i} < V4} \hfill \\ 9 \hfill & {{\text{if}}\,y_{i} < V2} \hfill \\ \end{array} } \right. $$

Appendix 3: The Hopit Model

The model for the correction of the scale response bias using anchoring vignettes is based on a simultaneous equation approach that jointly models the self-assessment response to the corruption perception question and the responses to the vignette questions. The model is a generalization of the ordinal probit model, also called HOPIT model (e.g. King et al. 2004; King and Wand 2007; Kapteyn et al. 2007; Rice et al. 2010) that allows for the thresholds defined by categorical variables to be modeled as a function of covariates. Let c _rj be the categorical self-assessment response of corruption perception framed into question r by individual j that can take a value from 1 to 5. Thus, we have the following behavioral equation:

$$ \tilde{c}_{rj} = \Upgamma z_{rj} + \zeta_{rj} $$

(1)

where Г is a vector of parameters, z _rj is a vector of socioeconomic covariates that explains the self-assessment responses, and $\zeta_{rj} \sim {\text{N (0, }}\sigma_{r}^{2} )$. Variable $\tilde{c}_{rj}$ is a latent or unobserved variable for which the researcher only observes the categorical response c _rj which is assumed to be related with $\tilde{c}_{rj}$in the following way:

$$ c_{rj} = j\;{\text{if}}\;\tau_{j}^{i - 1} \le \tilde{c}_{rj} < \tau_{j}^{i} ,\quad i = 1, \ldots ,5 $$

(2)

where τ ⁱ _j are the thresholds between categories that are given by:

$$ \tau_{j}^{0} = - \infty ,\quad \tau_{j}^{5} = \infty ,\quad \tau_{j}^{i} = \beta^{i} x_{j} + \mu_{j} ,\quad i = 1, \ldots ,4 $$

(3)

where the reporting behavior equation in (3) for τ ⁱ _j models the thresholds as a function of socioeconomic covariates x _j and parameter vector β ⁱ The error terms μ _j are assumed to be normally distributed with zero mean and σ _μ variance, and introduces the unobserved individual effect or heterogeneity in the response scale. In addition, μ _j is assumed to be independent of ζ _rj and z _rj . Further, the error term in the threshold or reporting behavior Eq. (3) implies that the evaluations of corruption in the vignettes are correlated with each other and with the self-assessment of corruption, because some respondents might use low thresholds while others might use high thresholds.

The hypothesis of differential item functioning (DIF) follows because of the fact that respondents use different response scales, i.e. this means that τ ⁱ _j are different across respondents because they depend on socioeconomic characteristics or covariates. The restricted model that does not allow for DIF does not allow variation of the thresholds across respondents. The unrestricted model estimates the reporting behavior equation in (3) simultaneously by utilizing the responses to the vignette questions.

That is, let c _vj be the categorical response of corruption perception to the vignette questions by individual j that can take a value from 1 to 5. We assume that the expected value of the underlying latent variable $\tilde{c}_{vj}$ depends only on the corresponding vignette, i.e.

$$ \tilde{c}_{vj} = \alpha_{v} + \varepsilon_{vj} $$

(4)

where α _v is a fixed vignette parameter that represents the level of corruption represented in the vignette question v, and $\varepsilon_{vj} \sim {\text{N (0, }}\sigma_{v}^{2} )$.

Similarly to the self-assessment response model outlined in Eqs. (1–3), variable $\tilde{c}_{vj}$ is a latent or unobserved variable for which the researcher only observes the categorical response for each vignette c _vj which is assumed to be related with the underlying $\tilde{c}_{vj}$in the following way:

$$ c_{vj} = j\;{\text{if}}\tau_{j}^{i - 1} \le \tilde{c}_{vj} < \tau_{j}^{i} ,\quad i = 1, \ldots ,5 $$

(5)

where τ ⁱ _j are the threshold between categories that are given by:

$$ \tau_{j}^{0} = - \infty ,\quad \tau_{j}^{5} = \infty ,\quad \tau_{j}^{i} = \beta^{i} x_{j} + \mu_{j} ,\quad i = 1, \ldots ,4 $$

(6)

where it can be seen that Eq. (6) is identical to Eq. (3). Therefore, the information provided by the individuals’ assessments to the levels of corruption defined by the anchoring vignettes allows us to correct for DIF by estimating the thresholds as functions of relevant socioeconomic variables simultaneously with the self-assessment responses.