Modelbased and actual independence for fairnessaware classification
Abstract
The goal of fairnessaware classification is to categorize data while taking into account potential issues of fairness, discrimination, neutrality, and/or independence. For example, when applying data mining technologies to university admissions, admission criteria must be nondiscriminatory and fair with regard to sensitive features, such as gender or race. In this context, such fairness can be formalized as statistical independence between classification results and sensitive features. The main purpose of this paper is to analyze this formal fairness in order to achieve better tradeoffs between fairness and prediction accuracy, which is important for applying fairnessaware classifiers in practical use. We focus on a fairnessaware classifier, Calders and Verwer’s twonaiveBayes (CV2NB) method, which has been shown to be superior to other classifiers in terms of fairness. We hypothesize that this superiority is due to the difference in types of independence. That is, because CV2NB achieves actual independence, rather than satisfying modelbased independence like the other classifiers, it can account for model bias and a deterministic decision rule. We empirically validate this hypothesis by modifying two fairnessaware classifiers, a prejudice remover method and a reject optionbased classification (ROC) method, so as to satisfy actual independence. The fairness of these two modified methods was drastically improved, showing the importance of maintaining actual independence, rather than modelbased independence. We additionally extend an approach adopted in the ROC method so as to make it applicable to classifiers other than those with generative models, such as SVMs.
Keywords
Fairness Discrimination Classification Costsensitive learning1 Introduction
The goal of fairnessaware data mining is to analyze data while taking into account potential issues of fairness, discrimination, neutrality, and/or independence. Techniques of fairnessaware data mining are helpful for avoiding unfair treatments as follows. Data mining techniques are increasingly being used for serious decisions that affect individual’s lives, such as decisions related to credit, insurance rates, or employment applications. For example, credit decisions are frequently made based on past credit data together with statistical prediction techniques. Such decisions are considered unfair in both a social and legal sense if they have been made with reference to sensitive features such as gender, religion, race, ethnicity, disabilities, or political convictions. Pedreschi et al. (2008) were the first to propose the concept of fairnessaware data mining to detect such unfair determinations. Since the publication of their pioneering work, several types of fairnessaware data mining tasks have been proposed.
In this paper, we discuss fairnessaware classification, which is a major task of fairnessaware data mining. Its goal is to design classifiers while taking fairness in the prediction of a class into account. Such fairness can be formalized based on independence or correlation between classification results and sensitive features. In general, some degree of prediction accuracy must be sacrificed to satisfy a fairness constraint. However, if a predictor violates the constraint, the predictor cannot be deployed in the real world, because social demands, such as equality of treatment, should not be ignored. Even though a predictor can classify accurately, if it violates a fairness constraint, it does not truly perform the classification task from a social perspective. Therefore, it is important to improve the tradeoff between fairness and accuracy in order that a fairnessaware classifier can effectively predict under a specified fairness constraint in practical use.
The main purpose of this paper is to discuss the theoretical background of formal fairness in classification, and to identify important factors for achieving a better tradeoff between accuracy and fairness. We here focus on Calders and Verwer’s twonaiveBayes (CV2NB) method (Calders and Verwer 2010), which is a pioneering fairnessaware classifier. This CV2NB classifier has achieved a high level of fairness, as we will show in our experimental section. We analyze this method and hypothesize that the effects of model bias and a deterministic decision rule are essential for improving fairness–accuracy tradeoffs.
We introduce two important factors: model bias and the deterministic decision rule. Model bias is the degree of difference between a true distribution to fit and an estimated distribution represented by a model of a classifier, and such bias has been well discussed in the context of biasvariance theory (Bishop 2006, Sect. 3.2). A fairness constraint must be satisfied based on a sensitive feature and the true distribution of a class. However, if we use a distribution restricted by a model instead of a true distribution, the satisfied fairness constraint diverges from the constraint that we have to satisfy. Hence, model bias may damage the fairness of the learned classifier. A deterministic decision rule is another factor that can worsen the quality of fairness. Once class posteriors or decision functions of a classifier are learned, a class label for a new instance is deterministically chosen by applying a decision rule. For example, a class whose posterior is maximum among a set of classes is deterministically chosen to minimize the risk of misclassification (Bishop 2006, Sect. 1.5). If we assume that classes are probabilistically generated according to a class posterior when designing a fairnessaware classifier, the class labels that are actually produced will deviate from the expected ones. This deviation worsens the quality of fairness. For these two reasons, the influence of model bias and a deterministic decision rule must be carefully maintained in order to satisfy a fairness constraint with the least possible loss of a classifier.
Our first contribution is to distinguish notions of two types of independence: modelbased independence and actual independence. Modelbased independence is defined as statistical independence between a class and a sensitive feature following a model distribution of a classifier. On the other hand, in the case of actual independence, the effects of model bias and a deterministic rule are considered in the context of a fairness constraint. We formally state these two types of independence, which are important in a context of fairnessaware data mining.
Our second contribution is modifying two existing fairnessaware classifiers so as to satisfy actual independence in order to validate the above hypothesis. The first classifier is our logistic regression with a prejudice remover regularizer (Kamishima et al. 2012), which was originally designed to satisfy a modelbased independence condition. The second classifier is a reject optionbased classification (ROC) method (Kamiran et al. 2012), which changes decision thresholds according to the values of sensitive features. Though the degree of fairness is adjusted by a free parameter in the original method, we here develop a method to find settings of parameters so that the resultant classifiers respectively satisfy modelbased independence and actual independence conditions. By comparing the performance of classifiers satisfying modelbased and actual independence, we validate the hypothesis that the effects of model bias and a deterministic rule cannot be negligible.
Our final contribution is to extend an approach adopted in the ROC method so as to make it applicable to classifiers beyond those with generative models. Any type of classifier, such as those with discriminative models or discriminant function, can be modified so as to make fair decisions using this extension technique.

We propose notions of modelbased and actual independence, the difference between which is an essential factor for improving tradeoffs between the fairness and accuracy of fairnessaware classifiers.

We empirically show that the fairness of classifiers was drastically improved by modifying them to satisfy actual independence. This fact validates the importance of the difference between the two types of independence.

We extend an approach adopted in the ROC method so as to make it applicable to any type of classifiers.
2 Fairnessaware classification
This section summarizes the concept of fairnessaware classification. Following the definitions of notations and tasks, we introduce a formal notion of fairness.
2.1 Notations and task formalization
The goal of fairnessaware data mining is to analyze data while taking into account potential issues of fairness. Formal tasks of fairnessaware data mining can currently be classified into two groups: unfairness discovery and unfairness prevention (Ruggieri et al. 2010). We here focus on fairnessaware classification, which is a major task of unfairness prevention. The goal of fairnessaware classification is to categorize data while simultaneously taking into account issues or potential issues of fairness, discrimination, neutrality, and independence. Three types of variables, Y, \(\mathbf {X}\), and S, are considered in fairnessaware classification. The random variables S and \(\mathbf {X}\) denote a sensitive feature and a set of nonsensitive features, respectively. A sensitive feature represents information with respect to which fairness must be maintained. For example, in the case of avoiding discrimination in credit decisions, a sensitive feature might correspond to gender, religion, race, or some other characteristic specified from a social or legal viewpoint, and credit decisions must be fair in terms of these features. Nonsensitive features, \(\mathbf {X}\), consist of all other features. \(\mathbf {X}\) is composed of m random variables, \(X^{(1)},\ldots ,X^{(m)}\). The random variable Y denotes a class variable that represents a class, such as the result of a credit decision.
We next define notations of probability distributions over the space \((Y, \mathbf {X}, S)\). Figure 1 depicts a geometrical view of the distributions. We first introduce distributions that are also managed in a standard machine learning process. These distributions are depicted in the left half of Fig. 1. Each object is represented by a pair of instances, \((\mathbf {x}, s)\), which are generated from a true distribution. Given the object, the corresponding class instance value, y, is generated from a true distribution, \(\mathop {\Pr }[Y  \mathbf {X}{=}\mathbf {x}, S{=}s]\). It should be noted that this true distribution may lead to a potentially unfair decision that depends on a sensitive feature, S. The true joint distribution, \(\mathop {\Pr }[Y, \mathbf {X}, S]\), is in a family of all distributions over \((Y, \mathbf {X}, S)\), which corresponds to the entirety of Fig. 1. We cannot know the true distribution itself, but we can observe data sampled from the true distribution. These data comprise a (training) dataset, \(\mathcal {D}= {\mathord {\left\{ (y_{i}, \mathbf {x}_{i}, s_{i}) \right\} }},\, i=1, \ldots , n\). We additionally define \(\mathcal {D}_{s}\) as a subset that consists of all the data in \(\mathcal {D}\) whose sensitive value is s. A family of model distributions, \(\mathop {\hat{\Pr }}[Y, \mathbf {X}, S]\), is also given. Joint model distributions are on a model subspace, depicted by a horizontal plane in Fig. 1. Examples of model distributions are naive Bayes or logistic regression. Note that because the true distribution is not on the model subspace in general, the problem of model bias arises, as we will discuss in Sect. 3.2.1. Given a training dataset, the goal of the standard classification problem is to specify the model distribution that would best approximate a true distribution among all candidate model distributions on the model subspace.
Next, we turn to distributions that are particularly required to maintain fairness in classification. A fairness constraint is assumed to be formally specified, and a set of all distributions satisfying the fairness constraint constitutes a fair subspace, \(\mathop {{\Pr }^{\mathord {\scriptscriptstyle \circ }}}[Y, \mathbf {X}, S]\), depicted by a vertical plane in Fig. 1. In this paper, we mainly discuss a fairness constraint formalized as unconditional independence between a class variable, Y, and a sensitive feature, S, as in the next Sect. 2.2. In this case, a fair subspace is equivalent to a set of all distributions satisfying the independence condition. The intersection of the fair subspace and a model subspace is a fair model subspace, which consists of all candidate estimated fair distributions, Open image in new window , as depicted by a thick line in Fig. 1. Given a training dataset, the goal of fairnessaware classification is to find the fair model distribution that would best approximate a true distribution among all candidate distributions on the fair model subspace.
2.2 Fairness in classification
Here we review formal definitions of fairness in classification. Though many types of fairness have been proposed, we will highlight a few representative examples. First, conditional independence, Open image in new window , corresponds to the simple elimination of a sensitive feature. Note that Open image in new window denotes the (unconditional) independence between variables A and B, and Open image in new window denotes the conditional independence between A and B given C. The simple elimination of a sensitive feature from prediction models is insufficient for avoiding an inappropriate determination process because of the indirect influence of sensitive information. Such a phenomenon is called a redlining effect (Calders and Verwer 2010). An example of a redlining effect in online ad delivery has been reported (Sweeney 2013). When a full name is used as a query for a Web search engine, online ads with words indicating arrest records will be more frequently displayed for first names that are more common among individuals of African descent than individuals of European descent. In this delivery system, no information about the race or actual first name of users is exploited intentionally. Rather, the online ads are unfairly delivered as the result of automatic optimization of the clickthrough rate based on the feedback of users.
We next focus on unconditional independence, Open image in new window . This condition must be satisfied to avoid the redlining effect, as shown below. Consider a simple regression case such that \(Y = X + \epsilon _{X}\) and \(S = X + \epsilon _{S}\), where \(\epsilon _{X}\) and \(\epsilon _{S}\) are mutually independent Gaussian noises. A condition Open image in new window is satisfied because Gaussian noises, \(\epsilon _{X}\) and \(\epsilon _{S}\), are independent if X is observed. However, the redlining effect is caused because both variables, Y and S, depend on a common variable, X. As observed in this example, Y and S must not depend on any common variables, and thus unconditional independence Open image in new window must be satisfied, to avoid the redlining effect. We would like to note that this fairness condition implies the assumption that class labels of a training dataset may be unfair or unreliable due to unfavorable decisions that have been made for people in a protected group. Fairness conditions which assume that training labels are fair have been discussed in Hardt et al. (2016), Zafar et al. (2017).
To represent a fairness constraint in formulae, a fairness index to measure the degree of fairness, such as Open image in new window , is introduced. Many types of fairness indices have been proposed: discrimination score (Calders and Verwer 2010), mutual information (Kamishima et al. 2012), \(\chi ^{2}\)statistics (Berendt and Preibusch 2012; Sweeney 2013), \(\eta \)neutrality (Fukuchi et al. 2013), neutrality risk (Fukuchi and Sakuma 2014), and a combination of statistical parity and the Lipschitz condition (Dwork et al. 2012; Zemel et al. 2013). Note that a previously published tutorial (Hajian et al. 2016) provides a good survey of these indices. If these fairness indices are worse than a prespecified level, the corresponding decisions are considered unfair.
3 Analysis of fairness in classification
We first review the CV2NB method, which achieves a better accuracy–fairness tradeoff, as shown in experimental Sect. 6.2. We then hypothesize that this superiority is due to the effects of model bias and a deterministic decision rule being taken into account. Based on this hypothesis, we here formalize the notions of modelbased independence and actual independence.
3.1 Calders and verwer’s twonaivebayes
As proved in our experimental Sect. 6, the CV2NB method is highly efficient; that is to say, this classifier can precisely and fairly predict class labels. We next discuss the reason for this superiority.
3.2 Why is the CV2NB method superior?
CV2NB tends to achieve better tradeoffs between accuracy and fairness, even though the other models explicitly impose fairness constraints. We hypothesized two reasons for this. The first is model bias, which makes an estimated distribution different from a true distribution. The second reason is a deterministic decision rule. Though class labels are in fact chosen according to a deterministic decision rule, nonCV2NB methods assume that the labels are probabilistically generated.
3.2.1 Model bias
3.2.2 A deterministic decision rule
3.3 Modelbased independence and actual independence
Based on the above discussion of the influences of model bias and a deterministic decision rule, we here formalize the notions of modelbased independence and actual independence. Figure 4 shows the subspaces required for these two types of independence. A common model subspace, \(\mathop {\hat{\Pr }}[Y, \mathbf {X}, S]\), depicted by the horizontal plane in the figure, is shared in both types of independence. On the other hand, as depicted by the two vertical planes in the figure, there are two distinct fair subspaces. The two fair subspaces are the same from the standpoint that they satisfy unconditional independence between a class variable and a sensitive feature, but their distributions generating class labels differ. In the case of modelbased independence, class labels are directly generated from a distribution on the modelsubspace. However, in the case of actual independence, class labels are generated from a distribution induced by taking into account the influence of model bias and a decision rule in the real world. For each type of independence, we provide a procedure to derive the distributions generating class labels in cases of classifiers with a generative model and a discriminative model (Bishop 2006, Sect. 1.5.4).
3.3.1 Modelbased independence
As we will show in Sect. 6, classifiers satisfying this modelbased independence are poor in fairness evaluation indexes; this is due to unrealistic assumptions. Modelbased independence can be considered as a valid fairness constraint. However, the assumptions adopted in this constraint don’t match the practical use of classifiers. Specifically, this constraint is assumed to ignore the influences of model bias and a deterministic decision rule, as discussed in the previous section. Therefore, we introduce another constraint based on a more realistic assumption.
3.3.2 Actual independence
As described above, the key difference between the two types of fairness constraints, modelbased independence and actual independence, is the difference in the distributions to generate class labels. In order to show that the difference of these fairness constraints is important for fairnessaware classification, we then modify two existing fairnessaware classifiers so as to satisfy these fairness constraints.
4 A prejudice remover regularizer
We introduce a prejudice remover regularizer that constrains a modelbased independence condition. This term is then modified so as to satisfy an actual independence constraint.
5 Rejectoptionbased classification
Kamiran et al. proposed a method, reject optionbased classification (ROC), to change decision thresholds for making fairer classification (Kamiran et al. 2012). After reviewing the original ROC method, we show how to select decision thresholds to satisfy modelbased and actual independence for a naive Bayes case. We then extend our method so as to make it applicable to classifiers other than those with a generative model.
5.1 The original ROC method
Kamiran et al. discussed a theory for determining class labels based on a class posterior distribution so that a fairness constraint was satisfied (Kamiran et al. 2012). In standard classification, objects are classified to class 1 if the class posteriors satisfy the inequality \(\mathop {\hat{\Pr }}[Y{=}1  \mathbf {X}] \ge \mathop {\hat{\Pr }}[Y{=}0  \mathbf {X}]\), which is equivalent to \(\mathop {\hat{\Pr }}[Y{=}1  \mathbf {X}] \ge 1/2\). The threshold 1 / 2 is referred to as a decision threshold, and it is modified to make the decisions fair. Given a threshold parameter, \(1 > \tau \ge 1/2\), objects such that \(S{=}0\) are classified to class 1 if \(\mathop {\hat{\Pr }}[Y{=}1  \mathbf {X}, S{=}0] \ge 1  \tau \). Inversely, objects such that \(S{=}1\) are classified to class 1 if \(\mathop {\hat{\Pr }}[Y{=}1  \mathbf {X}, S{=}1] \ge \tau \).
In the following subsections, we slightly generalize the original ROC method. Decision thresholds are changed symmetrically in the original method, but we relax this limitation. Specifically, the thresholds are changed to \(\tau _{0} \in (0, 1)\) for an \(S=0\) group, while they are changed to \(\tau _{1} \in (0, 1)\) for an \(S=1\) group.
5.2 A ROC method satisfying modelbased independence
We here describe how to select priors for achieving modelbased independence when targeting a naive Bayes classifier. We first define a naive Bayes model satisfying a modelbased independence constraint, and parameters of the model are estimated by maximizing a likelihood. We then show that this method corresponds to a special case of the ROC method.
5.3 A ROC method satisfying actual independence
Algorithm 2 shows the outline of a ROC naive Bayes method for satisfying an actual independence constraint (a ROCNBAI method). Fundamentally, this algorithm is designed to find the best parameters by a grid search under an actual independence constraint. Because only priors are changed, all parameters other than priors are copied (line 1). The distribution of a class label obtained by applying a deterministic decision rule, Open image in new window , is temporally fixed (line 3). For the distribution, priors of naive Bayes, Open image in new window , are adjusted to satisfy the actual independence constraint Eq. (9) (line 4). Using the adjusted priors, the temporal likelihood is calculated (line 5) and is compared with the current best (line 6), and this algorithm finally outputs the best parameters (line 9).
5.4 A universal ROC method
Next, we will extend the applicable target of the concept of actual independence. There are three types of classifiers: a generative model, a discriminative model, and a discriminant function (Bishop 2006, Sect. 1.5.4). However, the approach in the previous section is only applicable to a classifier with a generative model. To relax this restriction, we developed a procedure, which we call the universal ROC method, to make the approach applicable to all three types of classifiers.
6 Experiment
We implemented fairnessaware classifiers satisfying modelbased independence and actual independence, and empirically compared these classifiers on real benchmark datasets and a synthetic dataset. This comparison revealed the importance of an actual independence condition, which takes the effects of model bias and a deterministic decision rule into account.
6.1 Experimental conditions
We examined classifiers as described below.^{1} As baselines, we tested standard classifiers trained by using only nonsensitive features. These were three types of classifiers, naive Bayes, logistic regression, and a linear SVM (respectively, NB, LR, and SVM), which were implemented in the scikitlearn (Pedregosa et al. 2011) packages. Note that these classifiers may make potentially unfair decisions. Fairnessaware classifiers were variants of these three classifiers. Variants of naive Bayes classifiers were Calders & Verwer’s twonaiveBayes (CV2NB) in Sect. 3.1, the ROC method satisfying modelbased independence (ROCNBMI) in Sect. 5.2, and that satisfying actual independence (ROCNBAI) in Sect. 5.3. Regarding logistic regression, we adopted the prejudice remover regularizers satisfying modelbased and actual independence conditions (respectively, PRMI and PRAI) in Sect. 4, and the universal ROC (ROCLRAI) in Sect. 5.4. Finally, we tested a universal ROC using the linear SVM (ROCSVMAI) in Sect. 5.4.
6.2 Results on real benchmark datasets
Accuracy and fairness indexes for the Adult dataset
Modelbased independence  Actual independence  

Methods  Acc  NMI  CVS  Methods  Acc  NMI  CVS 
NB  0.820  \(1.11\,{\times }\,10^{01}\)  0.348  CV2NB  0.825  \(7.94\,{\times }\,10^{10}\)  0.000 
ROCNBMI  0.820  \(2.25\,{\times }\,10^{02}\)  0.160  ROCNBAI  0.837  \(8.59\,{\times }\,10^{06}\)  0.002 
LR  0.862  \(4.51\,{\times }\,10^{02}\)  0.172  PRAI  0.828  \(7.08\,{\times }\,10^{05}\)  0.008 
PRMI  0.814  \(2.57\,{\times }\,10^{02}\)  0.056  ROCLRAI  0.840  \(1.14\,{\times }\,10^{09}\)  \(\)0.000 
SVM  0.862  \(4.32\,{\times }\,10^{02}\)  0.160  ROCSVMAI  0.839  \(1.09\,{\times }\,10^{07}\)  \(\)0.000 
Accuracy and fairness indexes for the Dutch dataset
Modelbased independence  Actual independence  

Methods  Acc  NMI  CVS  Methods  Acc  NMI  CVS 
NB  0.787  \(1.83\,{\times }\,10^{02}\)  0.159  CV2NB  0.757  \(8.47\,{\times }\,10^{06 }\)  \(\)0.003 
ROCNBMI  0.808  \(8.82\,{\times }\,10^{02}\)  0.346  ROCNBAI  0.765  \(1.80\,{\times }\,10^{06}\)  \(\)0.002 
LR  0.819  \(2.19\,{\times }\,10^{02}\)  0.171  PRAI  0.716  \(4.63\,{\times }\,10^{07}\)  0.001 
PRMI  0.790  \(2.28\,{\times }\,10^{02}\)  0.161  ROCLRAI  0.778  \(5.68\,{\times }\,10^{07}\)  \(\)0.001 
SVM  0.817  \(1.89\,{\times }\,10^{02}\)  0.158  ROCSVMAI  0.777  \(2.35\,{\times }\,10^{07}\)  \(\)0.001 
We present our experimental results for the Adult dataset in Table 1 and those for the Dutch dataset in Table 2. For each dataset and each classifier, we computed three evaluation measures: accuracy (Acc), normalized mutual information (NMI), and the Caldars & Verwer score (CVS). We show the results obtained by baseline methods or methods to satisfy modelbased independence in the left half of each table, and those obtained by methods to satisfy actual independence in the right half of each table. For PRMI and PRAI methods, we chose \(3{\times }10^{1}\) and \(1{\times }10^{4}\) as an independence regularization parameter, \(\eta \), respectively.
We evaluated the accuracy and fairness of classifiers on these datasets in order to examine the following two questions. First, is the difference between modelbased independence and actual independence essential to improve the tradeoffs between accuracy and fairness? This validates the importance of the effects of model bias and deterministic decision as analyzed in Sect. 3.2. Second, can the universal ROC methods in Sect. 5.4 improve fairness effectively?
We begin with the first question: is the difference between modelbased independence and actual independence essential for the performance in fairnessaware classification? To answer this, we compared the results in the left half of the tables with those in the right half. Comparing the fairnessaware classifiers with their corresponding baseline methods, the relative losses in accuracy by satisfying actual independence were at most about \(5\%\) except for the Dutch PRAI case (\(12.5\%\)). Moreover, the prediction accuracy was improved in some cases, e.g., the ROCNBAI for the Adult dataset. In terms of fairness, the improvements were drastic. The NMIs and CVSes of the baselines were worse than \(1{\times }10^{02}\) and 0.1, respectively. On the other hand, the methods satisfying actual independence achieved better performance than the order of \(10^{04}\) in NMIs and than 0.01 in CVS.
We then compared methods satisfying actual independence with those satisfying modelbased independence, which are aligned in the same row in the tables. Specifically, the ROCNBAI was compared with the ROCNBMI, and the PRAI was compared with the PRMI. The performances in accuracy appeared to be comparable. Each of the PRAI and the ROCNBAI methods won in two cases and lost in two cases. Note that the differences were all significant at the level of 1%. In terms of fairness, methods satisfying actual independence again achieved drastic improvements. While the NMIs obtained by the ROCNBMI and PRMI methods were worse than \(10^{02}\), those obtained by ROCNBAI and PRAI were better than \(10^{04}\). In terms of CVS, the methods satisfying actual independence could achieve scores of nearly zero, but methods satisfying modelbased independence could not. From the above results, we can conclude that satisfying a constraint of actual independence, rather than a constraint of modelbased independence, improved fairness while minimizing the loss of accuracy.
We now show the results of the supplemental examination of the effects of an independence parameter \(\eta \) of prejudice removers, the PRMI and the PRAI, to adjust the balance between accuracy and fairness. Figure 5 shows the change of performance in accuracy and fairness depending on the parameter \(\eta \). The increase of \(\eta \) generally worsened accuracy and improved fairness as we intended. The PRMI method failed if \(\eta > 10^{2}\) because all the data were classified into one class, while the PRAI method worked relatively stably even for larger \(\eta \). Therefore, we chose \(\eta {=} 3{\times }10^{1}\), at the point where just before the accuracy started to fall, for the PRMI, and chose \(\eta {=}10^{4}\), at which Acc and NMI became saturated, for the PRAI. Note that NMIs were unstable for large \(\eta \) because the nonconvexity of a prejudice remover regularizer made it difficult to optimize the objective function.
Additional accuracy indexes and estimated positive ratios for the Adult dataset
Modelbased independence  Actual independence  

Methods  EPR  Acc  Precision  Reccall  Methods  EPR  Acc  Precision  Recall 
NB  0.323  0.820  0.584  0.803  CV2NB  0.234  0.825  0.628  0.627 
ROCNBMI  0.304  0.820  0.591  0.763  ROCNBAI  0.162  0.837  0.722  0.497 
ROCNBFF  0.235  0.825  0.628  0.629  
LR  0.185  0.862  0.762  0.599  PRAI  0.241  0.828  0.631  0.646 
PRMI  0.050  0.814  0.986  0.211  ROCLRAI  0.169  0.840  0.722  0.520 
ROCLRFF  0.235  0.833  0.44  0.645  
SVM  0.170  0.862  0.786  0.568  ROCSVMAI  0.174  0.839  0.711  0.528 
ROCSVMFF  0.236  0.832  0.641  0.644 
Additional accuracy indexes and estimated positive ratios for the Dutch dataset
Modelbased independence  Actual independence  

Methods  EPR  Acc  Precision  Reccall  Methods  EPR  Acc  Precision  Recall 
NB  0.496  0.787  0.765  0.797  CV2NB  0.474  0.757  0.746  0.742 
ROCNBMI  0.493  0.808  0.788  0.815  ROCNBAI  0.401  0.765  0.801  0.674 
ROCNBFF  0.477  0.758  0.745  0.746  
LR  0.422  0.819  0.850  0.753  PRAI  0.635  0.716  0.652  0.869 
PRMI  0.320  0.790  0.916  0.616  ROCLRAI  0.433  0.778  0.793  0.721 
ROCLRFF  0.476  0.774  0.763  0.763  
SVM  0.404  0.817  0.863  0.733  ROCSVMAI  0.436  0.777  0.790  0.724 
ROCSVMFF  0.477  0.774  0.762  0.763 

Fairness could be drastically improved with less sacrifice in accuracy by satisfying actual independence instead of modelbased independence. This implies the importance of the effects of model bias and a deterministic decision rule in terms of fairness.

The universal ROC method worked as well as the other fairnessaware classifiers, and any type of classifier could be modified to a fairnessaware classifier.
6.3 Results for a synthetic dataset
We here investigate whether class labels generated by distributions on a fairsubspace can be estimated by fairnessaware classifiers. In the previous section, we examined accuracy to evaluate how correctly unfair labels were predicted. However, we really want to evaluate how correctly fair labels were predicted. Because such fair labels cannot be observed in real datasets, we will use a synthetic dataset to test accuracy for the fair labels.
Accuracy and fairness indexes for a synthetic dataset
Modelbased independence  Actual independence  

Methods  FAcc  UAcc  CVS  Methods  FAcc  UAcc  CVS 
NB  0.891  0.965  0.183  CV2NB  0.932  0.908  0.014 
ROCNBMI  0.909  0.945  0.126  ROCNBAI  0.925  0.911  0.013 
LR  0.894  0.984  0.189  PRAI  0.906  0.924  0.020 
PRMI  0.920  0.928  0.033  ROCLRAI  0.922  0.933  0.012 
SVM  0.893  0.980  0.189  ROCSVMAI  0.922  0.929  0.012 
We tested the same set of classifiers tested in the previous section on synthetic datasets generated by the procedure described above. 100 pairs of datasets were generated: one of each pair was used for training, and the other was used for testing. Note that only unfair labels were used in training. Table 5 shows the mean accuracies over 100 datasets for both fair and unfair labels, denoted by FAcc and UAcc, respectively. Means of absolutes of the fairness indexes between predicted labels and sensitive values, CVS, are also shown. Note that we did not show means of NMI because they are meaningless due to their large variance over 100 datasets.
We first focus on FAcc and UAcc. All the standard classifiers could successfully predict unfair labels, but performed poorly in predicting fair labels. Inversely, all fairnessaware classifiers could improve the accuracy on fair labels, but worsened the accuracy on unfair labels, compared to their corresponding standard classifiers, e.g., NB for CV2NB. Further, in terms of CV2NB and ROCNBAI, the accuracies on fair labels were better than those on unfair labels. These results were what we intended, because standard classifiers and fairnessaware classifiers were designed to predict unfair and fair labels, respectively. We next discuss the fairness index, CVS. All fairnessaware classifiers could make fairer decisions than their corresponding standard classifiers, as we intended. In addition, classifiers satisfying actual independence exhibited greater fairness than those satisfying modelbased independence. CVS for ROCNBAI was smaller than that for ROCNBAI, and PRAI classified more fairly than PRMI. This proved the advantage of achieving actual independence.

Fairnessaware classifiers performed better than their corresponding standard classifiers in terms of accuracy on fair labels and in fairness indexes.

Classifiers satisfying actual independence could make fairer decisions than those satisfying modelbased independence.
7 Related work
This section reviews fairnessaware classifiers. Figure 6 geometrically represents approaches to fairnessaware classification as in Fig. 1. Approaches to fairnessaware classification can be classified into three types (Ruggieri et al. 2010): preprocess, inprocess, and postprocess. In the preprocess approach, potentially unfair data are mapped onto the fair subspace (\(\textcircled {a}\) in Fig. 6), and the fair model is learned by a standard classifier (\(\textcircled {b}\)). Any classifier can in principle be used in this approach, but the development of a mapping method might be difficult without making any assumption on a classifier. In particular, we consider that actual independence will not be satisfied without specifying a classifier. Massaging is a technique to relabel a dataset based on the predicted probability of class labels (Kamiran and Calders 2012). Hajian and DomingoFerrer (2013) changed labels or sensitive features by exploiting frequent pattern mining. Zemel et al. (2013) tried to obtain an intermediate representation that fulfilled three constraints: statistical parity, minimizing the distortion, and maximizing the classification accuracy. Feldman et al. (2015) proposed a method to transform nonsensitive features so that a sensitive feature cannot be predicted from the transformed nonsensitive features.
In the inprocess approach, a fair model is learned directly from a potentially unfair dataset as in \(\textcircled {c}\) in Fig. 6. This approach can potentially achieve better tradeoffs than the other approaches because classifiers are less restricted in their design. However, it is technically difficult to formalize or optimize an objective function. In addition, for each distinct type of classifier, its fair variant must be developed. The prejudice remover in Sect. 4 is categorized into this approach. Kamiran et al. (2010) developed algorithms to learn decision trees for a fairnessaware classification task, in which the labels at leaf nodes were changed so as to decrease the CVS. Fukuchi et al. introduced two constraint terms, \(\eta \) neutrality (Fukuchi et al. 2013) and neutrality risk (Fukuchi and Sakuma 2014). Zafar et al. (2015) developed SVMs and logistic regression with constraint terms that make classes uncorrelated (instead of independent) with a sensitive feature. They also proposed a classifier to satisfy a fairness condition that misclassification rates for groups sharing the same sensitive values were equal (Zafar et al. 2017).
In the postprocess approach, a standard classifier is first learned (\(\textcircled {d}\)), and then the learned classifier is modified to satisfy a fairness constraint (\(\textcircled {e}\)). This approach adopts the rather restrictive assumption, obliviousness (Hardt et al. 2016), that fair class labels are determined based only on labels of a standard classifier and a sensitive value, and are independent from nonsensitive features. However, this obliviousness assumption makes the development of a fairnessaware classifier easier. Calders & Verwer’s twonaiveBayes method in Sect. 3.1 and the ROC method in Sect. 5.1 are categorized into this approach. Kamiran et al. discussed the relabeling technique for fairer decisions while considering the effects of confounding variables (Kamiran et al. 2013). Hardt et al. (2016) developed a postprocessstyle method to match misclassification rates between groups.
Finally, we will review other aspects of fairnessaware classification. Fairnessaware data mining is an emerging research topic and involves many controversial problems. Hajian et al. provide a good tutorial on the relevant literature (Hajian et al. 2016). When using a fairnessaware classifier, a sensitive feature may not be provided for various reasons, such as the protection of privacy. To alleviate this problem, Fukuchi et al. (2013) proposed to use a predictor for a sensitive feature, learned from an independent dataset. In Sweeney (2013), to investigate the fairness online of ad delivery, a sensitive feature, race, is predicted from an independent public dataset, the birth records of the state of California. Even if both a class and a sensitive feature depend on a common factor, the use of the factor in classification is legal for various reasons, such as a genuine occupational requirement. In the context of fairnessaware data mining, such a factor is referred to as an explainable variable (Kamiran et al. 2013). Given such an explainable variable, \(\mathbf {E}\), a fair constraint can be relaxed from unconditional independence, Open image in new window , to conditional independence, Open image in new window . Because an explainable variable can be treated as a confounding variable in a causal inference context, a propensity score is used to maintain the effect of a explainable variable (Calders et al. 2013).
8 Conclusions
In this paper, we discussed an independence condition in terms of a fairnessaware classifier. We proposed notions of modelbased and actual independence, in which the treatments of model bias and a decision rule are different. We then developed two types of pairs of classifiers, one of which achieves modelbased independence and the other actual independence. Empirical comparison of these pairs of classifiers validated that the distinction of two types of independence is essential for improving tradeoffs between fairness and accuracy. Finally, We extended an approach exploited in the ROC method to make it applicable to any type of classifiers.
Though we can now achieve a higher level of fairness by satisfying an actual independence condition, the time complexity of algorithms must be improved in the future. Due to the discrete property of a deterministic decision rule, the objective function to optimize becomes indifferentiable, and this fact makes it difficult to find optimal parameters. Approximation and relaxation techniques would be helpful for alleviating this problem.
Footnotes
 1.
Our implementations of these methods are available at http://www.kamishima.net/faclass/.
 2.
Notes
Acknowledgements
We wish to thank Dr. Sicco Verwer for providing detailed information about his work, Dr. Žliobaitė for providing datasets, and anonymous reviewers for their helpful suggetions to improve the clarity of this paper. This work is supported by MEXT/JSPS KAKENHI Grant Numbers JP24500194, JP15K00327, and JP16H02864.
References
 Berendt B, Preibusch S (2012) Exploring discrimination: A usercentric evaluation of discriminationaware data mining. In: Proceedings of the IEEE Int’l Workshop on Discrimination and PrivacyAware Data Mining, pp 344–351Google Scholar
 Bishop CM (2006) Pattern Recognition and Machine Learning. Information Science and Statistics. Springer, New YorkzbMATHGoogle Scholar
 Calders T, Verwer S (2010) Three naive Bayes approaches for discriminationfree classification. Data Min Knowl Discov 21:277–292MathSciNetCrossRefGoogle Scholar
 Calders T, Karim A, Kamiran F, Ali W, Zhang X (2013) Controlling attribute effect in linear regression. In: Proceedings of the 13th IEEE Int’l Conference on Data Mining, pp 71–80Google Scholar
 Dwork C, Hardt M, Pitassi T, Reingold O, Zemel R (2012) Fairness through awareness. In: Proceedings of the 3rd Innovations in Theoretical Computer Science Conference, pp 214–226Google Scholar
 Elkan C (2001) The foundations of costsensitive learning. In: Proceedings of the 17th Int’l Joint Conference on Artificial Intelligence, pp 973–978Google Scholar
 Feldman M, Friedler SA, Moeller J, Scheidegger C, Venkatasubramanian S (2015) Certifying and removing disparate impact. In: Proceedings of the 21st ACM SIGKDD Int’l Conference on Knowledge Discovery and Data Mining, pp 259–268Google Scholar
 Frank A, Asuncion A (2010) UCI machine learning repository. University of California, Irvine, School of Information and Computer Sciences, http://archive.ics.uci.edu/ml
 Fukuchi K, Sakuma J (2014) Neutralized empirical risk minimization with generalization neutrality bound. In: Proceedings of the ECML PKDD 2014, Part I, pp 418–433 [LNCS 8724]Google Scholar
 Fukuchi K, Sakuma J, Kamishima T (2013) Prediction with modelbased neutrality. In: Proceedings of the ECML PKDD 2013, Part II, pp 499–514 [LNCS 8189]Google Scholar
 Hajian S, DomingoFerrer J (2013) A methodology for direct and indirect discrimination prevention in data mining. IEEE Trans Knowl Data Eng 25(7):1445–1459CrossRefGoogle Scholar
 Hajian S, Bonchi F, Castillo C (2016) Algorithmic bias: from discrimination discovery to fairnessaware data mining. The 22nd ACM SIGKDD Int’l Conference on Knowledge Discovery and Data Mining, TutorialGoogle Scholar
 Hardt M, Price E, Srebro N (2016) Equality of opportunity in supervised learning. In: Advances in Neural Information Processing Systems 29Google Scholar
 Kamiran F, Calders T (2012) Data preprocessing techniques for classification without discrimination. Knowl Inf Syst 33:1–33CrossRefGoogle Scholar
 Kamiran F, Calders T, Pechenizkiy M (2010) Discrimination aware decision tree learning. In: Proceedings of the 10th IEEE Int’l Conferene on Data Mining, pp 869–874Google Scholar
 Kamiran F, Karim A, Zhang X (2012) Decision theory for discriminationaware classification. In: Proceedings of the 12th IEEE Int’l Conference on Data Mining, pp 924–929Google Scholar
 Kamiran F, Žliobaitė I, Calders T (2013) Quantifying explainable discrimination and removing illegal discrimination in automated decision making. Knowl Inf Syst 35:613–644CrossRefGoogle Scholar
 Kamishima T, Akaho S, Asoh H, Sakuma J (2012) Fairnessaware classifier with prejudice remover regularizer. In: Proceedings of the ECML PKDD 2012, Part II, pp 35–50 [LNCS 7524]Google Scholar
 Kamishima T, Akaho S, Asoh H, Sakuma J (2013) The independence of the fairnessaware classifiers. In: Proceedings of the IEEE 13th Int’l Conference on Data Mining Workshops, pp 849–858Google Scholar
 Pedregosa F, et al (2011) Scikitlearn: Machine learning in python. Journal of Machine Learning Research 12:2825–2830, http://scikitlearn.org
 Pedreschi D, Ruggieri S, Turini F (2008) Discriminationaware data mining. In: Proceedings of the 14th ACM SIGKDD Int’l Conference on Knowledge Discovery and Data Mining, pp 560–568Google Scholar
 Ruggieri S, Pedreschi D, Turini F (2010) Data mining for discrimination discovery. ACM Transactions on Knowledge Discovery from Data 4(2):Article 9Google Scholar
 Sweeney L (2013) Discrimination in online ad delivery. Commun ACM 56(5):44–54CrossRefGoogle Scholar
 Zafar MB, Martinez IV, Rodriguez MG, Gummadi K (2015) Fairness constraints: A mechanism for fair classification. In: ICML2015 Workshop: Fairness, Accountability, and Transparency in Machine LearningGoogle Scholar
 Zafar MB, Valera I, Rogriguez MG, Gummadi KP (2017) Fairness beyond disparate treatment & disparate impact: Learning classification without disparate mistreatment. In: Proceedings of the 26th Int’l Conference on World Wide Web, pp 1171–1180Google Scholar
 Zemel R, Wu Y, Swersky K, Pitassi T, Dwork C (2013) Learning fair representations. In: Proceedings of the 30th Int’l Conference on Machine Learning, pp 325–333Google Scholar
 Žliobaitė I (2015) On the relation between accuracy and fairness in binary classification. In: ICML2015 Workshop: Fairness, Accountability, and Transparency in Machine LearningGoogle Scholar
Copyright information
Open AccessThis article is distributed under the terms of the Creative Commons Attribution 4.0 International License (http://creativecommons.org/licenses/by/4.0/), which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.