Making individually fair predictions with causal pathways

Abstract

Machine learning is being increasingly used to make algorithmic decisions that have strong societal impact on people’s lives. Due to their huge societal impact, such algorithmic decisions need to be accurate and fair with respect to sensitive features, including race, gender, religion, and sexual orientation. To achieve a good balance between prediction accuracy and fairness, causality-based methods have been proposed, which utilize a causal graph with unfair pathways. However, none of these methods can ensure fairness for each individual without making restrictive functional assumptions about the data generating processes, which are not satisfied in many cases. In this paper, we propose a far more practical causality-based framework for learning an individually fair classifier. To avoid impractical functional assumptions, we introduce a new criterion, the probability of individual unfairness, and derive its upper bound that can be estimated from data. We then train a classifier by solving an optimization problem where the upper bound value is forced to be close to zero. We elucidate why solving such an optimization problem can guarantee fairness for each individual. Moreover, we provide two extensions for dealing with challenging real-world scenarios where there are unobserved variables called latent confounders, and the true causal graph is uncertain. Experimental results show that our method can learn an individually fair classifier at a slight cost of prediction accuracy.

Appendices

Differences from previous conference publication

This article is an expanded version of a previous conference publication (Chikahara et al. 2021). The following are the main differences from the conference publication:

• We extended the framework described in the conference publication to address real-world scenarios where the true causal graph is uncertain (Sect. 6.2). To test this extended framework, we performed synthetic data experiments (Sect. 7.3.2) and confirmed that the performance of our extended framework is comparable to the original one with a true causal graph and better than the original one with a misspecified causal graph.

• We improved its clarity by presenting a more detailed explanation of our fairness measure (Sect. 2). We first introduced two example real-world scenarios (i.e., Example 1 and Example 2) in Sect. 2.2. Then based on these scenarios, we defined SEM (Definition 2.1) in Sect. 2.3.1 and provided a formulation of potential outcomes and path-specific causal effects for each scenario in Sect. 2.3.2.

• We enhanced the reliability of our experiments by conducting additional, extensive experiments (Sect. 7.2 and Sect. 7.3). In the conference publication (Chikahara et al. 2021), to confirm the robustness of our method, we used 10 randomly generated synthetic datasets and computed the mean and the standard deviation of the test accuracy and unfair effects. However, we did not evaluate such mean and standard deviations in real-world data experiments. To improve the trustworthiness, for all the experiments, we performed 20 runs by randomly splitting each dataset into training and test data and computed the mean and the standard deviation of the test accuracy and unfair effects.

Definition of path-specific causal effects

In this section, we provide a formal definition of path-specific causal effects (Avin et al. 2005).

Using interventional SEM ${\mathscr {M}}^p_{A=a}$ ($a \in \{0, 1\}$) defined in Sect. 2.3.1, we define path-specific causal effects (see the original paper (Avin et al. 2005) for details). As already described in Sect. 2.3.2, a path-specific causal effect is formulated by the difference between two potential outcomes, $Y_{A \Leftarrow 0}$ and $Y_{A \Leftarrow 1 \parallel \pi }$, as $Y_{A \Leftarrow 1 \parallel \pi }- Y_{A \Leftarrow 0}$.

To define potential outcome $Y_{A \Leftarrow 0}$, we consider interventional SEM ${\mathscr {M}}^p_{A=0}$, which is obtained by simply performing intervention $do(A=0)$. Suppose that this SEM expresses each variable $V \in \{{{\varvec{X}}}, Y\}$ by the following structural equation:

$$\begin{aligned} V = f_V ({{\varvec{pa}}}(V)_{A=0}, {{\varvec{U}}}_V), \end{aligned}$$

(37)

where ${{\varvec{pa}}}(V)_{A=0}$ denotes variables ${{\varvec{pa}}}(V)$ (i.e., parents of variable V), whose values are determined by interventional SEM ${\mathscr {M}}^p_{A=0}$. Then potential outcome $Y_{A \Leftarrow 0}$ is defined as prediction Y, whose structural equation is expressed by (37) where function $f_Y$ is given by classifier $h_{\theta }$.

By contrast, to define potential outcome $Y_{A \Leftarrow 1 \parallel \pi }$, we need an SEM that is modified using interventional SEMs ${\mathscr {M}}^p_{A=0}$ and ${\mathscr {M}}^p_{A=1}$. To formulate this modified SEM, for each variable $V \in \{{{\varvec{X}}}, Y\}$, we partition its parents ${{\varvec{pa}}}(V)$ into two subsets, ${{\varvec{pa}}}(V) = \{{{\varvec{pa}}}(V)^{\pi }, {{\varvec{pa}}}(V)^{{\overline{\pi }}}\}$, where ${{\varvec{pa}}}(V)^{\pi }$ is the members of ${{\varvec{pa}}}(V)$ connected with V on unfair pathways $\pi$, and ${{\varvec{pa}}}(V)^{{\overline{\pi }}}$ is a complementary set (i.e., ${{\varvec{pa}}}(V)^{{\overline{\pi }}} = {{\varvec{pa}}}(V) \backslash {{\varvec{pa}}}(V)^{\pi }$). Based on these two subsets, we consider the following structural equation over $V \in \{{{\varvec{X}}}, Y\}$:

$$\begin{aligned} V = f_V ({{\varvec{pa}}}(V)^{\pi }_{A=1}, {{\varvec{pa}}}(V)^{{\overline{\pi }}}_{A=0}, {{\varvec{U}}}_V), \end{aligned}$$

(38)

where ${{\varvec{pa}}}(V)^{\pi }_{A=1}$ is a set of the variables in ${{\varvec{pa}}}(V)^{\pi }$ whose values are determined by interventional model ${\mathscr {M}}^p_{A=1}$, and ${{\varvec{pa}}}(V)^{{\overline{\pi }}}_{A=0}$ is a set of the variables in ${{\varvec{pa}}}(V)^{{\overline{\pi }}}$ whose values are provided by ${\mathscr {M}}^p_{A=0}$. Then potential outcome $Y_{A \Leftarrow 1 \parallel \pi }$ is defined as prediction Y, whose structural equation is represented by (38).

Derivation of Eqs. (24) and (25)

Following the original paper (Huber 2014), we derived the following formulation of the existing estimators of the marginal potential outcome probabilities:

$$\begin{aligned} \begin{aligned} {\hat{p}}^{A \Leftarrow 0}_{\theta }&= \frac{1}{n} \sum _{i=1}^n {{\,\mathrm{{\textbf{1}}}\,}}(a_i = 0) {\hat{w}}^{A \Leftarrow 0}_ic_{\theta }(a_i, q_i, d_i, m_i), \quad \\ {\hat{p}}^{A \Leftarrow 1 \parallel \pi }_{\theta }&= \frac{1}{n} \sum _{i=1}^n {{\,\mathrm{{\textbf{1}}}\,}}(a_i = 1) {\hat{w}}^{A \Leftarrow 1 \parallel \pi }_ic_{\theta }(a_i, q_i, d_i, m_i), \end{aligned} \end{aligned}$$

(24)

where $c_{\theta }({{\varvec{X}}}) = {{\,\mathrm{\textrm{P}}\,}}(Y=1 | {{\varvec{X}}})$ is the conditional distribution given by classifier $h_{\theta }$, ${{\,\mathrm{{\textbf{1}}}\,}}(\cdot )$ is an indicator function, and ${\hat{w}}^{A \Leftarrow 0}_i$ and ${\hat{w}}^{A \Leftarrow 1 \parallel \pi }_i$ are the following weights:

$$\begin{aligned}&{\hat{w}}^{A \Leftarrow 0}_i= \frac{1}{{\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=0 | q_i)}, \\&{\hat{w}}^{A \Leftarrow 1 \parallel \pi }_i= \frac{{\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=1 | q_i, d_i) {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=0 | q_i, d_i, m_i)}{{\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=1 | q_i) {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=0 | q_i, d_i) {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=1 | q_i, d_i, m_i)}, \end{aligned}$$

(25)

where ${\hat{{{\,\mathrm{\textrm{P}}\,}}}}$ is the conditional distribution that is estimated by learning the statistical models (e.g., neural networks) to the training data beforehand.

Following the notations in the original paper (Huber 2014), let the potential outcomes denote $Y_{A \Leftarrow 0}= Y(0, D(0), M(0))$ and $Y_{A \Leftarrow 1 \parallel \pi }= Y(1, D(1), M(0))$.

Then with the causal graph in Fig. 2c, marginal probability ${{\,\mathrm{\textrm{P}}\,}}(Y_{A \Leftarrow 1 \parallel \pi }= 1)$ can be written as

$$\begin{aligned}&{{\,\mathrm{\textrm{P}}\,}}(Y_{A \Leftarrow 1 \parallel \pi }= 1) \\&\quad = {{\,\mathrm{\textrm{P}}\,}}(Y(1, D(1), M(0) ) = 1) \\&\quad = {{\,\mathrm{{\mathbb {E}}}\,}}_{Q}[ {{\,\mathrm{{\mathbb {E}}}\,}}_{D(1) | Q}[ {{\,\mathrm{{\mathbb {E}}}\,}}_{M(0) | Q, D(1)} [ {{\,\mathrm{\textrm{P}}\,}}( Y(1, d, m) = 1 | A = 1, Q = q, D(1) = d, \\&\qquad M(0) = m ) ] ] ] . \end{aligned}$$

Using Assumption 2, this can be rewritten as

$$\begin{aligned} {{\,\mathrm{\textrm{P}}\,}}(Y_{A \Leftarrow 1 \parallel \pi }= 1)&= {{\,\mathrm{{\mathbb {E}}}\,}}_{Q}[ {{\,\mathrm{{\mathbb {E}}}\,}}_{D | A=1, Q}[ {{\,\mathrm{{\mathbb {E}}}\,}}_{M | A=0, Q, D} [ {{\,\mathrm{\textrm{P}}\,}}( Y(1, d, m) = 1 | A = 1,\\&\qquad Q = q, D = d, M = m ) ] ] ]. \end{aligned}$$

With Bayes’ theorem, this can be expressed as

$$\begin{aligned}&{{\,\mathrm{\textrm{P}}\,}}(Y_{A \Leftarrow 1 \parallel \pi }= 1) \\&\quad = {{\,\mathrm{{\mathbb {E}}}\,}}_{Q}[ {{\,\mathrm{{\mathbb {E}}}\,}}_{D | Q}[ {{\,\mathrm{{\mathbb {E}}}\,}}_{M | Q, D} [ {\omega }^{A \Leftarrow 1 \parallel \pi }{{\,\mathrm{\textrm{P}}\,}}( Y = 1 | d A = 1, Q = q, D = d, M = m ) ] ] ], \end{aligned}$$

where ${\omega }^{A \Leftarrow 1 \parallel \pi }$ is expressed as follows:

$$\begin{aligned}&{\omega }^{A \Leftarrow 1 \parallel \pi }= \frac{{{\,\mathrm{\textrm{P}}\,}}(A=1 | Q=q, D=d){{\,\mathrm{\textrm{P}}\,}}(A=0 | Q=q, D=d, M=m)}{{{\,\mathrm{\textrm{P}}\,}}(A=1 | Q=q) {{\,\mathrm{\textrm{P}}\,}}(A=0 | Q=q, D=d)}. \end{aligned}$$

With indicator function ${{\,\mathrm{{\textbf{1}}}\,}}(\cdot )$, this can be formulated as

$$\begin{aligned}&{{\,\mathrm{\textrm{P}}\,}}(Y_{A \Leftarrow 1 \parallel \pi }= 1) = {{\,\mathrm{{\mathbb {E}}}\,}}[ {{\,\mathrm{{\textbf{1}}}\,}}(A=1) w^{A \Leftarrow 1 \parallel \pi }{{\,\mathrm{\textrm{P}}\,}}( Y = 1 | A = 1, q, d, m ) ] , \end{aligned}$$

(39)

where weight $w'$ is expressed as

$$\begin{aligned} w^{A \Leftarrow 1 \parallel \pi }= \frac{1}{{{\,\mathrm{\textrm{P}}\,}}(A=1 | Q=q, D=d, M=m)} {\omega }^{A \Leftarrow 1 \parallel \pi }. \end{aligned}$$

In a similar manner, marginal probability ${{\,\mathrm{\textrm{P}}\,}}(Y_{A \Leftarrow 0}= 1)$ can be represented as

$$\begin{aligned}&{{\,\mathrm{\textrm{P}}\,}}(Y_{A \Leftarrow 0}= 1) = {{\,\mathrm{{\mathbb {E}}}\,}}[ {{\,\mathrm{{\textbf{1}}}\,}}(A=0) w^{A \Leftarrow 0}{{\,\mathrm{\textrm{P}}\,}}( Y = 1 | A = 0, q, d, m ) ] , \end{aligned}$$

(40)

where weight $w^{A \Leftarrow 0}$ is formulated as

$$\begin{aligned} w^{A \Leftarrow 0}= \frac{1}{{{\,\mathrm{\textrm{P}}\,}}(A=0 | Q=q)}. \end{aligned}$$

Given empirical distribution, by plugging conditional distribution $c_{\theta }$ into ${{\,\mathrm{\textrm{P}}\,}}(Y=1 | A=1, Q=q, D=d, M=m)$, we can estimate (40) and (39) as (24).

Computation time and convergence guarantee

We minimize the objective function (1) with the stochastic gradient descent method (Sutskever et al. 2013).

To do so, we need to compute the penalty term and its gradient over the samples in each minibatch. However, adding the penalty function to the loss function does not increase the complexity of computing the objective function value and the gradient very much.

Whether we can guarantee that the gradient descent method converges depends on the choice of classifier $h_{\theta }$. For instance, if we choose a neural network classifier, we cannot guarantee that the stochastic gradient descent method (Sutskever et al. 2013) converges because the objective function (1) becomes nonconvex, and its gradient does not become Lipschitz continuous; that is, the maximum rate of change in the gradient is not bounded. However, if the neural network only contains activation functions whose gradients are Lipschitz continuous (e.g., the sigmoid function), since in this case, the gradient of the objective function becomes locally Lipschitz continuous (Ferrera 2013, Chapter 2), we can optimize the objective function with convergence guarantees using e.g., the gradient sampling method (Burke et al. 2005).

Proof of (29)

Since we already proved the upper bound in (14), below we derive the lower bound in it. Since $\alpha$ and $\beta$ are marginal probabilities, we have

$$\begin{aligned} \begin{aligned} p_{10}+ p_{11}= \alpha , \quad p_{01}+ p_{11}= \beta , \end{aligned} \end{aligned}$$

which are equivalent to

$$\begin{aligned} \begin{aligned} p_{10}= \alpha - p_{11}, \quad p_{01}= \beta - p_{11}, \end{aligned} \end{aligned}$$

respectively. By summing up both, we have

$$\begin{aligned} p_{01}+ p_{10}= \alpha + \beta - 2 p_{11}. \end{aligned}$$

Since joint probability $p_{11}$ is less than marginal probabilities $\alpha$ and $\beta$, we have $p_{11}\le \min \{\alpha , \beta \}$. Therefore,

$$\begin{aligned}&p_{01}+ p_{10}\ge \alpha + \beta - 2 \min \{\alpha , \beta \} = |\alpha - \beta |. \end{aligned}$$

(41)

Combined with the upper bound on $p_{01}+ p_{10}$ in (14), we obtain (29).

Experimental settings

This section details the experimental settings presented in Sect. 7.

1.1 Settings in synthetic data experiments

1.1.1 Data

In synthetic data experiments, we used four datasets: Synth1, Synth2, Synth3, and Synth4.

Synth1 and Synth2 datasets were generated based on a scenario of hiring decisions for physically demanding jobs, described in Example 2 in Sect. 2.2, whose causal graph is shown in Fig. 2c. As described below, while the Synth2 dataset follows the functional assumption of the PSCF method, the Synth1 dataset does not.

To prepare the Synth1 dataset, we sampled gender $A \in \{0, 1\}$, qualification Q, number of children D, physical strength M, and hiring decision outcome $Y \in \{0, 1\}$ from the following SEM:

$$\begin{aligned} \begin{aligned}&A = U_A, \quad U_A \sim \textrm{Bernoulli}(0.6), \\&Q = \lfloor U_Q \rfloor , \quad U_Q \sim {\mathcal {N}}(2, 5^2), \\&D = A + \lfloor 0.5 Q U_D \rfloor , \quad U_D \sim \textrm{Tr}{\mathcal {N}}(1, 0.5^2, 0.1, 3.0),\\&M = 3 A + 0.4 Q U_M, \quad U_M \sim \textrm{Tr}{\mathcal {N}}(1.5, 0.5^2, 0.1, 3.0), \\&Y = h(A, Q, D, M), \end{aligned} \end{aligned}$$

(42)

where $\textrm{Bernoulli}$, ${\mathcal {N}}$, and $\textrm{Tr}{\mathcal {N}}$ represent the Bernoulli, Gaussian, and truncated Gaussian distributions, respectively, and $\lfloor \cdot \rfloor$ is a floor function that returns an integer by removing the decimal places. To output hiring decision outcome Y, we used function h, which is a logistic regression model that provides the following conditional distribution:

$$\begin{aligned}&{{\,\mathrm{\textrm{P}}\,}}(Y=1|A, Q, D, M) = \textrm{Bernoulli}(\varsigma ( -10 + 5A + Q + D + M) ), \end{aligned}$$

where $\varsigma (x)$ $=$ $1 / (1 + \textrm{exp}(-x))$ is a standard sigmoid function. Note that SEM (42) does not satisfy the functional assumption of the PSCF method because the structural equations over D and M are not expressed by additive noise models (Hoyer et al. 2009) due to multiplicative noises $U_D$ and $U_M$.

By contrast, the Synth2 dataset is generated using the following SEM, which follows the functional assumption of the PSCF method:

$$\begin{aligned} \begin{aligned}&A = U_A, \quad U_A \sim \textrm{Bernoulli}(0.6), \\&Q = \lfloor U_Q \rfloor , \quad U_Q \sim {\mathcal {N}}(5, 2.5^2), \\&D = A + \lfloor 0.1 Q + U_D \rfloor , \quad U_D \sim \lfloor {\mathcal {N}}(1, 0.5^2) \rfloor ,\\&M = 3 A + 0.4 Q + U_M, \quad U_M \sim \lfloor {\mathcal {N}}(1, 0.5^2) \rfloor , \\&Y = h(A, Q, D, M), \end{aligned} \end{aligned}$$

(43)

where function h is given by the following conditional distribution:

$$\begin{aligned}&{{\,\mathrm{\textrm{P}}\,}}(Y=1|A, Q, D, M)\\&\quad = \textrm{Bernoulli}(\varsigma ( -10 + 2A + 2Q + 2D + 2M) ). \end{aligned}$$

The Synth3 dataset is associated with the causal graph in Fig. 8, which contains latent confounder H. To prepare this dataset, we considered the following SEM:

$$\begin{aligned} \begin{aligned}&A = U_A, \quad U_A \sim \textrm{Bernoulli}(0.6), \\&R = 3 A + \lfloor 10 H \rfloor + \lfloor U_R \rfloor , \quad U_R \sim {\mathcal {N}}(1, 0.5^2),\\&M = A + R + \lfloor U_M \rfloor , \quad U_M \sim {\mathcal {N}}(1, 0.5^2), \\&Y = h(A, R, M, H), \end{aligned} \end{aligned}$$

(44)

where H denotes a latent confounder, which is sampled by $H \sim {\mathcal {N}}(1, 0.5^2)$, and function h is expressed by the following conditional distribution:

$$\begin{aligned} {{\,\mathrm{\textrm{P}}\,}}(Y=1|A, R, M, H) = \textrm{Bernoulli}(\varsigma ( -15 + 3A + R + M + 5 H) ). \end{aligned}$$

We generated the Synth4 dataset using the following SEM:

$$\begin{aligned} \begin{aligned}&A = U_A, \quad U_A \sim \textrm{Bernoulli}(0.6), \\&Q_1 = \lfloor U_{Q_1} \rfloor , \quad U_{Q_1} \sim {\mathcal {N}}(2, 1^2), \\&Q_2 = \lfloor U_{Q_2} \rfloor , \quad U_{Q_2} \sim {\mathcal {N}}(2, 1^2), \\&Q_3 = A + \lfloor U_{Q_3} \rfloor , \quad U_{Q_3} \sim {\mathcal {N}}(0, 1^2), \\&D = A + \lfloor 0.1 (Q_1 + Q_2) U_D \rfloor , \quad U_D \sim {\mathcal {N}}(1, 0.5^2) \\&M = 2A + \lfloor 0.01 \textrm{exp}(Q_1) + 0.2 * (Q_2 + Q_3) ) \rfloor + \lfloor U_{M} \rfloor , \quad U_M \sim {\mathcal {N}}(1, 1^2), \\&Y = h(A, Q_1, Q_2, Q_3, D, M), \end{aligned} \end{aligned}$$

(45)

where function h is given by the following conditional distribution:

$$\begin{aligned} {{\,\mathrm{\textrm{P}}\,}}(Y=1|A, Q_1, Q_2, Q_3, D, M)&= \textrm{Bernoulli}(\varsigma ( -5 + 2A + 0.5(Q_1 + Q_2 + Q_3) \\ {}&\quad + 0.5D + 2M) ). \end{aligned}$$

1.1.2 Computing unfair effects

With such synthetic data, we computed the four statistics of unfair effects for Proposed, FIO, and Unconstrained as follows.

To compute (i) the mean unfair effect and (iii) the upper bound on PIU, we estimated marginal potential outcome probabilities ${\hat{p}}^{A \Leftarrow 0}_{\theta }$ and ${\hat{p}}^{A \Leftarrow 1 \parallel \pi }_{\theta }$ with estimators (24).

To obtain (ii) the standard deviation in the conditional mean unfair effects and (iv) the PIU value, we sampled potential outcomes $Y_{A \Leftarrow 0}$ and $Y_{A \Leftarrow 1 \parallel \pi }$ based on the SEM. For instance, in case of SEM (42), we sampled $Y_{A \Leftarrow 0}$ from the following (interventional) SEM:

$$\begin{aligned} \begin{aligned}&A = 0, \\&Q = \lfloor U_Q \rfloor , \quad U_Q \sim {\mathcal {N}}(2, 5^2), \\&D(0) = \lfloor 0.5 Q U_D \rfloor , \quad U_D \sim \textrm{Tr}{\mathcal {N}}(2, 1^2, 0.1, 3.0),\\&M(0) = 0.4 Q U_M, \quad U_M \sim \textrm{Tr}{\mathcal {N}}(3, 2^2, 0.1, 3.0), \\&Y_{A \Leftarrow 0}= h_{\theta }(0, Q, D(0), M(0)), \end{aligned} \end{aligned}$$

where $h_{\theta }$ is the classifier. We sampled $Y_{A \Leftarrow 1 \parallel \pi }$ from

$$\begin{aligned} \begin{aligned}&A = 1, \\&Q = \lfloor U_Q \rfloor , \quad U_Q \sim {\mathcal {N}}(2, 5^2), \\&D(1) = 1 + \lfloor 0.5 Q U_D \rfloor , \quad U_D \sim \textrm{Tr}{\mathcal {N}}(2, 1^2, 0.1, 3.0),\\&M(0) = 0.4 Q U_M, \quad U_M \sim \textrm{Tr}{\mathcal {N}}(3, 2^2, 0.1, 3.0), \\&Y_{A \Leftarrow 1 \parallel \pi }= h_{\theta }(1, Q, D, M). \end{aligned} \end{aligned}$$

Then using n pairs of these samples $\{(y_{A\Leftarrow 0, i}, y_{A \Leftarrow 1 \parallel \pi , i})\}_{i=1}^n$, we evaluated the PIU value by

$$\begin{aligned} {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(Y_{A \Leftarrow 0}\ne Y_{A \Leftarrow 1 \parallel \pi }) = \frac{1}{n} \sum _{i=1}^n {{\,\mathrm{{\textbf{1}}}\,}}( y_{A\Leftarrow 0, i} \ne y_{A \Leftarrow 1 \parallel \pi , i} ), \end{aligned}$$

where ${{\,\mathrm{{\textbf{1}}}\,}}(\cdot )$ is an indicator function that takes 1 if $y_{A\Leftarrow 0, i} \ne y_{A \Leftarrow 1 \parallel \pi , i}$ (i $\in$ $\{1, \ldots , n\}$) and 0 otherwise.

We computed the standard deviation in conditional mean unfair effects as follows. We separated n individuals into K subgroups who have the same values of features ${{\varvec{X}}}$, took the mean unfair effects over individuals in each subgroup, and computed their standard deviation. Let the individuals in the k-th subgroup ($k = 1, \ldots , K$) have identical feature attributes ${{\varvec{X}}}$ $=$ ${{\varvec{x}}}^k$, where superscript k represents the k-th subgroup. Then using $\{(y_{A\Leftarrow 0, i}, y_{A \Leftarrow 1 \parallel \pi , i})\}_{i=1}^n$, we estimated the standard deviation of the conditional mean unfair effects over the K subgroups as

$$\begin{aligned} {\hat{\sigma }} = \sqrt{ \frac{ \sum _{k=1}^K {\hat{\mu }}^k - {\hat{\mu }} }{K} }. \end{aligned}$$

(46)

Here ${\hat{\mu }}^k$ is the estimated conditional mean unfair effect in the k-th subgroup of individuals with identical attributes ${{\varvec{X}}}$ $=$ ${{\varvec{x}}}^k$, i.e.,

$$\begin{aligned} {\hat{\mu }}^k = \frac{1}{n^k} \sum _{i \in \{1, \ldots , n\} | {{\varvec{x}}}_i = {{\varvec{x}}}^k} {{\,\mathrm{{\textbf{1}}}\,}}(y_{A\Leftarrow 0, i}\ne y_{A \Leftarrow 1 \parallel \pi , i}), \end{aligned}$$

where $n^k$ is the number of individuals in the k-th subgroup and ${\hat{\mu }}$ is the mean of ${\hat{\mu }}^k$ over $k=1, \ldots , K$, i.e.,

$$\begin{aligned} {\hat{\mu }} = \frac{1}{K} \sum _{k}^K {\hat{\mu }}^k. \end{aligned}$$

1.1.3 Unfair effects of PSCF

With PSCF, we did not evaluate the two statistics of unfair effects, (i) the mean unfair effect and (iii) the upper bound on PIU, since they are not well-defined for this method.

These statistics measure the unfairness of the learned predictive model of Y (i.e., classifier $h_{\theta }$ in our method); however, PSCF aims to ensure fairness using unfair predictive models.

To do so, PSCF approximates the SEM by learning predictive models of each variable in ${{\,\mathrm{\textrm{P}}\,}}({{\varvec{X}}}, Y)$, which is unfair due to the discriminatory bias in the observed data, and removes the unfairness by sampling the fair feature values based on the approximated SEM.

For instance, in the case of synthetic data experiments, PSCF approximates the SEM (42) as follows. Using latent variable $H_D$, PSCF learns the predictive models of A, Q, D, M, and Y and the distribution of $H_D$, which are expressed as follows:

$$\begin{aligned} \begin{aligned}&A = {{\,\mathrm{\textrm{P}}\,}}_{\theta }(A), \\&Q = {{\,\mathrm{\textrm{P}}\,}}_{\theta }(Q), \\&D = {{\,\mathrm{\textrm{P}}\,}}_{\theta }(D | A, Q, H_D), \quad H_D = {{\,\mathrm{\textrm{P}}\,}}_{\theta }(H_D),\\&M = {{\,\mathrm{\textrm{P}}\,}}_{\theta }(M | A, Q),\\&Y = {{\,\mathrm{\textrm{P}}\,}}_{\theta }(Y | A, Q, D, M), \end{aligned} \end{aligned}$$

(47)

where ${{\,\mathrm{\textrm{P}}\,}}_{\theta }$ denotes a (conditional) distribution, which is parameterized as a neural network. Here latent variable $H_D$ approximates the additive noise in the structural equation over D.

By using the approximated SEM (47), PSCF aims to make fair predictions. For each individual i $\in$ $\{1, \ldots , n\}$ with attributes $\{a_i, q_i, d_i, m_i\}$, PSCF samples their fair attribute of D by

$$\begin{aligned} {\hat{d}}(0)_i \sim {{\,\mathrm{\textrm{P}}\,}}_{\theta }(D | A=0, q_i, h_{D, i}), \quad h_{D, i} \sim {{\,\mathrm{\textrm{P}}\,}}_{\theta }(H_D). \end{aligned}$$

(48)

Using this attribute, PSCF makes a prediction using the following Monte Carlo estimate:

$$\begin{aligned}&{\hat{y}}^{PSCF}_i = \frac{1}{J} \sum _{j=1}^J {\hat{y}}^{PSCF}_{i, j} \quad \text {where}\quad {\hat{y}}^{PSCF}_{i, j} \sim {{\,\mathrm{\textrm{P}}\,}}_{\theta }(Y | A=0, q_i, d(0)_i, m_i). \end{aligned}$$

(49)

Here J is the number of Monte Carlo samples, which is set to $J=5$ in our experiments.

According to Chiappa and Gillam (2019), if there is slight mismatch between the approximated and true SEMs (i.e., (47) and (42)), PSCF can eliminate the conditional mean unfair effect and achieve individual-level fairness. Intuitively, this is because in (48) and (49), A’s values are fixed in the approximated structural equations over Y and D, which involve unfair pathways $\pi = \{A \rightarrow Y, A \rightarrow D \rightarrow Y\}$.

In this way, PSCF aims to achieve fairness by sampling the fair feature values using unfair predictive models. Therefore, we cannot measure the unfairness of this method using the two statistics (i) and (iii), which measure the unfairness of the predictive model.

By contrast, it is appropriate to measure unfairness based on two other statistics, i.e., (ii) the standard deviation in the conditional mean unfair effects and (iv) the PIU value. Since both are formulated using the true SEM, they can be used to quantify the unfairness due to SEM’s approximation error.

To compute the unfairness of PSCF using these two statistics, we made a prediction in the same way as (49), except that we used the true SEM (42). Let such predicted values be $\{y^{PSCF}_i\}_{i=1}^n$. Then using n pairs of predicted values $\{(y^{PSCF}_{i}, {\hat{y}}^{PSCF}_{i})\}_{i=1}^n$, we estimated (iv) the PIU value as

$$\begin{aligned} {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(Y_{A \Leftarrow 0}\ne Y_{A \Leftarrow 1 \parallel \pi }) = \frac{1}{n} \sum _{i=1}^n {{\,\mathrm{{\textbf{1}}}\,}}( y^{PSCF}_{i} \ne {\hat{y}}^{PSCF}_{i} ), \end{aligned}$$

and (ii) the standard deviation of conditional mean unfair effects in the same way as (46) except that ${\hat{\mu }}^k$ is estimated as

$$\begin{aligned} {\hat{\mu }}^k = \frac{1}{n^k} \sum _{i \in \{1, \ldots , n\} | {{\varvec{x}}}_i = {{\varvec{x}}}^k} {{\,\mathrm{{\textbf{1}}}\,}}(y^{PSCF}_{i}\ne {\hat{y}}^{PSCF}_{i}). \end{aligned}$$

1.2 Settings in real-world data experiments

In real-world data experiments, to evaluate the two statistics of unfair effects (i.e., (i) and (iii)), we computed the marginal probabilities of potential outcomes ${\hat{p}}^{A \Leftarrow 0}_{\theta }$ and ${\hat{p}}^{A \Leftarrow 1 \parallel \pi }_{\theta }$ based on the existing estimators (Huber 2014).

With the German credit dataset, using the attributes of n individuals $\{a_i, {{\varvec{c}}}_i, {{\varvec{s}}}_i, {{\varvec{r}}}_i\}_{i=1}^n$, we estimated the marginal probabilities as

$$\begin{aligned}&{\hat{p}}^{A \Leftarrow 0}_{\theta }= \frac{1}{n} \sum _{i=1}^n {{\,\mathrm{{\textbf{1}}}\,}}(a_i = 0) {\hat{w}}^{A \Leftarrow 0}_ic_{\theta }(a_i, {{\varvec{c}}}_i, {{\varvec{s}}}_i, {{\varvec{r}}}_i), \nonumber \\&{\hat{p}}^{A \Leftarrow 1 \parallel \pi }_{\theta }= \frac{1}{n} \sum _{i=1}^n {{\,\mathrm{{\textbf{1}}}\,}}(a_i = 1) {\hat{w}}^{A \Leftarrow 1 \parallel \pi }_ic_{\theta }(a_i, {{\varvec{c}}}_i, {{\varvec{s}}}_i, {{\varvec{r}}}_i), \end{aligned}$$

(50)

respectively, where weights ${\hat{w}}^{A \Leftarrow 0}_i$ and ${\hat{w}}^{A \Leftarrow 1 \parallel \pi }_i$ are expressed as

$$\begin{aligned} {\hat{w}}^{A \Leftarrow 0}_i= \frac{1}{{\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=0 | {{\varvec{c}}}_i)}, \quad {\hat{w}}^{A \Leftarrow 1 \parallel \pi }_i= \frac{{\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=1 | {{\varvec{c}}}_i, {{\varvec{s}}}_i) {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=0 | {{\varvec{c}}}_i, {{\varvec{s}}}_i, {{\varvec{r}}}_i)}{{\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=1 | {{\varvec{c}}}_i) {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=0 | {{\varvec{c}}}_i, {{\varvec{s}}}_i) {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=1 | {{\varvec{c}}}_i, {{\varvec{s}}}_i, {{\varvec{r}}}_i)}. \end{aligned}$$

To compute these weights, we inferred the conditional probabilities of A by fitting the logistic regression model to the training data beforehand.

For the Adult dataset, given the attributes of n individuals $\{a_i, m_i, l_i, {{\textbf {r}}}_i, {{\varvec{c}}}_i\}_{i=1}^n$, we estimated the marginal probabilities as

$$\begin{aligned}&{\hat{p}}^{A \Leftarrow 0}_{\theta }= \frac{1}{n} \sum _{i=1}^n {{\,\mathrm{{\textbf{1}}}\,}}(a_i = 0) {\hat{w}}^{A \Leftarrow 0}_ic_{\theta }(0, m_i, l_i, {{\varvec{r}}}_i, {{\varvec{c}}}_i), \\&{\hat{p}}^{A \Leftarrow 1 \parallel \pi }_{\theta }= \frac{1}{n} \sum _{i=1}^n {{\,\mathrm{{\textbf{1}}}\,}}(a_i = 1) {\hat{w}}^{A \Leftarrow 1 \parallel \pi }_ic_{\theta }(1, m_i, l_i, {{\varvec{r}}}_i, {{\varvec{c}}}_i), \end{aligned}$$

where weights ${\hat{w}}^{A \Leftarrow 0}_i$ and ${\hat{w}}^{A \Leftarrow 1 \parallel \pi }_i$ are provided by

$$\begin{aligned}&{\hat{w}}^{A \Leftarrow 0}_i= \frac{1}{{\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=0|{{\varvec{c}}}_i)}, \\&{\hat{w}}^{A \Leftarrow 1 \parallel \pi }_i= \frac{{\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=1|m_i, {{\varvec{c}}}_i) {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=0|m_i,l_i, {{\varvec{r}}}_i, {{\varvec{c}}}_i) }{{\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=1|{{\varvec{c}}}_i) {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=0|m_i, {{\varvec{c}}}_i) {\hat{{{\,\mathrm{\textrm{P}}\,}}}}(A=1|m_i,l_i, {{\varvec{r}}}_i, {{\varvec{c}}}_i) }. \end{aligned}$$

To obtain these weight values, we estimated each conditional probability of A by fitting the logistic regression model to the data beforehand.

1.3 Computing infrastructure

In our experiments, we used Python 3.6.8 with PyTorch 1.6.0 as an implementation of the optimization algorithm (Sutskever et al. 2013) and a 64-bit CentOS machine with 2.6GHz Xeon E5-2697A-v4 (x2) CPUs and 512-GB RAM.