FaiRecSys: mitigating algorithmic bias in recommender systems

Abstract

Recommendation and personalization are useful technologies which influence more and more our daily decisions. However, as we show empirically in this paper, the bias that exists in the real world and which is reflected in the training data can be modeled and amplified by recommender systems and in the end returned as biased recommendations to the users. This feedback process creates a self-perpetuating loop which progressively strengthens the filter bubbles we live in. Biased recommendations can also reinforce stereotypes such as those based on gender or ethnicity, possibly resulting in disparate impact. In this paper we address the problem of algorithmic bias in recommender systems. In particular, we highlight the connection between predictability of sensitive features and bias in the results of recommendations and we then offer a theoretically founded bound on recommendation bias based on that connection. We continue to formalize a fairness constraint and the price that one has to pay, in terms of alterations in the recommendation matrix, in order to achieve fair recommendations. Finally, we propose FaiRecSys—an algorithm that mitigates algorithmic bias by post-processing the recommendation matrix with minimum impact on the utility of recommendations provided to the end-users.

A Proofs

1.1 A.1 Proof of Theorem 1

Fix $f \in \varPi$ and let $Y_f$ be the subset of all vectors $\mathbf{y}\in \{0,1\}^I$ for which $f(\mathbf{y}) \ne f_H(\mathbf{y})$. Let us define $Y_0 = f_H^{-1}(0) \setminus Y_f$ and $Y_1 = f_H^{-1}(1)\setminus Y_f$. Therefore, $Y_0$ is the subset of all recommendation vectors $\mathbf{y}\in \{0,1\}^I$ that both $f_H$ and f map to 0, $Y_1$ is the subset of all recommendation vectors $\mathbf{y}\in \{0,1\}^I$ that both $f_H$ and f map to 1, and $Y_f$ is the complementing subset, where $f_H$ and f disagree.

Given the partition of columns into the above-mentioned three subsets, $\{0,1\}^I = Y_0 \bigcup Y_1 \bigcup Y_f$, we denote the summation of cells of row number $i=0,1$ in the contingency table H in each of those subsets as follows:

$$\begin{aligned} r_i:= & {} \sum _{\mathbf{y}\in Y_0} H(i,\mathbf{y}) \,,\\ s_i:= & {} \sum _{\mathbf{y}\in Y_1} H(i,\mathbf{y}) \,,\\ t_i:= & {} \sum _{\mathbf{y}\in Y_f} H(i,\mathbf{y}) \,. \end{aligned}$$

Recall that $b_0$ and $b_1$ denote the number of men and women, respectively, and therefore $r_0+s_0+t_0=b_0$ and $r_1+s_1+t_1=b_1$.

Let us assume first that $Y_f$ includes just one vector $\mathbf{y}$. There are two possible cases regarding the value that $f_H$ and f assign to $\mathbf{y}$: If $\frac{t_0}{b_0} \ge \frac{t_1}{b_1}$ then $f_H(\mathbf{y})=0$ and, hence, $f(\mathbf{y})=1$. Therefore,

$$\begin{aligned} 2BER(\mathbf{f}_H, \mathbf{b})= & {} Pr[\mathbf{f}_H = 1 | \mathbf{b}= 0 ] + Pr[\mathbf{f}_H = 0 | \mathbf{b}= 1 ] \\= & {} \frac{s_0}{b_0} +\frac{r_1+t_1}{b_1}\,, \end{aligned}$$

while

$$\begin{aligned} 2BER(\mathbf{f}, \mathbf{b})= & {} Pr[\mathbf{f}= 1 | \mathbf{b}= 0 ] + Pr[\mathbf{f}= 0 | \mathbf{b}= 1 ] \\= & {} \frac{s_0+t_0}{b_0} +\frac{r_1}{b_1} \,. \end{aligned}$$

Comparing the last two equalities, we infer that $BER(\mathbf{f}, \mathbf{b}) \ge BER(\mathbf{f}_H, \mathbf{b})$ in this case. If, on the other hand, $\frac{t_0}{b_0} < \frac{t_1}{b_1}$ then $f_H(\mathbf{y})=1$ and, hence, $f(\mathbf{y})=0$. Therefore,

$$\begin{aligned} 2BER(\mathbf{f}_H, \mathbf{b})= & {} Pr[\mathbf{f}_H = 1 | \mathbf{b}= 0 ] + Pr[\mathbf{f}_H = 0 | \mathbf{b}= 1 ] \\= & {} \frac{s_0+t_0}{b_0} +\frac{r_1}{b_1}\,, \end{aligned}$$

while

$$\begin{aligned} 2BER(\mathbf{f}, \mathbf{b})= & {} Pr[\mathbf{f}= 1 | \mathbf{b}= 0 ] + Pr[\mathbf{f}= 0 | \mathbf{b}= 1 ] \\= & {} \frac{s_0}{b_0} +\frac{r_1+t_1}{b_1} \,. \end{aligned}$$

Comparing the last two equalities, we infer that $BER(\mathbf{f}, \mathbf{b}) \ge BER(\mathbf{f}_H, \mathbf{b})$ in this case as well.

This concludes the proof that $BER(\mathbf{f}_H, \mathbf{y}) \le BER(\mathbf{f}, \mathbf{y})$ when $|Y_f| = 1$. The proof for $|Y_f| > 1$ goes along the same lines. $\square$

Table 3 The a-priori contingency table H

1.2 A.2 Proof of Theorem 2

It is easy to see that the first addend on the right-hand side of Eq. (3) equals $Pr[\mathbf{f}_H = 1 | \mathbf{b}= 0 ]$, since the denominator in that fraction equals $b_0$ — the total number of men ($b(u)=0$), while the numerator equals the total number of men for whom $f_H(u)=1$. Similarly, the second addend on the right-hand side of Eq. (3) equals $Pr[\mathbf{f}_H = 0 | \mathbf{b}= 1 ]$. Hence, by Eq. (1), $BER(\mathbf{f}_H,\mathbf{b}) = \frac{\beta _H}{2}$.

In order to prove the upper bound of $\frac{1}{2}$, we introduce the following notations for $i=0,1$:

$$\begin{aligned} r_i := \sum _{\mathbf{y}\in f_H^{-1}(0)} H(i,\mathbf{y}) \,,~~ s_i: = \sum _{\mathbf{y}\in f_H^{-1}(1)} H(i,\mathbf{y}) \,. \end{aligned}$$

Namely, $r_0$ is the total number of men that $f_H$ predicted to be men, while $r_1$ is the total number of women that $f_H$ predicted to be men. Similarly, $s_0$ is the total number of men that $f_H$ predicted to be women, while $s_1$ is the total number of women that $f_H$ predicted to be women. With these notations, we infer from our discussion above that

$$\begin{aligned} BER(\mathbf{f}_H,\mathbf{b}) = \frac{1}{2} \cdot \left[ \frac{s_0}{r_0+s_0} + \frac{r_1}{r_1+s_1}\right] \,. \end{aligned}$$

As, by the definition of $f_H$, $\frac{s_0}{b_0} \le \frac{s_1}{b_1}$, where $b_0=r_0+s_0$ and $b_1=r_1+s_1$ are the total numbers of men and women, respectively, we infer that

$$\begin{aligned} BER(\mathbf{f}_H,\mathbf{b})&= \frac{1}{2} \cdot \left[ \frac{s_0}{b_0} + \frac{r_1}{b_1}\right] \le \frac{1}{2} \cdot \left[ \frac{s_1}{b_1} + \frac{r_1}{b_1}\right] \nonumber \\&= \frac{1}{2} \cdot \frac{b_1}{b_1} = \frac{1}{2} \,. \end{aligned}$$

(8)

It is easy to see that the inequality in Eq. (8) holds in the strict sense unless $f_H \equiv 0$ (in which case $s_0=s_1=0$) or $f_H \equiv 1$ (in which case $r_0=r_1=0$).

Finally, the last claim that C is $\varepsilon$-fair iff $\beta _H > 2\varepsilon$ follows directly from Theorem 1 and the equality $BER(\mathbf{f}_H,\mathbf{b}) = \frac{\beta _H}{2}$ which we just proved. That concludes the proof. $\square$

1.3 A.3 Proof of Lemma 1

The first case is trivial since here $f_H$ does not change, so BER will increase by $\frac{1}{2b_0}$ since the move will make $f_H$ wrong for the man that was moved (whereas before $f_H$ was right for that man).

To prove the second claim, we look at the contingency table before and after the move, H and $H'$, respectively, in a summarized manner as shown in Tables 3 and 4.

The notation $\#$ is used in all tables in this proof which shows the contingency table in a summarized manner in order to mark the entries that contribute to the BER. So

$$\begin{aligned} 2BER_H = \frac{y_0+v_0}{b_0} + \frac{u_1+x_1}{b_1} \end{aligned}$$

while

$$\begin{aligned} 2BER_{H'} = \frac{v_0}{b_0} + \frac{u_1+x_1+y_1}{b_1} \,. \end{aligned}$$

We infer that

$$\begin{aligned} BER_{H'}-BER_H = \frac{1}{2} \cdot \left( \frac{y_1}{b_1} -\frac{y_0}{b_0} \right) \end{aligned}$$

as claimed. That difference is indeed positive since, from the fact that $\mathbf{y}_1 \in Y_1$ (prior to the move), we know that $f_H(\mathbf{y}_1)=1$, namely that $\frac{y_1}{b_1} >\frac{y_0}{b_0}$. On the other hand, since $\mathbf{y}_1$ is unstable under addition of a man, we know that after the move $\frac{y_0+1}{b_0} \ge \frac{y_1}{b_1}$. But this implies that $\frac{1}{2} \cdot \left( \frac{y_1}{b_1} -\frac{y_0}{b_0} \right) \le \frac{1}{2b_0}$, in accord with our claim.

The proof of the third case is similar. Here, the contingency table $H'$ is as given in Table 5. So now

$$\begin{aligned} 2BER_{H'} = \frac{x_0+y_0+v_0}{b_0} + \frac{u_1}{b_1} \,, \end{aligned}$$

and, therefore,

$$\begin{aligned} BER_{H'}-BER_H = \frac{1}{2} \cdot \left( \frac{x_0}{b_0} -\frac{x_1}{b_1} \right) \ge 0\,. \end{aligned}$$

That difference is strictly smaller than $\frac{1}{2b_0}$ since $\mathbf{y}_0$ is unstable under removal of a man.

Table 4 The a-posteriori contingency table $H'$ in Case 2

Table 5 The a-posteriori contingency table $H'$ in Case 3

Table 6 The a-posteriori contingency table $H'$ in Case 4

We now turn to prove the fourth case. Here, $H'$ is as given in Table 6. So now

$$\begin{aligned} 2BER_{H'} = \frac{x_0-1+v_0}{b_0} + \frac{u_1+y_1}{b_1} \,, \end{aligned}$$

and, therefore,

$$\begin{aligned} 2(BER_{H'}-BER_H) = \frac{x_0-1-y_0}{b_0} +\frac{y_1-x_1}{b_1} \,. \end{aligned}$$

It is easy to see that the instability of $\mathbf{y}_0$ and $\mathbf{y}_1$ (i.e., the value of $f_H$ flipped on both of those vectors due to the move of a single man from the former to the latter) implies that $x_0 = \frac{b_0x_1}{b_1}+ \delta +1$, while $y_0 = \frac{b_0y_1}{b_1}+ \theta$, for some $\delta ,\theta \in [-1,0)$. Plugging this in the last equation reveals that $2(BER_{H'}-BER_H) = \frac{\eta }{b_0}$ where $\eta = \delta -\theta$ can take values in the interval $(-1,1)$. $\square$

1.4 A.4 Proof of Lemma 2

The proof of Lemma 2 is similar to that of Lemma 1 and thus omitted.

1.5 A.5 Proof of Theorem 3

The first claim in the theorem follows directly from Lemma 1 (by considering each of the four possible cases in the lemma), while the second one follows similarly from Lemma 2. To see that, observe that a vector $\mathbf{y}_0$ is stable with respect to man-removals if $s(\mathbf{y}_0) \ge 1/b_0$ and stable with respect to woman-additions if $s(\mathbf{y}_0) \ge 1/b_1$; similarly, the vector $\mathbf{y}_1$ is stable with respect to man-additions if $s(\mathbf{y}_1) < -1/b_0$ and stable with respect to woman-removals if $s(\mathbf{y}_1) < -1/b_1$. $\square$

1.6 A.6 Proof of Theorem 4

Let H ($H'$) be the a-priori (a-posteriori) contingency table corresponding to C ($C'$). Let $f_H$ and $f_{H'}$ be the corresponding MBPs. Denote by $Y_0 = f_H^{-1}(0)$ and $Y_1 = f_H^{-1}(1)$ the sets of vectors in Y that are mapped by the MBP $f_H$ to 0 and 1, respectively. We assume that $Y_0,Y_1 \ne \emptyset$, since otherwise $f_H$ would have been constant and then, by Theorem 2, already the initial BER would have equaled the maximal possible value of 1 / 2 and then C is the optimal solution (in which case $Y'=Y$).

Assume that there exists a vector $\mathbf{z}\in Y' \setminus Y$. Assume, without loss of generality, that $f_{H'}(\mathbf{z})=0$. Let $\mathbf{y}$ be any vector in $Y_0$. We proceed to prove that if we replace in $C'$ all rows that equal $\mathbf{z}$ with the row $\mathbf{y}$, we will get a matrix $C''$ for which (a) the induced BER is the same as that of $C'$, and (b) $\text{ dist }(C,C'') \le \text{ dist }(C,C')$. Since $C''$ is an optimal solution, it implies that $\text{ dist }(C,C'') = \text{ dist }(C,C')$. Therefore, $C''$ is also an optimal solution and it does not include the row $\mathbf{z}$. By repeating the same argument for all rows in $C''$ which do not exist in the original C, we will arrive at an optimal solution matrix $C^*$ in which the set of rows is included in the set of rows in C. That will conclude the proof.

We now prove claims (a) and (b) above. Denote

$$\begin{aligned} r_i = H'(i,\mathbf{y})\,,~~~ s_i = H'(i,\mathbf{z}). \end{aligned}$$

Since $C'$ is an MBP-respecting solution and $\mathbf{y}\in Y_0$, then $\frac{r_0}{b_0} \ge \frac{r_1}{b_1}$. Also, since $f_{H'}(\mathbf{z})=0$, we have $\frac{s_0}{b_0} \ge \frac{s_1}{b_1}$. The overall contribution of those two columns to $BER(\mathbf{f}_{H'},\mathbf{b})$ is $\frac{r_1+s_1}{2b_1}$. In $H''$ (the contingency table of $C''$), those two columns will be merged into $\mathbf{y}$ and it is then clear that the contribution of the merged column to $BER(\mathbf{f}_{H''},\mathbf{b})$ (where $f_{H''}$ is the corresponding MBP) would be still $\frac{r_1+s_1}{2b_1}$, while the contribution of all other columns would remain unchanged. Hence, $BER(\mathbf{f}_{H''},\mathbf{b}) = BER(\mathbf{f}_{H'},\mathbf{b})$. That proves claim (a). Next, since the recommendation vector $\mathbf{z}$ does not exist as a row in C, its contribution to $\text{ dist }(C,C')$ is $s_0+s_1$. In the transition from $C'$ to $C''$, we replace the recommendation vector of all $s_0+s_1$ users that were offered $\mathbf{z}$ to $\mathbf{y}$. Since the latter vector does exist in C, we infer that its contribution to $\text{ dist }(C,C'')$ is at most$s_0+s_1$. Therefore, $\text{ dist }(C,C'') \le \text{ dist }(C,C')$. Claim (b) is thus proved too. $\square$

1.7 A.7 Proof of Lemma 3

By Theorem 3 and its proof, we infer that: (a) for any $\mathbf{y}\in Y_0$, $\lfloor b_1 s(\mathbf{y}) \rfloor$ is the largest number of women whose recommendation vector can be changed from some vector in $Y_1$ to $\mathbf{y}$, without changing the value of $f_H$ on $\mathbf{y}$; and (b) for any $\mathbf{y}\in Y_1$, $\lceil -b_1 s(\mathbf{y}) - 1 \rceil$ is the largest number of women whose recommendation vector can be changed from $\mathbf{y}$ to some vector in $Y_0$, without changing the value of $f_H$ on $\mathbf{y}$.

Therefore, since any BER-increasing and MBP-respecting woman-move is of the form $(u,\mathbf{y},\mathbf{z})$ where u is a woman, $\mathbf{y}\in Y_1$ is her original recommendation vector and $\mathbf{z}\in Y_0$ is her new recommendation vector, the maximal number of such moves that is possible (while maintaining the MBP) is the minimum between the total number of women that the $Y_0$ vectors can “take in” and the total number of women that the $Y_1$ vectors can “loose” without affecting the MBP, as given in Eq. (5).

The proof of the corresponding claim for man-moves goes along the same lines. $\square$

1.8 A.8 Proof of Theorem 5

Each matrix in ${\mathcal {M}}(C)$ is obtained from C by a sequence of moves which are MBP-respecting. Hence, as implied by Theorem 3, each such move increases the BER of the underlying MBP either by $1/2b_0$ (if the move is of a man) or by $1/2b_1$ (woman). Moreover, the order of moves in that sequence does not matter, since none of those moves changes the MBP and, consequently, none of those moves has an effect on the utility (in terms of the increase in the BER) of moves that are made later on. Hence, we may encode any solution in ${\mathcal {M}}(C)$ by two integers: $n_0$ (the number of man-moves) and $n_1$ (the number of woman-moves). We would like to stress that the pair $(n_0,n_1)$ does not characterize the solution (since, for such a characterization, it is needed to spell out the identity of the $n_0+n_1$ affected users and their modified recommendation vectors). However, in the context of this proof, that pair is sufficient since it determines the final BER and also the resulting $\mu _1$-distance. Indeed, if $C^* \in {\mathcal {M}}(C)$ is characterized by $(n_0,n_1)$ then

$$\begin{aligned} BER^* = BER + \frac{n_0}{2b_0} + \frac{n_1}{2b_1} \end{aligned}$$

(where BER and $BER^*$ are the a-priori and a-posteriori BER values of the MBP) and

$$\begin{aligned} \text{ dist }(C,C^*) = n_0 + n_1 \,. \end{aligned}$$

(9)

Next, ${\mathcal {M}}(C,\varepsilon )$ consists of all matrices in ${\mathcal {M}}(C)$ for which

$$\begin{aligned} \frac{n_0}{2b_0} + \frac{n_1}{2b_1} \ge \varepsilon -BER \,. \end{aligned}$$

(10)

If that set is not empty, then it is clear that an optimal solution is one in which $n_1$ is maximized (if $b_0 \ge b_1$) since an optimal solution minimizes the distance in Eq. (9) while still meeting the condition in Eq. (10). The solution issued by Algorithm FaiRecSys is indeed a solution that uses the maximal possible $n_1$ woman-moves. Hence, it is an optimal solution within the class ${\mathcal {M}}(C)$. If, on the other hand, the upper limits on $n_0$ and $n_1$, as dictated by Lemma 3, do not allow meeting the condition in Eq. (10), then it is clear that the practice implemented in Algorithm FaiRecSys of exhausting first all possible woman-moves results in the highest possible a-posteriori BER, $\delta$. $\square$

1.9 A.9 Proof of Lemma 4

Under our assumption that $b_0 \ge b_1$, the only MBP-improving and respecting moves are $(u,\mathbf{y},\mathbf{z})$ where u is a man, $\mathbf{y}\in Y_0$ is his original recommendation vector, and $\mathbf{z}\in Y_1$ is an existing recommendation vector for a woman. Let $q \ge 1$ be the integer such that $b_0 = qb_1+r$, $r \in [0,b_1)$. Then, each column $\mathbf{z}\in Y_1$ can take at most q men without flipping the MBP’s value on $\mathbf{y}$. Since $|Y_1|=b_1$, we may perform up to $b_1q$ MBP-improving and respecting man-moves. As each such move yields a BER increase of $\frac{1}{2b_1}$, then after completing all of those moves, the BER will increase from 0 to $\frac{b_1q}{2b_0} = \frac{1}{2} \cdot \frac{b_0-r}{b_0} = \frac{1}{2} \cdot \frac{qb_1}{qb_1 + r}$. Since $r \le b_1-1$ and $q \ge 1$, we infer that we may increase BER to

$$\begin{aligned} \frac{1}{2} \cdot \frac{qb_1}{qb_1 + r} \ge \frac{1}{2} \cdot \frac{qb_1}{qb_1 + b_1-1} \ge \frac{1}{2} \cdot \frac{b_1}{b_1 + b_1-1} > \frac{1}{4}\,. \end{aligned}$$

$\square$