Dealing with Data Bias in Classification: Can Generated Data Ensure Representation and Fairness?

Abstract

Fairness is a critical consideration in data analytics and knowledge discovery because biased data can perpetuate inequalities through further pipelines. In this paper, we propose a novel pre-processing method to address fairness issues in classification tasks by adding synthetic data points for more representativeness. Our approach utilizes a statistical model to generate new data points, which are evaluated for fairness using discrimination measures. These measures aim to quantify the disparities between demographic groups that may be induced by the bias in data. Our experimental results demonstrate that the proposed method effectively reduces bias for several machine learning classifiers without compromising prediction performance. Moreover, our method outperforms existing pre-processing methods on multiple datasets by Pareto-dominating them in terms of performance and fairness. Our findings suggest that our method can be a valuable tool for data analysts and knowledge discovery practitioners who seek to yield for fair, diverse, and representative data.

This work was supported by the Federal Ministry of Education and Research (BMBF) under Grand No. 16DHB4020.

A Proof of Time Complexity

Theorem 1

(Time complexity). If the number of candidates m and fraction r are fixed and calculating the discrimination $\psi (\mathcal {D})$ of any dataset $\mathcal {D}$ takes a linear amount of time, i.e., $\mathcal {O}(n)$, Algorithm 1 has a worst-case time complexity of $\mathcal {O}(n^2)$ where n is the dataset’s size when neglecting learning the data distribution.

Proof

In this proof, we will focus on analyzing the runtime complexity of the for-loop within our algorithm as the steps before such as learning the data distribution depends heavily on the used method. The final runtime of the complete algorithm is simply the sum of the runtime complexities of the for-loop that is focus of this analysis and the step of learning the data distribution.

Our algorithm firstly checks whether the discrimination of the dataset $\mathcal {\hat{D}}$ is already fair. The dataset grows at each iteration and runs for $\lfloor rn\rfloor - n = \lfloor n(r-1)\rfloor$ times. For simplicity, we use $n(r-1)$ and yield,

$$\begin{aligned} \sum _{i=0}^{n(r-1)-1} n + i&= \sum _{i=1}^{n(r-1)} n + i + 1 \\&= \sum _{i=1}^{n(r-1)} n + \sum _{i=1}^{n(r-1)} i + \sum _{i=1}^{n(r-1)} 1 \\&= n^2(r-1) + \frac{(n(r-1))^2 + (n(r-1)+1)}{2} + n(r-1) \in \mathcal {O}(n^2), \end{aligned}$$

making the first decisive step for the runtime quadratic.

The second step that affects the runtime is returning the dataset that minimizes the discrimination where each of the m candidates $c \in C$ is merged with the dataset, i.e., $\psi (\hat{\mathcal {D}} \cup \{c\})$. The worst-case time complexity of it can be expressed by

$$\begin{aligned} \sum _{i=1}^{n(r-1)} m (n + i)&= m \cdot \sum _{i=1}^{n(r-1)} n+i = m \cdot \left( \sum _{i=1}^{n(r-1)} n + \sum _{i=1}^{n(r-1)} i\right) \\&= m \cdot \left( n^2(r-1) + \frac{(n(r-1))^2 + (n(r-1))}{2}\right) \in \mathcal {O}(n^2), \end{aligned}$$

which is also quadratic. Summing both time complexities makes the overall complexity quadratic. $\square$

Although the theoretical time complexity of our algorithm is quadratic, measuring the discrimination, which is a crucial part of the algorithm, is very fast and can be assumed to be constant for smaller datasets. Conclusively, the complexity behaves nearly linearly in practice.

In our experimentation, measuring the discrimination of the Adult dataset [17], which consists of 45 222 samples, did not pose a bottleneck for our algorithm.