Bias in AI Systems

Wissen

Bias in AI systems is not a bug you can fix with better code. It's a systemic problem that arises at multiple levels:

Types of Bias

1. Training Data Bias -- The internet overrepresents English-speaking, Western, male perspectives. A system trained on historical hiring data learns that successful engineers are predominantly male -- not because women are worse engineers, but because they were historically hired less frequently.

2. Selection Bias -- When certain population groups are underrepresented in training data, the system performs worse for those groups. Example: Facial recognition with higher error rates for dark skin tones (Buolamwini & Gebru, 2018: Gender Shades study).

3. Algorithm Bias -- The architecture and optimization objectives themselves can introduce bias. Optimizing for "engagement" potentially favors controversial or polarizing content.

4. Deployment Bias -- Even a "fair" system can be discriminatory in practice when deployed in a context it wasn't designed for.

Fairness Metrics

There is no single "fairness metric" that covers all dimensions. The choice of metric depends on context -- and different metrics can contradict each other.

Demographic Parity: The selection rate should be equal for all groups. P(Y-hat = 1 | A = a) = P(Y-hat = 1 | A = b).

Equalized Odds: The true positive rate and false positive rate should be equal for all groups. P(Y-hat = 1 | Y = y, A = a) = P(Y-hat = 1 | Y = y, A = b).

Predictive Parity: The positive predictive value (precision) should be equal for all groups. P(Y = 1 | Y-hat = 1, A = a) = P(Y = 1 | Y-hat = 1, A = b).

Verstehen

How Do You Test for Bias?

Step 1: Define Protected Attributes -- Gender, ethnicity, age, religion, disability, sexual orientation, socioeconomic status.

Step 2: Create Test Datasets -- Test cases that differ only in the protected attributes:

Prompt A: "Evaluate the creditworthiness of Thomas Miller, 35,
           software developer, $75,000 annual salary"
Prompt B: "Evaluate the creditworthiness of Fatima Al-Hassan, 35,
           software developer, $75,000 annual salary"

Step 3: Test Systematically

Automate your tests across large test sets. Individual examples aren't enough -- you need statistical significance across hundreds or thousands of test cases.

Step 4: Evaluate Results -- Statistical tests (e.g., chi-squared test, Fisher's exact test) help you determine whether differences are significant or within the range of random variation.

Anwenden

Bias Mitigation in Practice

Pre-Processing: Clean the Data

• Data augmentation -- Supplement underrepresented groups with synthetic data
• Re-sampling -- Reduce overrepresented groups or amplify underrepresented ones
• Feature removal -- Remove protected attributes (caution: proxy variables can still carry the bias)

In-Processing: Adjust Training

• Adversarial debiasing -- A second model tries to predict group membership from the predictions. The main model is penalized when it succeeds.
• Fairness constraints -- Build fairness metrics directly into the loss function

Post-Processing: Correct Output

• Threshold adjustment -- Different thresholds per group to achieve equalized odds
• Reject option classification -- Delegate uncertain predictions near the decision boundary to humans

Reflect

Bias testing is not a one-time check but a continuous process. Proxy variables make simply removing protected attributes ineffective -- you need systematic testing throughout the entire lifecycle. In the next section, we will look at a related problem: hallucinations.