Bias in AI: What It Actually Is and Why It's Hard to Fix

Sofia: "AI bias" gets used to mean a lot of different things. Let's start with the technical definition.

Sofia: There are actually several distinct types of bias, and conflating them causes a lot of confusion. The clearest distinction is between representation bias and measurement bias. Representation bias is when your training data over- or under-represents certain groups — a facial recognition system trained mostly on light-skinned faces performs worse on dark-skinned faces because the training data didn't represent them proportionally. That's a data problem.

Sofia: And measurement bias is different?

Sofia: Measurement bias is when the thing you're measuring is itself a biased proxy for what you actually care about. Classic example: criminal recidivism prediction tools that use prior arrests as a feature. Prior arrests reflect who police chose to arrest, which reflects existing racial disparities in policing, not underlying criminality. The model learns and amplifies a social bias that was already in the measurement.

Sofia: Is there a way to make a model "fair"?

Sofia: This is the hardest question in AI ethics. There are mathematically incompatible definitions of fairness. You can have equal error rates across groups, or equal positive predictive values across groups — but not both, in most real-world distributions. You have to choose which definition of fairness you're optimising for, and that's an ethical and political choice, not a technical one.