AI bias lurks beneath the surface of our increasingly automated world. As algorithms continue to make decisions that impact human lives, the potential for discrimination grows and grows. It’s important to understand why this is happening and how companies (and individuals) can continue efforts to mitigate.
Otherwise we risk a world where AI’s negative effects on society hit on a deeper level that is less obvious — but more harmful.
Why it matters
A 2022 DataRobot survey found 56% of technology leaders fear AI bias will erode customer trust. This apprehension stems from high-profile cases of algorithmic discrimination.
In 2015, Google's image recognition system labeled photos of Black individuals as "gorillas," exposing racial bias in its training data In 2018, Amazon scrapped an AI hiring system that penalized resumes containing the word "women's" or mentioning all-women's colleges. The system, trained on historical data from a male-dominated tech industry, learned to discriminate against female applicants. Elsewhere, a widely-used system for predicting patient risk has systematically underestimated the health needs of Black patients compared to equally sick white patients. This bias arose from using healthcare costs as a proxy for health needs, failing to account for disparities in healthcare access.
These incidents fuel public skepticism about AI fairness. A 2019 Pew Research survey found 58% of Americans believe computer programs will always reflect human biases. This perception threatens to undermine trust in AI systems across sectors.
Experts warn algorithmic bias could exacerbate existing inequalities. AI researcher Kate Crawford cautions, "The reproduction of harmful ideas is particularly dangerous now that AI has moved from being an experimental discipline used only in laboratories to being tested at scale on millions of people". As algorithms make high-stakes decisions in areas like healthcare, criminal justice, and employment, the societal cost of biased systems grows. Addressing algorithmic fairness has become essential for responsible AI development and deployment.
What research says
Academic research provides valuable insights into the nature, causes, and impacts of AI bias. Recent studies have helped demystify how bias can muddy AI systems and cause irreparable damage.
Causes of bias
Research identifies several key sources of algorithmic bias:
- Biased training data. AI systems learn from historical data, inheriting and amplifying existing societal biases. One study found commercial facial recognition systems had much high errors rates for darker-skinned females than lighter-skinned males due to unrepresentative training data.
- Algorithmic design choices. The selection of features, model architectures, and optimization criteria can introduce bias. Researchers at Columbia University developed techniques to detect vulnerabilities in neural networks that could lead to biased outputs.
- Human bias in development. The personal biases of AI developers can seep into system design. A lack of diversity in tech workforces exacerbates this issue, according to one study.