4 Cognitive Biases In AI/ML Systems

• Reporting bias refers to people’s (conscious and unconscious) tendencies to suppress the information they report.
• Example: Many amazon products have more 5-star and 1-star reviews than 2, 3, or 4-star reviews because people who have extreme experiences (either positive or negative) are more likely to post a review than those who have neutral experiences.

• The problem: An engineer that uses online reviews as their primary data source may create an AI model that is great at detecting extreme sentiments but not so much at detecting more neutral, subtle sentiments.
• The solution: Consider broadening the data collection scope to account for the underrepresented data.

• Implicit bias refers to people’s unconscious tendencies to make assumptions/associate stereotypes with others.
• Example: An engineer who is often bitten by dogs believes that dogs are more aggressive than cats, even though that may not be scientifically true.
• The problem: The engineer would believe that the ground truth is “dogs = aggressive” and thus fine-tuned her AI model to label dogs as being more aggressive than cats.
• The solution: As implicit bias is unconscious to an individual, having multiple engineers code the AI and establishing proper peer review procedures would reduce such biases

• Framing bias refers to people’s tendency to be influenced based on how information is presented.
• Example: An engineer who sees a dark and dull website for a product believes that the product must have poor sales, ignoring the actual positive sales number of the product.
• The problem: In designing the AI algorithm, the engineer may take into account subjective variables, such as the color of a website, instead of focusing on objective metrics. 
• The solution: Avoid subjective (usually qualitative) data and prioritize objective, factual data instead.

• Selection Bias (selecting data not representative of the entire population)
• Reporting Bias (people’s tendency to underreport information)
• Implicit Bias (people’s unconscious tendencies to assume)
• Framing Bias (people’s tendency to be affected by how information is presented)

1. Always select a random sample (instead of the first or last hundred data points)
2. Verify your data sources through comparison with other data sources
3. Assign more (diverse) engineers to develop the AI/ML system
4. Establish proper peer review procedures to cross-examine logic and unconscious bias
5. Prioritize objective and factual data over subjective (usually qualitative) data.

As you can see, humans’ perceptions are deeply flawed and our imperfection may trickle down into the systems we build. By acknowledging such biases, however, we can optimize the AI systems we build and make them more fair and objective.

I hope you learned something new today. If you like what you’re reading, do drop a clap or a follow!

I’ll catch you in the next article. Cheers!