What is Simpson’s Paradox?

Simpson's paradox occurs when there are variables that split data into multiple separate distributions. This hidden variable is referred to as the Lurking variable.

With Simpson's paradox at hand, decision-making can be hard. But understanding and identifying this paradox is important for interpreting the data clearly.

Let us assume that we are observing several groups of data and we establish a relationship or correlation between them.

According to Simpson's Paradox, when we combine all the groups together and look at the aggregate form of the data, the correlation that we noticed may reverse itself!

To avoid Simpson's Paradox, which leads us to two different interpretations of the same data,

we need to analyse the data either by splitting it into groups or by aggregating it together.

One of the most famous examples of Simpson's paradox is, admission to graduate school at US Berkeley in 1973. Due to this turmoil UC Berkeley Almost Got Sued For Sex Discrimination! 

When we look at the overall Admissions, men are most likely to get accepted than women

However, when overall admissions are divided into department-wise admissions, women are more likely to be admitted than men.

Out of the 6 departments 4 of the departments accepted women more than men.