Interpreting Machine Learning Models

• Global Interpretability: How well can we understand the relationship between each feature and the predicted value at a global level — for our entire observation set. Can we understand both the magnitude and direction of the impact of each feature on the predicted value?
• Local Interpretability: How well can we understand the relationship between each feature and the predicted value at a local level — for a specific observation.
• Feature Selection: Does the model help us focus on only the features that matter? Can it zero out the features that are just “noise”?

An ordinary least squares (OLS) model generates coefficients for each feature. These coefficients are signed, allowing us to describe both the magnitude and direction of each feature at the global level. For local interpretability, we need only multiply the coefficient vector by a specific feature vector to see the predicted value, and the contribution of each feature to that prediction.

In the middle of the accuracy-interpretability spectrum are random forests. We’ve often seen them described as “black boxes,” which we think this is unfair — maybe “gray” but certainly not “black”!

Random forests are collections of decision trees, like the one drawn below. The splits in each tree are chosen from random subsets of our features, so the trees all look slightly different. A single tree can be easily interpreted, assuming it is not grown too deep. But how we can interpret a random forest that contains hundreds or thousands of trees?

As the hottest topic in machine learning over the past decade, we’d be remiss if we didn’t mention neural networks. Hailed for outstanding accuracy in difficult domains like image recognition and language translation, they’ve also generated criticism for lacking interpretability.

Nobody understands how these systems — neural networks modelled on the human brain — produce their results. Computer scientists “train” each one by feeding it data, and it gradually learns. But once a neural net is working well, it’s a black box.