Five critical questions to explain Explainable AI

The audience for the explanation or whom to explain should be the first question to answer. Understanding the motivation of the audience, what action or decision the audience is planning to make, their mathematical or technical knowledge and expertise are all important aspects that should be considered during the formulation of the explanation. Based on our experience we propose four main categories of audience:

1. End users
2. Business sponsors
3. Data Scientists
4. Regulators

This list of four types is by no means exhaustive, but it does capture some of the key differences between different groups.

End users require an explanation of the decision or action recommended made by the AI system in order to carry out the recommendation.

Business users require an explanation to ensure corporate governance and manage reputational risk to their group or company.

Data scientists require explanations to validate models and also perform trade-offs between accuracy of the model and performance criteria.

Regulators require explanations to ensure compliance to existing regulations and ensuring that no harm comes to consumers.

Explanations may be generated before the model is built, also called ex-ante or the model is trained and tested first and then the explanation may be generated, also called post-hoc.

Visual or graphical explanations, tabular data-driven explanation, natural language descriptions or voice explanations are some of the existing modes of explanation.

A salesperson might be comfortable with an explanation that shows a graph of increasing sales and how the increase in sales is achieved.

The instructions for a construction worker and the explanations for why those instructions are being given may be better provided through a voice interface as opposed to a detailed written explanation.

There are six broad approaches as it relates to post-hoc explainability.

• Feature relevance: These approaches to explainability focus on the inner functioning of the model and highlight the features that best explain the outcome of the model.
• Model simplification: These approaches focus on building a new model that is a simplification of a more complex model that is to be explained.
• Local explanations: These approaches segment the solutions space and provide explanations for smaller segments that are less complex.
• Explanations by example
• Visualization
• Text explanations

There are six broad approaches as it relates to post-hoc explainability.

• Feature relevance: These approaches to explainability focus on the inner functioning of the model and highlight the features that best explain the outcome of the model.
• Model simplification: These approaches focus on building a new model that is a simplification of a more complex model that is to be explained.
• Local explanations: These approaches segment the solutions space and provide explanations for smaller segments that are less complex.
• Explanations by example
• Visualization
• Text explanations