Explaining Predictive Uncertainty in Bayesian Neural Networks with Layer-Wise Relevance Propagation

Østergaard, Lasse ; Stenkær, Mads Nørgaard ; Frydkjær, Casper Krogh

4. term

Software, Master

2021

2021-06-18

73 pages

In this project, we investigate the use of BNNs for bridging the gap between expressing uncertainty in AI models and explaining the predictions of the models. The BNN used in this project is build with TensorFlow and TensorFlow Probability, and is trained through Variational Inference. Here, we propose the possibility of sampling models from a BNN and explaining these with LRP. By using the relevance scores calculated with LRP from multiple sampled models, we can consider the variance in relevance scores for each individual feature. We argue that this variance reflects the uncertainty in the predictions, and gives an insight into which features affect the uncertainty the most. The LRP approach is evaluated by setting the values of features to $0$, where we find that when setting features to $0$, for features with low relevance scores and low variance, the predictions and uncertainty in these remain similar. On the other hand, when doing the same for features with high relevance scores and high variance in these, the predictions differ from the original predictions, often with a lower uncertainty. We conclude that there is a clear correlation between the variance in relevance scores and the uncertainty in predictions, and that our method is able to give an insight into which features contribute most to the uncertainty in predictions.

Download
View record in AAU Student Projects