Explaining Predictive Uncertainty in Bayesian Neural Networks with Layer-Wise Relevance Propagation
Authors
Østergaard, Lasse ; Stenkær, Mads Nørgaard ; Frydkjær, Casper Krogh
Term
4. term
Education
Publication year
2021
Submitted on
2021-06-18
Pages
73
Abstract
I dette projekt undersøger vi, hvordan man kan forbinde to vigtige mål i AI: at vurdere, hvor usikker en model er, og at forklare, hvorfor den giver en bestemt forudsigelse. Vi bruger et bayesiansk neuralt netværk (BNN), bygget i TensorFlow og TensorFlow Probability og trænet med variational inference (en metode, der approksimerer en fordeling over modellens parametre, så usikkerhed kan modelleres). Fra denne BNN sampler vi mange konkrete modeller og forklarer hver af deres forudsigelser med Layer-wise Relevance Propagation (LRP), som tildeler en relevansscore til hver inputfeature (inputvariabel). Ved at sammenligne variansen (spredningen) i disse relevansscorer på tværs af de samplede modeller kobler vi forklaring til usikkerhed: Høj varians betyder, at modellens forklaringer ændrer sig meget fra prøve til prøve, hvilket tyder på højere usikkerhed, og det fremhæver, hvilke features der driver usikkerheden. Vi tester idéen ved at sætte udvalgte features til nul. Når vi sætter features med lav relevans og lav varians til nul, forbliver forudsigelserne og deres usikkerhed stort set uændrede. Når vi gør det samme for features med høj relevans og høj varians, ændrer forudsigelserne sig i forhold til de oprindelige, ofte med lavere usikkerhed. Resultaterne viser en klar sammenhæng mellem variansen i LRP-relevansscorer og usikkerhed i forudsigelser, og at vores metode kan pege på, hvilke features der bidrager mest til usikkerheden.
This project explores how to connect two key goals in AI: estimating how uncertain a model is and explaining why it makes a prediction. We use a Bayesian neural network (BNN), built in TensorFlow and TensorFlow Probability and trained with variational inference (an approach that approximates a distribution over model parameters to model uncertainty). From this BNN we sample many concrete models and explain each of their predictions with Layer-wise Relevance Propagation (LRP), which assigns a relevance score to each input feature (input variable). By comparing the variance (spread) of these relevance scores across the sampled models, we link explanation to uncertainty: High variance means the model’s explanations change a lot from sample to sample, suggesting higher predictive uncertainty, and it highlights which features drive that uncertainty. We test this idea by setting selected features to zero. When we zero features with low relevance and low variance, the predictions and their uncertainty stay roughly the same. When we do the same for features with high relevance and high variance, the predictions change relative to the original, often with lower uncertainty. These results show a clear correlation between the variance of LRP relevance scores and predictive uncertainty, and that our method identifies which features contribute most to uncertainty.
[This summary has been rewritten with the help of AI based on the project's original abstract]
Documents