Reading Assignment: Introspection

The blog about tf-explain gives a nice overview about different introspection techniques. (it’s not very comprehensive though) The toolbox works with TF 2.0, but currently only works on image data.

This overview blog post and a more recent addition will give you a nice overview about feature visualization.

Now it’s time to look at some specific techniques. Dig as deep as you like! These websites (and papers) provide plenty of background information.

1. Deepvis (code and paper are linked)

2. LSTMVis (code and paper are linked)

3. exBert - Visual Analysis of Transformer Models (code and paper are linked)

4. Heatmapping - especially Methods for Interpreting and Understanding Deep Neural Networks (2017), Montavon et al.

5. GradCAM Gradient-weighted Class Activation Mapping

Bonus:

• For those who are interested in the beginnings of this field of research: Deep Inside Convolutional Networks
• Linear Classifier Probes