I tried to use shap in order to do a feature importance analysis. I am using keras and I want to get a bar chart and violin charts. With my DNN, I got something like that: Violin chart bar chart
However, when I tried it with my SimpleRNN, I had problem with the shape. The input shape is (samples,time,features), whereas my output shape is (samples,features). So it is a many-to-one RNN. KernelExplainer, the one that I used in my static models, does not work because of the dimension. DeepExplainer does not work either. It show me this error:
Your TensorFlow version is newer than 2.4.0 and so graph support has been removed in eager mode. See PR #1483 for discussion.
Traceback (most recent call last):
File "..."
... = model.predict(data)
File "...", line 103, in predict
explainer = shap.DeepExplainer(self.model, self.background)
File ".../lib/python3.6/site-packages/shap/explainers/_deep/__init__.py", line 84, in __init__
self.explainer = TFDeep(model, data, session, learning_phase_flags)
File ".../lib/python3.6/site-packages/shap/explainers/_deep/deep_tf.py", line 131, in __init__
self.graph = _get_graph(self)
File ".../lib/python3.6/site-packages/shap/explainers/tf_utils.py", line 46, in _get_graph
return explainer.model_output.graph
AttributeError: 'KerasTensor' object has no attribute 'graph'
Do you know what this error could be due to? Here I have seen people with similar problems with shap and LSTM, but there is no a clear solution. Do you recommend an alternative method to shap to get those charts?
my tensorflow version is 2.4.1 and my shap version is 0.38.1
Thanks in advance
I managed to fix it by adding at the beginning of my code
import tensorflow as tf
tf.compat.v1.disable_v2_behavior()
Shap does not work with 2.4.1 tensorflow version