I'm putting together a 3D CNN to classify 3D images of fruit flies as cancerous or non-cancerous. The model is just a simple couple of blocks of 2 convolutions, max pooling, and batch normalization followed byou a couple FC layers and then a prediction layer. However I cannot seem to get it to run getting a InvalidArgumentError: Graph execution error:.
So the model is defined as below:
inputs = keras.Input(shape = InSize + (1,))
x = keras.layers.Conv3D(filters = 64, kernel_size = 3, activation = 'relu', name = '3DConv1')(inputs)
x = keras.layers.Conv3D(filters = 64, kernel_size = 3, activation = 'relu', name = '3DConv2')(x)
x = keras.layers.MaxPool3D(pool_size = 2, name = 'MaxPool1')(x)
x = keras.layers.BatchNormalization(name = 'BatchNorm1')(x)
# x = keras.layers.Conv3D(filters = 64, kernel_size = 3, activation = 'relu', name = '3DConv3')(x)
# x = keras.layers.Conv3D(filters = 64, kernel_size = 3, activation = 'relu', name = '3DConv5')(x)
# x = keras.layers.MaxPool3D(pool_size = 2, name = 'MaxPool2')(x)
# x = keras.layers.BatchNormalization(name = 'BatchNorm2')(x)
x = keras.layers.GlobalAveragePooling3D(name = 'GlobalNorm1')(x)
# x = keras.layers.Flatten()(x)
x = keras.layers.Dense(units = 50, activation = 'relu', name = 'FC1')(x)
x = keras.layers.Dense(units = 50, activation = 'relu', name = 'FC2')(x)
x = keras.layers.Dropout(0.3, name = 'Dropout1')(x)
outputs = keras.layers.Dense(units = 1, activation = 'sigmoid', name = 'Classifier')(x)
model = keras.Model(inputs, outputs)
model.summary()
model.compile(loss = loss,
optimizer = keras.optimizers.legacy.Adam(learning_rate = LR),
metrics = metrics)
which produces the model:
Model: "3dcnn"
Layer (type) Output Shape Param #
input_1 (InputLayer) [(None, 128, 128, 64, 1)] 0
conv3d (Conv3D) (None, 126, 126, 62, 64) 1792
max_pooling3d (MaxPooling3D) (None, 63, 63, 31, 64) 0
batch_normalization (BatchNo (None, 63, 63, 31, 64) 256
conv3d_1 (Conv3D) (None, 61, 61, 29, 64) 110656
max_pooling3d_1 (MaxPooling3 (None, 30, 30, 14, 64) 0
batch_normalization_1 (Batch (None, 30, 30, 14, 64) 256
conv3d_2 (Conv3D) (None, 28, 28, 12, 128) 221312
max_pooling3d_2 (MaxPooling3 (None, 14, 14, 6, 128) 0
batch_normalization_2 (Batch (None, 14, 14, 6, 128) 512
conv3d_3 (Conv3D) (None, 12, 12, 4, 256) 884992
max_pooling3d_3 (MaxPooling3 (None, 6, 6, 2, 256) 0
batch_normalization_3 (Batch (None, 6, 6, 2, 256) 1024
global_average_pooling3d (Gl (None, 256) 0
dense (Dense) (None, 512) 131584
dropout (Dropout) (None, 512) 0
dense_1 (Dense) (None, 1) 513
Total params: 1,352,897 Trainable params: 1,351,873 Non-trainable params: 1,024
I have 30 input images in a nparray of shape (30, 27, 54, 182). The elements of the images is intensity data (max of 10965). The labels are an nparray of shape (30,) with uint8 elements.
Tto execute the code I use:
history = model.fit(
xTrain,
yTrain,
batch_size = batch_size,
epochs = epochs,
verbose = verbose,
shuffle = shuffle,
validation_data = (xVal,yVal),
class_weight = class_weight
)
and get
InvalidArgumentError: Graph execution error:
2 root error(s) found. (0) INVALID_ARGUMENT: indices[0] = 1 is not in [0, 1) [[{{node GatherV2}}]] [[IteratorGetNext]] [[IteratorGetNext/_2]] (1) INVALID_ARGUMENT: indices[0] = 1 is not in [0, 1) [[{{node GatherV2}}]] [[IteratorGetNext]] 0 successful operations. 0 derived errors ignored. [Op:__inference_train_function_21420]
as the error. Can anyone help point me in the right direction to solve this?
Thank you!
Edit: Issue has been solved!
For anyone who finds this in the future the issue was that I had accidently defined class_weights = {0: 0.5, 0: 0.5} instead of {0:0.5, 1:0.5}. If you're having the same problem double check your class_weight dictionary!
Do you know what the following means?
1 is not in [0, 1)
It's mathematical notation for "all numbers greater than or equal to 0 and less than 1." This means your model is not properly calibrated for a classification problem that is supposed to output 1 or 0 because it cannot accept the value 1.
Here are some potential sources of error:
Your input tensor expects a shape of (None, 128, 128, 64, 1), but your actual data is (30, 27, 54, 182). You'll need to address this discrepancy.
If using class_weight during training, ensure the keys match the labels in your dataset. Given that your output layer suggests binary classification, the class_weight dictionary should only have keys 0 and/or 1.
Example of a correct format:
class_weight = {0: weight_for_class_0, 1: weight_for_class_1}
Ensure no additional keys are mistakenly provided.
Your labels should be of shape (30, 1) instead of (30,) for compatibility. Also, ensure the labels are either 0 or 1.