Search code examples
pythontensorflowkerasgpumulti-gpu

Keras multi_gpu_model error: "swig/python detected a memory leak of type 'int64_t *', no destructor found"

I am using tensorflow 1.5.0, tensorflow-gpu 1.3.0, keras 2.0.9, keras-gpu 2.1.4

I created my model using this code:

inputs = Input([1,44,44,44])
x = Conv3D(64, (3,3,3), activation='relu', data_format='channels_first')(inputs)
x = Conv3D(16, (3,3,3), activation='relu', data_format='channels_first')(x)
x = Conv3D(1, (3,3,3), activation='relu', data_format='channels_first')(x)
model = Model(inputs=inputs, outputs=x)

I wanted to use two GPUs on my machine. But when I use

from keras.utils import multi_gpu_model
parallel_model = multi_gpu_model(model, gpus=2)

I get this error:

swig/python detected a memory leak of type 'int64_t *', no destructor found.
swig/python detected a memory leak of type 'int64_t *', no destructor found.
swig/python detected a memory leak of type 'int64_t *', no destructor found.

Any suggestions?

Solution

  • This is a bug in tensorflow 1.3. The fix has been ported to 1.4 (and later, of course). You have tensorflow 1.5.0, but not tensorflow-gpu - 1.3.0. When you run multi_gpu_model from keras, it's clearly using tensorflow-gpu ops.

    So you should update tensorflow-gpu to at least 1.4, the latest version would be even better.