Search code examples
pythonkeraspytorchlstm

Keras LSTM is extremely slow in comparison to PyTorch LSTM

I am using CPU for small LSTM model that needs to be realtime. I encountered an issue with Keras LSTM, the minimal model takes about 70ms to make ONE prediction. Same model made with PyTorch takes about 0.5ms to make prediction.

It's 130 times difference. Is it a bug or am I doing smth wrong?

I've created a really simple repo to recreate this issue: https://github.com/Al-Garifov/lstm_issue_demo/tree/master

I've tried "unroll" option in Keras LSTM = no difference.

Solution

  • Answer is found. In Keras you should avoid using loops, instead pass whole dataset in operator(). In PyTorch you should use tensors for speedup, not numpy.

    Btw, PyTorch is still faster than Keras, but only 8 times faster with new fixes.

    All the above is demonstated in updated repository, feel free to check out.