Search code examples
pythonhadoopmapreducehadoop-streamingdeep-learning

Deep Learning: is there any open-source library that can be integrated with Hadoop streaming and MapReduce?

Google search popped out quite a few open source deep learning frameworks. Here is a collected list

Google TensorFlow
Theano
mxnet
keras
Pylearn2
Blocks
Lasagne
chainer
scikit-neuralnetwork
theano-lights
deepy
idlf
reinforce.js
opendeep
mxnet.js
CGT
Torch
Caffe
scikit-cuda
cuda4py

Which one can be implemented in a straight-forward way with Hadoop streaming and MapReduce?

Python is prefered, other languages can also be considered.

EDIT:

Currently I am mainly interested in Deep Reinforcement Learning and LSTM.

Solution

  • Deeplearning4j claims to be scalable on Hadoop and Spark:

    http://deeplearning4j.org/