Quick question :
Is RMSProp optimizer compatible with online (stochastic, update weights every turn) learning ? All I can read of is about RMSProp being used with mini-batch or full-batch update, but none seems to explicitely state that online stochastic learning would be out of question.
Very short answer: it is. You can use it with SGD. Example: http://www.erogol.com/comparison-sgd-vs-momentum-vs-rmsprop-vs-momentumrmsprop/