Could someone recommend me lightweight, easy to use Java library for stochastic gradient descent optimization?
This may be a late reply. Try MLLib by apache (part of SPARK project) or WEKA. Only to optimize a linear regression, you might like to code yourself rather than going for a library.