As a CNTK learning exercise, I figured I would modify the Logistic Regression example from lr_bs.cntk, and try to get a basic linear regression working.
Instead of this in the logistic example:
# parameters to learn
b = Parameter (LDim, 1) # bias
w = Parameter (LDim, SDim) # weights
# operations
p = Sigmoid (w * features + b)
lr = Logistic (labels, p)
err = SquareError (labels, p)
# root nodes
featureNodes = (features)
labelNodes = (labels)
criterionNodes = (lr)
evaluationNodes = (err)
outputNodes = (p)
... I simply changed the code to this:
# operations
p = (w * features + b)
lr = SquareError (labels, p)
err = SquareError (labels, p)
I got this to work on a synthetic dataset I created. However, I tried then to run it on files I created off the Wine Quality dataset. I can't get it to work, and I am at a loss on how to move forward.
The Train command fails, with the following diagnosis:
EXCEPTION occurred: The training criterion is not a number (NAN).
I interpret this to mean that lr is not producing a valid number. I just don't understand how SquareError could fail, and how to approach fixing the issue.
For information, here is how the dataset, after preparation, looks like:
|features 7.400 0.700 0.000 1.900 |labels 5.000
|features 7.800 0.880 0.000 2.600 |labels 5.000
|features 7.800 0.760 0.040 2.300 |labels 5.000
|features 11.200 0.280 0.560 1.900 |labels 6.000
|features 7.400 0.700 0.000 1.900 |labels 5.000
I cannot see any blatantly problematic data problem. I use the CNTKTextFormatReader to read the data, perhaps the problem is with the data reading part, but without debugging I can't be sure.
Any advice on how to approach this would be really appreciated.
I had a very similar idea for getting started, except that I modified the Python tutorial for logistic regression in order to create a linear regression example.
I found that the learning rate specified in the logistic example is far too big to use with the squared error loss function required for linear regression purposes. So as a first suggestion, I would suggest you try decreasing learningRatesPerSample to something like 0.001 or smaller.
I did a quick google search of the error code you are saw and that returned this issue, which also suggests learning rate might be your culprit.
If you are interested I wrote a blog post about my linear regression example in Python.