Search code examples
javascripttensorflow

Tensor flow Getting started


Ok i'm trying to get my head around TensorFlow so downloaded their Getting started example and it works but the result was nothing to write home about so figured that the perfect place to start was to improve the accuracy by increasing the number of training cases from 6 to 60

except now its incapable of resolving the value

the original code

  const xs = tf.tensor2d([-1, 0, 1, 2, 3, 4], [6, 1]);
  const ys = tf.tensor2d([-3, -1, 1, 3, 5, 7], [6, 1]);

my modified version

  const d = [];
  const r = [];
  for (let x = -50; x < 10; x++) {
    const y = x * 2 - 1;
    d.push(x);
    r.push(y);
  }
  const xs = tf.tensor(d);
  const ys = tf.tensor(r);

i figured ok i've increased the dataset by a factor of 10 so maybe the default learning time of 250 wasn't sufficient so i increases to incrementally to 5000 and no difference every time the result is NaN, if i change the for loop to match the orginal array then the code works again

now given my understanding of AI the more training data the better so surely 60 records isn't an excessive quanity so i assume the problem lies elsewhare but i have no idea what

any suggestions would be appreciated

edit the orginal script in full

async function run() {
  // Create a simple model.
  const model = tf.sequential();
  model.add(tf.layers.dense({units: 1, inputShape: [1]}));

  // Prepare the model for training: Specify the loss and the optimizer.
  model.compile({loss: 'meanSquaredError', optimizer: 'sgd'});

  // Generate some synthetic data for training. (y = 2x - 1)
  const xs = tf.tensor2d([-1, 0, 1, 2, 3, 4], [6, 1]);
  const ys = tf.tensor2d([-3, -1, 1, 3, 5, 7], [6, 1]);

  // Train the model using the data.
  await model.fit(xs, ys, {epochs: 250});

  // Use the model to do inference on a data point the model hasn't seen.
  // Should print approximately 39.
  document.getElementById('micro-out-div').innerText =
      model.predict(tf.tensor2d([20], [1, 1])).dataSync();
}

run();

Solution

  • The spread of values [-50 , 10 ] might be causing something called a gradient Explosion. Gradient Explosion leads to a NaN

    Read more here

    Might consider normalizing the values? try keeping the values between -1 to 1

    You can use tf.keras.utils.normalize and tf.keras.utils.standardize to normalize it