Search code examples
pythonarraysnumpyrandombackpropagation

Why do we multiply calls to numpy.random.random by numbers and subtract numbers?


I found this snippet of code in an article about backpropagation and I got confused on how exactly it works. The article says the following

"This is our weight matrix for this neural network. It's called "syn0" to imply "synapse zero". Since we only have 2 layers (input and output), we only need one matrix of weights to connect them. Its dimension is (3,1) because we have 3 inputs and 1 output."

I want to specify that by input the author is referring to array "X" and output refers to array "Y".

My first question is why does the article claim we have only 3 inputs. A glance at the code reveals that our array, X, has size 4. Am I just misunderstanding something?

My second question is why are multiplying the call to np.random.random() by 2?

Thanks for the help!

import numpy as np    

X = np.array([ [0,0,1],
               [0,1,1],
               [1,0,1],
               [1,1,1] ])

y = np.array([[0,0,1,1]]).T  

# initialize weights randomly with mean 0
syn0 = 2*np.random.random((3,1)) - 1

Solution

  • The shape (3, 1) gives you a 3 by 1 matrix; 1 column in each of 3 rows. That’s only 3 values.

    The multiplication and subtraction takes the range of values that numpy.random.random() produces to produce a broader range.

    random() always produces floating point values between 0 and 1, and multiplying those by 2, then subtracting 1 means you now have values between -1 and 1, instead.

    See the numpy.random.random() documentation:

    Return random floats in the half-open interval [0.0, 1.0).

    Results are from the “continuous uniform” distribution over the stated interval. To sample Unif[a, b), b > a multiply the output of random_sample by (b-a) and add a:

    (b - a) * random_sample() + a
    

    (Note that np.random.random() is an alias of np.random.random_sample())

    So to produce values between a = -1 and b = 1 you need to multiply by b - a = 2 and then subtract 1 (+ -1).

    You can try running the code in an interactive session:

    >>> import numpy as np
    >>> np.random.random((3, 1))  # a 3 x 1 matrix of values between 0 and 1
    array([[0.11605033],
           [0.31756365],
           [0.4690499 ]])
    >>> 2 * np.random.random((3, 1))  # a 3 x 1 matrix of values between 0 and 2
    array([[1.30127808],
           [0.3982432 ],
           [1.96544242]])
    >>> 2 * np.random.random((3, 1)) - 1  # a 3 x 1 matrix of values between -1 and 1
    array([[ 0.39767412],
           [-0.83410998],
           [-0.62446309]])