Division by neural network; fixed value produced each time

Question

I'm trying to develop a neural network to multiply and divide two numbers using this method. The multiplication part goes well, but the division doesn't - each time the program is run, for each pair of inputs, a small near-fixed value(fixed each time the program is run) is obtained as output. With the help of an answer on a previous question I asked (I'd recommend not going back and reading all of it, to avoid confusion), I made some progress, but have still run into problems, and thought it best to ask a new question. The code for my neural network is:

import tensorflow as tf
import numpy as np
from tensorflow.keras.models import Sequential
from tensorflow.keras.layers import Dense
from tensorflow.keras import regularizers

num_train = 1000

X_train = np.random.rand(num_train, 2)
y_train_add = X_train[:, 0] + X_train[:, 1]

model_add = Sequential(
        [
            Dense(10),
            Dense(1)
            ]
        )

batch_size = 32
epochs = 100

model_add.compile(loss = 'mse', optimizer='adam')
model_add.fit(X_train, y_train_add, batch_size=batch_size, epochs=epochs, verbose = 1)

a = np.random.random((2000000,1))*100-50
b = np.random.random((100000,1))*10
x_train = np.append(a, b)
#This is because previously there had been severe inaccuracies near 0, which led to problems during division, so I increased the training data near zero
y_train = np.square(x_train)

model_sqr = Sequential(
        [
            Dense(8, activation = 'elu', kernel_regularizer = regularizers.l2(0.001), input_shape = (1,)),
            Dense(8, activation = 'elu', kernel_regularizer = regularizers.l2(0.001)),
            Dense(1)
            ]

        )

batch_size = 32
epochs = 100

model_sqr.compile(loss = 'mse', optimizer='adam')
model_sqr.fit(x_train, y_train, batch_size=batch_size, epochs=epochs, verbose = 1)

x = "n" 
while True:
    print("enter first num:")
    x = input()
    if x == "end":
        break

    print("Enter operation:")
    op= input()

    print("enter second num:")
    y = input()

    X = int(x)
    Y = int(y)
    Ydiv = np.reciprocal(Y)

    if op == "*":
        predicted_product = model_sqr.predict(model_add.predict(np.array([[X, Y]]))) - model_sqr.predict(np.array([X])) -  model_sqr.predict(np.array([Y]))
        print(predicted_product/2)
    elif op =="/":
        predicted_quot = model_sqr.predict(model_add.predict(np.array([[X, Ydiv]]))) - model_sqr.predict(np.array([X])) -  model_sqr.predict(np.array([Ydiv]))
        print(predicted_quot/2)

For example, on saving the weights and running the program, for each pair of numbers used, approximately -0.123 is obtained each time I run it.

In an effort to understand why this was going on, I ran 25/5, for example, with the program modified to show (25+(1/5))^2, 25^2, and (1/5)^2 too. I got the values 623.7364, 623.73645, and 0.24594116.

To understand why these values were obtained and how to fix them, I plotted the square function(orange) and my neural network model for square(blue) together:

Zoomed in close to 25;

Zoomed in close to 0;

The problem is, the intermediate values that I see were predicted from the plots aren't the ones I printed out. Using the cursor, I could see, for example, that 25^2 was predicted to be approximately 624, (25+(1/5))^2 was predicted to be approximately 634 and (1/5)^2 was predicted to be 0.42. Using those values, you would get an acceptable answer, but the values I printed out are different.

(1): Why are these values different?

(2): How can I make the model make the acceptable predictions the plot seems to make?

Answer 1

I think the problem is that when you do the reciprocal of the integer value Y, it returns 0, so you are always calculating the product of X by zero.

Y = int(y)
Ydiv = np.reciprocal(Y) # Ydiv is zero because the reciprocal of a positive integer is zero

I think this is the main reason your program always returns the same result.

To overcome this situation you could e.g.:

X = int(x)
Y = int(y)
Ydiv = np.reciprocal(Y * 1.0)
print("Ydiv")
print(Ydiv)
print(np.array([X]))
print(np.array([Ydiv]))

if op == "*":
    add_predict = model_add.predict(np.array([[X, Y]]))
    add = float(add_predict[0][0]);
    msqr = model_sqr.predict(np.array([add]));

    xsqr = model_sqr.predict(np.array([X]));

    ysqr = model_sqr.predict(np.array([Y]));
    predicted_product = msqr - xsqr - ysqr
    result = predicted_product[0][0] / 2.0
    print(f'result of {X} * {Y}: {result}')
elif op =="/":
    add_predict = model_add.predict(np.array([[X, Ydiv]]))
    print("x + 1/y = ")
    print(add_predict)

    add = float(add_predict[0][0]);
    msqr = model_sqr.predict(np.array([add]));

    xsqr = model_sqr.predict(np.array([X]));

    ysqr = model_sqr.predict(np.array([Ydiv]));
    
    #predicted_quot = model_sqr.predict(model_add.predict(np.array([[X, Ydiv]]))) - model_sqr.predict(np.array([X])) -  model_sqr.predict(np.array([Ydiv]))
    predicted_quot = msqr - xsqr - ysqr
    print("result:")
    print(predicted_quot/2)
    print("---------")
    print("x + 1/y = ")
    print(add_predict)
    print(Ydiv)
    print("---------")
    print("(x + 1/y)^2 = ")
    print(msqr)
    print("---------")
    print("(x)^2 = ")
    print(xsqr)
    print("---------")
    print("(y)^2 = ")
    print(ysqr)
    print("---------")

I executed a 1000-loops run of your program for an x/y division giving results between 0 and 50, and the error between expected result and predicted one seems good, just there is an offset of 0.5 to compensate and some divergence for values close to zero