Just defined a new variable and now program is stuck in an infinite loop

Question

I am a beginner at programing, and I wrote a script that implements an Optimization algorithm. It worked fine at first; but then I tried to make it faster by defining a new variable, and now for some reason it appears to be stuck in an infinite loop. Here is the first version (I have indicated in a comment where the change will happen):

# This program uses the Steepest Descent Method to 
# minimize the Rosenbrock function
import numpy as np
import time

# Define the Rosenbrock Function
def f(x_k):
    x, y = x_k[0, 0], x_k[0, 1] 
    return 100 * (y - x**2)**2 + (1 - x)**2

# Gradient of f 
def gradient(x_k):
    x, y = x_k[0, 0], x_k[0, 1] 
    return  np.array([[-400*x*(y-x**2)-2*(1-x), 200*(y-x**2)]])




def main():
    start = time.time()
    # Define the starting guess
    x_k = np.array([[2, 2]])
    # Define counter for number of steps
    numSteps = 0

    # Keep iterating until both components of the gradient are less than 0.1 in absolute value
    while abs((gradient(x_k)[0, 0])) > 0.1 or abs((gradient(x_k))[0, 1]) > 0.1:
        numSteps = numSteps + 1

        # Step direction
        p_k = - gradient(x_k)
        gradTrans = - p_k.T

        # Now we use a backtracking algorithm to find a step length
        alpha = 1.0
        ratio = 0.8
        c = 0.01 # This is just a constant that is used in the algorithm

        # This loop selects an alpha which satisfies the Armijo condition  

        #####################################
        ###### CHANGE WILL HAPPEN HERE ######
        #####################################

        while f(x_k + alpha * p_k) > f(x_k) + (alpha * c * (gradTrans  @ p_k))[0, 0]:
            alpha = ratio * alpha

        x_k = x_k + alpha * p_k
    end =  time.time()
    print("The number of steps is: ", numSteps)
    print("The final step is:", x_k)
    print("The gradient is: ", gradient(x_k))
    print("The elapsed time is:", round(end - start, 2), "seconds.")



main()

Now, the program is very inefficient because in the second while loop, the quantity f(x_k) + (alpha * c * (gradTrans @ p_k))[0, 0]: is computed at each iteration, even though it is constant. So I decided to name this quantity RHS = f(x_k) + (alpha * c * (gradTrans @ p_k))[0, 0]: and just put that in the while loop. The new code is below. All I did was define this quantity as a variable, and now the program gets stuck in a infinite loop. Thank you very much for any help.

# This program uses the Steepest Descent Method to 
# minimize the Rosenbrock function
import numpy as np
import time

# Define the Rosenbrock Function
def f(x_k):
    x, y = x_k[0, 0], x_k[0, 1] 
    return 100 * (y - x**2)**2 + (1 - x)**2

# Gradient of f 
def gradient(x_k):
    x, y = x_k[0, 0], x_k[0, 1] 
    return  np.array([[-400*x*(y-x**2)-2*(1-x), 200*(y-x**2)]])




def main():
    start = time.time()
    # Define the starting guess
    x_k = np.array([[2, 2]])
    # Define counter for number of steps
    numSteps = 0

    # Keep iterating until both components of the gradient are less than 0.1 in absolute value
    while abs((gradient(x_k)[0, 0])) > 0.1 or abs((gradient(x_k))[0, 1]) > 0.1:
        numSteps = numSteps + 1

        # Step direction
        p_k = - gradient(x_k)
        gradTrans = - p_k.T

        # Now we use a backtracking algorithm to find a step length
        alpha = 1.0
        ratio = 0.8
        c = 0.01 # This is just a constant that is used in the algorithm

        # This loop selects an alpha which satisfies the Armijo condition  
        RHS = f(x_k) + (alpha * c * (gradTrans  @ p_k))[0, 0]

        #####################################
        ###### CHANGE HAS OCCURED ###########
        #####################################

        while f(x_k + alpha * p_k) > RHS:
            alpha = ratio * alpha

        x_k = x_k + alpha * p_k
    end =  time.time()
    print("The number of steps is: ", numSteps)
    print("The final step is:", x_k)
    print("The gradient is: ", gradient(x_k))
    print("The elapsed time is:", round(end - start), "seconds.")



main()

Answer 1

RHS needs to be re-calcuated inside the loop, using the new value for alpha. (Not sure how this is meant to speed things up.)