"Preprocessing" a Python function, to avoid excess evaluation of conditional logic

Question

I am writing some Python code that includes functions with two sets of parameters. The first set of parameters will be different every single time the function is called. The second set of parameters will be the same for large groups of function calls (e.g., these change every 10000s of evaluations of the function). I am hoping to write this function in such a way that the conditional logic is run only once whenever the values of the second set of parameters change. I have included some code that I think might work, but after some testing with the time module, the improvement seems very small. Is this code actually doing what I am describing? And if not, is there some way to do this in Python?

def preprocessor(parameter_array):
    # example parameters
    condition = parameter_array[0]
    index = parameter_array[1]

    def f(x_array):
        result = 0
        if (condition):
            result += x_array[index] ** 2
        else:
            result += x_array[index] - 5
        # ...
        return result

    return f

# define some test parameters
test_parameters = [True, 1]

# run the function 100000 times with these parameters
function = preprocessor(test_parameters)

sum = 0
for i in range(100000):
    sum += function([i, i])

print(sum)

My hope is that, written in this way, the if statement is only evaluated whenever I call the preprocessor function. Thus, it speeds up the evaluation of the for loop, where only the x values change. This took around 0.024 seconds to run, while a control took around 0.028 seconds to run. I have also included the code I used for the control below (everything is without the time module for readability).

def f(x_array, parameter_array):
    # example parameters
    condition = parameter_array[0]
    index = parameter_array[1]

    result = 0
    if (condition):
        result += x_array[index] ** 2
    else:
        result += x_array[index] - 5
    # ...
    return result

# define some test parameters
test_parameters = [True, 1]

# run the function 100000 times without using the preprocessed function
sum = 0
for i in range(100000):
    sum += f([i, i], test_parameters)

print(sum)

Answer 1

Answer:

Yes, it's doing what you think it's doing. An easy way to tell is just to add some print statements.

def initialize(initialState):
    state = initialState
    print(f"I am executed on initialization: {state}")
    def f(increment): 
        nonlocal state
        print(f"I am incrementing {state} by {increment}")
        state = state + increment

    return f

fun = initialize(10)
fun(5)
fun(20)

output

I am executed on initialization: 10
I am incrementing 10 by 5
I am incrementing 15 by 20

Aside:

Your example may just be too reductive, but the logic you're extracting is extremely simple. You're not going to get that much of speed up. I'm not sure if there's extra code between the conditional and the return, but declaring the return value and then incrementing it based on the condition's result is unnecessary.

Here, I've compared your approach, with two different simplifications and both achieve a speedup. Python is truly a strange language. Every compiled language I know of is able to eliminate branching instructions when using a ternary. I would expect the ternary approach to be the fastest.

import timeit

def approach1(): 
    def preprocessor (parameter_array):
        condition = parameter_array[0]
        index = parameter_array[1]
        return lambda x_array: x_array[index] ** 2 if condition else x_array[index] - 5
    test_parameters = [True, 1]
    function = preprocessor(test_parameters)
    sum = 0
    for i in range(100000):
        sum += function([i, i])
    
def approach2():
    def preprocessor (parameter_array):
        condition = parameter_array[0]
        index = parameter_array[1]
        def f (x_array):
            result = 0
            if (condition):
                result += x_array[index] ** 2
            else:
                result += x_array[index] - 5
            return result
        return f
    
    test_parameters = [True, 1]
    function = preprocessor(test_parameters)
    sum = 0
    for i in range(100000):
        sum += function([i, i])

def approach3():
    def preprocessor (parameter_array):
        condition = parameter_array[0]
        index = parameter_array[1]
        def f (x_array):
            if (condition):
                return x_array[index] ** 2
            return x_array[index] - 5
        return f
    test_parameters = [True, 1]
    function = preprocessor(test_parameters)
    sum = 0
    for i in range(100000):
        sum += function([i, i])

n = 100
print(timeit.timeit(approach1, number=n))
print(timeit.timeit(approach2, number=n))
print(timeit.timeit(approach3, number=n))

output

0.8117037120027817
0.9467067930017947
0.8014805819984758

P.S.

While these types of optimizations can be fun and help you learn about performance characteristics of the language, your time is much better spent on optimizing things like the algorithms you're using. If you really need performance, Python is not the language for you.