Python, How to make a nested loop works in a function?

Question

I have a working function using loop, that works fine, in the example code I have, there are 4 CSV files, but it could go to 40 or more CSV files.

• this codes reads from first CSV file (file1.csv), loop from a range( in this example starts from row 0 to 3),
• use this data from the rows, (one at the time) to run the function,
• then it goes to the second CSV file and uses the data from same range(0-3), until the last CSV file.

I want to run this Python file on schedule (once,twice, or more daily), so I want to add another loop that when next time Python file runs, it reads CSV file from where it was left off. (row 3 to 6 in this example) for all CSV files, again the next time it runs, it starts from where it was left off ( row 6 t 9 in this example) and so on for all CSV files., of course this range is an example,

I am not sure if this is doable or not, I can not figure it out how to do that, appreciate your help.

here is the code I have:

from abc.zzz  import xyz
path_id_map = [
    {'path':'file1.csv', 'id': '12345678'},
    {'path':'file2.csv', 'id': '44556677'}
    {'path':'file3.csv', 'id': '33377799'}
    {'path':'file4.csv', 'id': '66221144'}]
s_id = None
for pair in path_id_map:
    with open(pair['path'], 'r') as f:
        for i in range(0, 3):      
            zzz.func1(id_1=f.readline().rstrip(), B_id=pair['id'], s_id=s_id)
            time.sleep(25)

Answer 1

• If you want to remember the exeution state of a program, you need to store any of these variables somewhere. In the example below I've used a plain and simple text file, but it could be a database or any other form of persistent storage.

• To help make things easier to change and adapt, you should break to program down into small functions that accomplish one task.

def read_range(pair, start_line, end_line):
    with open(pair['path'], 'r') as f:
        line_counter = 0
        for line in f:            
            if line_counter >= start_line and line_counter < end_line:
                zzz.func1(id_1=f.readline().rstrip(), B_id=pair['id'], s_id=s_id)
                time.sleep(25)
            elif line_counter >= end_line:
                break
            line_counter = line_counter + 1

def read_start_number():
    try:
        lines = open("save.txt", 'r').readlines()
    except FileNotFoundError:
        write_start_number(0)
        lines = read_start_number()
    return lines[0]
    

def write_start_number(line_number):
    with open('save.txt', 'w') as f:
        f.write(str(line_number))
        
path_id_map = [
    {'path':'file1.csv', 'id': '12345678'},
    {'path':'file2.csv', 'id': '44556677'}
    {'path':'file3.csv', 'id': '33377799'}
    {'path':'file4.csv', 'id': '66221144'}]
s_id = None

start_number = int(read_start_number())
print(start_number)
for pair in path_id_map:
    read_range(pair, start_number, start_number + 3)
write_start_number(start_number+3)