Python nested loop doesn't loop

Question

I'm a python beginner and I can't make my loop loop.

Here's the case : I've got two csv files (event logs).

First on is called bd8result.csv with 4 or 5 lines structured like these :

2015/10/30 09:53:44,blabla1,259373865,95,F,A1 IP Thers,A1SIP V1 (R),-,,1446195224
2015/10/30 11:03:14,blabla2,259431070,32,F,A7 IP MornOs,A7SIP V1 (R),-,,1446199394
2015/10/30 21:30:59,blabla3,259980991,86,F,A2 IP Hor4ain,A2IP V1 (R),-,,1446237059

First column is the date, second is the IP event (target), last on is the epoch time.

I need to find blabla1, blabla2 and blabla3 and events associated 20mn before and 20 minute after these events, in a bigger log file that has the same structure and write the result in a csv file. I just collect events located on the same lecteur info (so comes the test in if...).

My code looks like this :

with open('result_'+ namefile + '.csv', 'rb') as master1, open('epoch_'+ namefile + '.csv', 'rb') as hosts1:
    reader2 = csv.reader(master1)
    reader1 = csv.reader(hosts1)
    for row in reader2:
        target = row[1]
        lecteur = row[5]
        epoch_ref = int(row[-1])
        for row2 in reader1:
            epoch1 = int(row2[-1])
            lecteur1 = row2[5]
            with open('result_scout' + namefile + '.csv', 'a') as results1:
                 if epoch1 > (epoch_ref - ecart) and epoch1 < (epoch_ref + ecart) and lecteur1 == lecteur:
                    writer1 = csv.writer(results1)
                    writer1.writerow([target]+[sys.argv[1]]+row2)
                    results1.close()

My problem is that it executes correctly the first item (blabla1), but doesn' write anything for blabla2 and blabla3.

I've tried several thing but I'm stucked.

Any help appreciated. Thank you!

Answer 1

After one iteration of looping over reader2, reader1 is exhausted by its for loop and raises StopIteration and won't return anything in the successive loops.

You should get a new instance of csv.reader in every iteration:

with open('result_'+ namefile + '.csv', 'rb') as master1:
    for row in csv.reader(master1):
        target = row[1]
        lecteur = row[5]
        epoch_ref = int(row[-1])
        with open('epoch_'+ namefile + '.csv', 'rb') as hosts1:
            for row2 in csv.reader(hosts1):
                epoch1 = int(row2[-1])
                lecteur1 = row2[5]
                with open('result_scout' + namefile + '.csv', 'a') as results1:
                     if epoch1 > (epoch_ref - ecart) and epoch1 < (epoch_ref + ecart) and lecteur1 == lecteur:
                        writer1 = csv.writer(results1)
                        writer1.writerow([target]+[sys.argv[1]]+row2)
                        results1.close()

From the documentation:

csv.reader(csvfile)

Return a reader object which will iterate over lines in the given csvfile. csvfile can be any object which supports the iterator protocol and returns a string each time its next() method is called.

This implies that after the reader raises StopIteration it will be exhausted just like an exhausted generator.