I've been manipulating huge text files these days. Sometimes I need to delete lines. My way of doing is like below:
f=open('txt','r').readlines()
list=[]
for line in f:
if blablablabla:
list.append(line)
I know for large files, .readlines()is rate-limiting step, but what about .append() step? Does append cost lots of extra time after readlines? If so, maybe I should find way to directly delete lines I don't want, instead of appending lines I want.
thx
Why read the whole entire file in using readlines() if you're going to filter it later? Just iterate through it saving the lines you want to keep. You could reduce this down to a couple of lines using list comprehension instead:
with open('txt', 'r') as f:
myList = [ line for line in f if blablablabla ]