Search code examples
pythonpython-3.xfor-looptemp

How to create a loop with FOR in a temporary file?


I am working with an encrypted file, but I can't manage to create a loop with for in order to read it before it get closed and removed.

My intention is to read the data given in the encrypted file and loop it to assign each line to a variable.

Whenever I execute my code, Python just goes straight to finish, without working with the decrypted info; I believe it is because the with command close it before the loop starts.

This is what I want, not working, no errors either:

with open(input_file, 'rb') as fp:
  data = fp.read()

fernet = Fernet(key)
encrypted = fernet.decrypt(data)
with tempfile.TemporaryFile() as fp:
  fp.write(encrypted)
  for url in fp: #Python ignores the tempfile. I belive it is closed in the previous line.
    segment = url.strip()
    url = 'https://docs.python.org/3.3/tutorial/' + segment
    filename = segment + '.html'
    filePath = pjoin('Data/' + filename)
    response = urlopen(url)
    webContent = response.read()
    html_content = urlopen(url).read()
    matches = re.findall(b'string', html_content);

    if len(matches) == 0: 
      print(segment + ' unchanged.')

    else:  
      with open(filePath, 'wb') as w:
       w.write(webContent)

This is the working code (Sorry, tried to make it shorter but couldn't):

with open(input_file, 'rb') as fp:
  data = fp.read()

fernet = Fernet(key)
encrypted = fernet.decrypt(data)

with open(output_file, 'wb') as fp:
    fp.write(encrypted)

with open(output_file) as fp:
    for url in fp:
      segment = url.strip()
      url = 'https://docs.python.org/3.3/tutorial/' + segment
      filename = segment + '.html'
      filePath = pjoin('Data/' + filename)
      response = urlopen(url)
      webContent = response.read()
      html_content = urlopen(url).read()
      matches = re.findall(b'string', html_content);

    if len(matches) == 0: 
      print(segment + ' unchanged.')

    else:  
      with open(filePath, 'wb') as w:
       w.write(webContent) 

Header for both examples (apart to make it shorter):

#python 3.6.6

from urllib.request import urlopen
import urllib.request
from os.path import join as pjoin
import re, os, sys, tempfile, six, ctypes, time, fileinput
from cryptography.fernet import Fernet

print("[*] Checking list.dat for consistency . . .")
key = b'wTmVBRLytAmlfkctCuEf59K0LDCXa3sGas3kPg3r4fs=' #Decrypt list.dat
input_file = 'List.dat'
output_file = 'List.txt'

List.txt content:

errors
classes
stdlib

Any hints?


Solution

  • @LarryLustig pretty much answered why your code wasn't working, but IMO if you eliminate the temp file altogether (which shouldn't be necessary) you don't even need to worry about the cursor. See below commented changes on your desired code.

    # We'll use os.linesep to get the line terminator string for your os.
    import os
    
    ...
    
    with open(input_file, 'rb') as fp:
      data = fp.read()
    
    fernet = Fernet(key)
    
    # decode your decrypted bytes into strings.  Change 'utf-8' into whichever file encoding you're using if necessary.
    decrypted = fernet.decrypt(data).decode('utf-8')
    
    # Don't write to a temp file
    # Iterate directly on each line of the extracted data
    for url in decrypted.split(os.linesep): 
        segment = url.strip()
        url = 'https://docs.python.org/3.3/tutorial/' + segment
        filename = segment + '.html'
        filePath = pjoin('Data/' + filename)
        response = urlopen(url)
        webContent = response.read()
        html_content = urlopen(url).read()
        matches = re.findall(b'string', html_content);
    
        if len(matches) == 0: 
          print(segment + ' unchanged.')
    
        else:  
          with open(filePath, 'wb') as w:
           w.write(webContent)
    

    Alternatively, if you know for sure what is the line terminator used in the file (e.g. \r\n, or \n) then you can eliminate using os.linesep altogether.