Search code examples
pythonwith-statementconceptual

Should I have most of my code in the with statement? Python


I've noticed that most tutorials showing how to use a with statement they keep most of their code in that statement. Is there a benefit to keeping it open (other than accessing the contents again) rather than pulling the contents into a list and closing it as soon as possible.

Example:

with open('file.txt') as f:
    list = f.readlines()
    # Code that does stuff with list
# end of with open

# vs

with open('file.txt') as f:
    list = f.readlines()
# end of with open
# Code that does stuff with list

Solution

  • The with statement is generally used to ensure resources are cleaned up as soon as possible. So as a general principal, keeping the with statement open for the shortest time possible is best to keep resource usage low.

    In your example, the resource is a open file, which has quite a low overhead. But some context managers (e.g. a database connection) might represent much higher resource usage and benefit from early cleanup.

    This is only a general principal though, and legitimate reasons exist to keep the with statement open longer. For example:

    Resource reuse

    If you needed to use the file later in your code, keeping it open would avoid having to re-open it later.

    More efficient resource use

    In your example, you're reading the whole file into memory as a list. If you only needed to process each line once, you might benefit from a longer with statement and an iterative approach.

    In other words, rather than this, (which loads the whole file into memory):

    with open('file.txt') as f:
        lines = f.readlines()
    for line in lines:
        print(line)
    

    Use this (which only loads one line at a time into memory):

    with open('file.txt') as f:
        for line in f:
            print(line)
    

    In the second example, the with statement is held open longer, but the memory usage should be lower.