Search code examples
pythonfiledesign-patternsrefactoring

How to avoid nested "with" statements when working with multiple files in Python


When working with multiple files in Python code can get ugly when using the recommended style:

with open("foo.txt") as foo:
    with open("bar.txt", "w") as bar:
         with open("baz.txt", "w") as baz:
              # Read from foo, write different output to bar an baz

That's three indentation levels just for working with files! The alternative would be this

foo = open("foo.txt")
bar = open("bar.txt", "w")
baz = open("baz.txt", "w")
# Read from foo, write different output to bar an baz
foo.close()
bar.close()
baz.close()

I have a feeling that either of these examples could be refactored to something more elegant. Any examples?


Solution

  • Python 2.7 and up let you specify multiple context managers in one with statement:

    with open("foo.txt") as foo, open("bar.txt", "w") as bar, open("baz.txt", "w") as baz:
        # Read from foo, write different output to bar an baz
    

    The line does get long, and you cannot use parentheses to keep that below 80 characters. You can use \ backslash continuations however:

    with open("foo.txt") as foo,\
            open("bar.txt", "w") as bar,\
            open("baz.txt", "w") as baz:
        # Read from foo, write different output to bar an baz
    

    Or you could place newlines inside the parentheses of the ‘open()` calls:

    with open(
        "foo.txt"
    ) as foo, open(
        "bar.txt", "w"
    ) as bar, open(
        "baz.txt", "w"
    ) as baz:
        # Read from foo, write different output to bar an baz
    

    Starting Python 3.10, this has been made easier still with addition of parentheses support for multi-item context managers:

    with (
        open("foo.txt") as foo,
        open("bar.txt", "w") as bar,
        open("baz.txt", "w") as baz,
    ):
        # Read from foo, write different output to bar an baz
    

    Another option would be to use contextlib.ExitStack() context manager (only in Python 3.3 and up):

    from contextlib import ExitStack
    
    with ExitStack() as stack:
        foo = stack.enter_context(open("foo.txt"))
        bar = stack.enter_context(open("bar.txt"))
        baz = stack.enter_context(open("baz.txt"))