Search code examples
pythonhttptcppython-requests

How to properly close an http connection from python requests


I saw the answers in Closing python requests connection but I don't think the answers really specify how to close the connection, or maybe I am not really understanding what it does. For example, the following code:

import requests

with requests.Session() as s:
    s.post("https://example.com",  headers={'Connection':'close'})
    print(s)

# My understanding is that the Session is still alive as per the outputs
print(s)
s.close()
print(s)

returns

<requests.sessions.Session object at 0x106264e80>
<requests.sessions.Session object at 0x106264e80>
<requests.sessions.Session object at 0x106264e80>

I was expecting only the first print statement to work, since the second is already outside the with statement (hence the connection should have been closed, according to the documentation) and the third is after an explicit session object close(). Same as when you open a file, then close it, it becomes inaccessible:

with open("hello.txt") as f:
    print(f.read())

print(f.read())

prints:

Hello

Traceback (most recent call last):
  File "/Users/simon/junk/myfile.py", line 4, in <module>
    print(f.read())
ValueError: I/O operation on closed file.

I was expecting the 2nd and 3rd print to through some kind of error.


Solution

  • Session objects allow settings to persist across requests. In this case, closing a session frees any persistent settings, but the object still exists until any variables referencing it go out of scope. A new request on the session can be made without creating a new Session object.

    Below refers to an example on GeeksForGeeks:

    >>> import requests
    >>> s = requests.Session()
    >>> s.get('https://httpbin.org/cookies/set/sessioncookie/1234') # Set a cookie
    <Response [200]>
    >>> r = s.get('https://httpbin.org/cookies') # retrieve the cookie, in another request
    >>> print(r.text)
    {
      "cookies": {
        "sessioncookie": "1234"
      }
    }
    >>> s.close() # free resources
    >>> r = s.get('https://httpbin.org/cookies') # new session, no cookie for you!
    >>> print(r.text)
    {
      "cookies": {}
    }
    

    Files work differently. A file object can be closed and again the object still exists, but a closed file can't be used for read/write:

    >>> with open('test.txt','w') as f:
    ...   f.write('some text')
    ...   print(f.closed)  # still open
    ...
    9
    False
    >>> print(f.closed)  # outside with, it is closed
    True
    >>> print(f)   # but can still be printed
    <_io.TextIOWrapper name='test.txt' mode='w' encoding='cp1252'>
    >>> f.write('some text')  # but not written to
    Traceback (most recent call last):
      File "<stdin>", line 1, in <module>
    ValueError: I/O operation on closed file.