Issue with sending POST requests using the library requests

Question

import requests
while True: 
    try:
        posting = requests.post(url,json = data,headers,timeout = 3.05)
    except requests.exceptions.ConnectionError as e: 
        continue
    # If a read_timeout error occurs, start from the beginning of the loop
    except requests.exceptions.ReadTimeout as e:  
        continue

a link to more code : Multiple accidental POST requests in Python This code is using requests library to perform POST requests indefinitely. I noticed that when try fails multiple of times and the while loop starts all over multiple of times, that when I can finally send the post request, I find out multiple of entries from the server side at the same second. I was writing to a txt file at the same time and it showed one entry only. Each entry is 5 readings. Is this an issue with the library itself? Is there a way to fix this?! No matter what kind of conditions that I put it still doesn't work :/ ! You can notice the reading at 12:11:13 has 6 parameters per second while at 12:14:30 (after the delay, it should be every 10 seconds) it is a few entries at the same second!!! 3 entries that make up 18 readings in one second, instead of 6 only!

Answer 1

It looks like the server receives your requests and acts upon them but fails to respond in time (3s is a pretty low timeout, a load spike/paging operation can easily make the server miss it unless it employs special measures). I'd suggest to

• process requests asynchronously (e.g. spawn threads; Asynchronous Requests with Python requests discusses ways to do this with requests) and do not use timeouts (TCP has its own timeouts, let it fail instead).
• reuse the connection(s) (TCP has quite a bit of overhead for connection establishing/breaking) or use UDP instead.
• include some "hints" (IDs, timestamps etc.) to prevent the server from adding duplicate records. (I'd call this one a workaround as the real problem is you're not making sure if your request was processed.)

From the server side, you may want to:

• Respond ASAP and act upon the info later. Do not let pending action prevent answering further requests.