I have a simple webserver running on an RPi Pico W which just displays the input message and sends back JSON. I'm able to access and retrieve data via a web browser and with cURL. However, when I try to access through Python's requests, most of the time there's no data being passed to it.
Driver code for making the request:
import json
import requests
data = json.dumps({"KEY":"VALUE"})
res = requests.put("http://webserver.ip:port", data=data)
print(res.json())
Code on the Pico W to accept connections:
i = 0
while True:
client = conn.accept()[0]
print(f"[{i}] Client connected: {client}")
msg = client.recv(8192)
print(f"Message: {msg}")
data = msg.decode("utf-8").split("\r")[-1].strip()
print(f'Data: {data}, {type(data)}, {data == ""}')
message = {
"data": ujson.loads(data) if data != "" else None,
"key1": "value1",
"key2": "value2",
"key3": "value3",
}
client.send(b"HTTP/1.1 200 OK\n")
client.send(b"Content-Type: application/json\n")
client.send(b"Connection: keep-alive\n\n")
client.sendall(ujson.dumps(message).encode("utf-8"))
client.close()
i += 1
On the Pico, after processing the input message, I set a key in the response data to be the data passed in; if there is no data passed in, I set it to None. I expect the above request to send back:
{"key1": "value1", "key2": "value2", "key3": "value3", "data": {"KEY":"VALUE"}}
but instead I get:
{"key1": "value1", "key2": "value2", "key3": "value3", "data": None}
Running the same request with cURL does give me the expected response.
My question is: why is it that data isn't being sent with the request? I'm not quite sure what's causing the actual data to be empty when reaching the Pico.
It's all powered by simple sockets, so there's not too much complexity with getting the message. Changing the kwarg in the driver code from data=data to json=data also doesn't help (this assumes I'm sending a dict rather than a JSON string).
So I've fixed it for now! Thanks to @jasonharper for the insight. It was in fact because there's no guarantee that all of the message will have been sent when I run client.recv(). Thus, I added a retry based on the content length.
def serve(conn):
i = 0
while True:
sleep(1)
client = conn.accept()[0]
print(f"[{i}] Client connected: {client}")
msg = client.recv(4096)
print(f"Message: {msg}")
headers, data = process_message(msg)
if ("Content-Length" in headers and
int(headers["Content-Length"]) > len(data)):
mtmp = client.recv(4096)
_, dtmp = process_message(mtmp)
data = dtmp
print(f'Data: {data}, {type(data)}, {data == ""}')
print(f'Headers: {headers}')
message = {
"data": ujson.loads(data) if data != "" else None,
"key1": "value1",
"key2": "value2",
"key3": "value3",
}
client.send(b"HTTP/1.1 200 OK\n")
client.send(b"Content-Type: application/json\n")
client.send(b"Connection: keep-alive\n\n")
client.sendall(ujson.dumps(message).encode("utf-8"))
client.close()
i += 1
I needed a way to reliably get Content-Length, so I wrote this to process the received message:
def process_message(message):
message_dict = {
"Info": [],
}
message_split = list(map(lambda x: x.strip(), message.decode("utf-8").split("\r")))
headers = message_split[:-1]
body = message_split[-1]
for i in headers:
if i == "":
continue
key_value = list(map(lambda x: x.strip(), i.split(":")))
if len(key_value) < 2:
message_dict["Info"] += key_value
continue
message_dict[key_value[0]] = key_value[1]
return message_dict, body
And now it works! Running the driver code from my question gets the expected response.
[2024-3-27] [Edit] Looking at this code again, I was stumped why process_message looked so dense and tedious. Just have to keep in mind that on input, we're receiving bytes, thus we have to do string processing to get the data out. There's a probably a better way to accomplish what I'm doing here.