I have an API I can retrieve files from. I do this using urllib3 in python.
e.g.
url = "https://example.com/api/files/13"
r = http.request('GET', url, headers=headers)
where the headers dict contain information for authentication.
When decoding all the data in the response, get the same result as I would expect. I verified this with the response in the network information terminal of my browser:
load = json.loads(r.data.decode('utf-8'))
data = load['data']
data is at this stage a long string containing characters and digits and looks, depending on the file, similar to dG0seHJheSxtZWRQaG90b24uU04yMDE2LUlSMDA3LkltUmlfRGV0X3BvcyxtZWRQaG90b24uU04yMDE2LUlSMDA3LkltUmlfRGV0X3ZlbCxtZWRQaG90b24uU04yMDE2LUlSMDA3LkltUmlfRGV0X2FjYyxtZWRQaG90b24uU04yMDE2LUlSMDA3LkltUmlfUm5nX3BvcyxtZWRQaG90b24uU04yMDE2LUlSMDA3LkltUmlfUm5nX3ZlbCxtZWRQaG90b..., continuing after the dots. But it is just a csv file.
The only challenge I face now is to store the data to the disc. What I tried so far was:
with open(dst_file, 'wb') as out_file:
out_file.write(data)
Instead of the outfile.write(...) I also tried shutil.copyfileobj(r, out_file), but does not work either.
I guess it depends on the representation of the data I have it present. I guess having it as a string is not really beneficial. But how to store it?
EDIT:
One thing that maybe have to be noted: The api url does not contain the file, so the call will be redirected. I tried it with another file from the web, here i did get the desired data immediately presented, when accessing >> r.data.
The problem was a bit of a different one. I was not aware, how the data was encoded and that it even was (honestly, didn't expect a small csv to be encoded).
Using the retrieved data and decoding it with base64 solved the problem in this certain case.
import base64
...
data_str = base64.b64decode(load["data"])
...