javascriptpythonfileuploadcgi
Sending uploaded file from JS to Python CGI always results in ampty dictionary
I have an HTML form that has a file input element. When submitting the form, a JS function is executed which checks if the uploaded file is a PDF file and is less than 10MB. All of this works correctly and the correct filesize is logged in the console.
Then the JS function sends the file to the Python server through an HTTP POST request. However, the sent file is always an empty object / dictionary. When I console.log(file); before it is sent, all the file data is logged but when I look at the Network tab in the inspector, it's just an empty object / dictionary:
However, when I change my HTML form so it sends the data directly to the Python CGI script without calling the JS function first, it works perfectly.
Here's my HTML code:
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta http-equiv="X-UA-Compatible" content="IE=edge" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<script src="./js/new.js"></script>
<title>test</title>
</head>
<body>
<script></script>
<header>
<h1>File upload test</h1>
</header>
<main>
<fieldset>
<form
action="javascript:addFile()"
id="DataForm"
>
<div>
<label for="file_contract">contract:</label
><input
type="file"
accept="application/pdf"
id="file_contract"
name="file_contract"
/>
</div>
<button type="button" class="outlined" onclick="cancel()">
Annuleer
</button>
<input type="submit" value="Bewaar" />
</form>
</fieldset>
</main>
<script>
function cancel() {
const host = window.location.host;
const redirectUrl = `http://${host}`;
window.location.replace(redirectUrl);
}
</script>
</body>
</html>
JavaScript code:
async function addFile() {
const formData = new FormData(document.forms["DataForm"]);
// get uploaded files
// Function to check and append a file to the formData
function handleFileUpload(fileInputId) {
console.log("uploading file...");
const fileInput = document.getElementById(fileInputId);
const file = fileInput.files[0];
// Check if a file is selected
// Ensure the file is a PDF
if (file.type === "application/pdf") {
console.log(`filesize = ${file.size}`);
// 10MB in bytes
formData.append(fileInputId, fileInput.files[0]);
}
return true;
}
// Handle each file upload separately
if (!handleFileUpload("file_contract")) {
console.log("form submission prevented");
return false; // Prevent form submission
}
const data = {
answers: Object.fromEntries(formData),
};
console.log(data["answers"]["file_contract"]);
try {
const response = await fetch("../../cgi-bin/addRecord.py", {
method: "POST",
headers: {
"Content-type": "application/json; charset=UTF-8",
},
body: JSON.stringify(data),
});
const responseData = await response.json();
if (responseData.hasOwnProperty("status")) {
if (responseData["status"] === "failed") {
// session is invalid / expired
alert("Login sessie vervallen. Log opnieuw in.");
} else if (responseData.status === "success") {
const host = window.location.host;
const redirectUrl = `http://${host}`;
console.log("redirecting... (commented out)");
// window.location.replace(redirectUrl);
} else {
alert(
"Fout bij het opslaan. Probeer het nog eens of contacteer Sandra."
);
}
}
} catch (error) {
console.error(error);
alert("Fout bij het opslaan. Probeer het nog eens of contacteer Sandra.");
}
}
Python CGI:
#!/usr/bin/env python3
import cgi
import hashlib
import json
import os
import sys
import logging
# Get the absolute path of the current script
dirname = os.path.dirname(os.path.dirname(os.path.abspath(__file__)))
if os.path.exists(f'{dirname + "/log.log"}'):
# If it exists, open it in write mode to clear its contents
with open(f'{dirname + "/log.log"}', 'w'):
pass
logging.basicConfig(filename=f'{dirname + "/log.log"}', level=logging.DEBUG)
def generate_response(status, data=None):
response = {"status": status}
if data:
response["data"] = data
print("Content-Type: application/json")
print()
print(json.dumps(response))
def calculate_file_hash(file_data):
# Create an MD5 hash object
md5_hash = hashlib.md5()
# Read the file data in chunks to efficiently handle large files
for chunk in iter(lambda: file_data.read(4096), b''):
md5_hash.update(chunk)
# Return the hexadecimal representation of the MD5 hash
return md5_hash.hexdigest()
def main():
# read form data from HTTP POST request
data = sys.stdin.read(int(os.environ.get('CONTENT_LENGTH', 0)))
post_data = json.loads(data)
answers = post_data["answers"]
logging.debug(str(answers))
if "file_contract" in answers:
contract_file = answers['file_contract']
contract_file_filename = contract_file.filename
contract_file_hash = calculate_file_hash(contract_file.file)
# save the file data to the werkmap
# Save the PDF file to a folder
contract_filename_path = os.path.join(dirname, "documenten", contract_file_hash)
with open(contract_filename_path, 'wb') as contract_file_handle:
contract_file_handle.write(contract_file.file.read())
generate_response("success")
if __name__ == "__main__":
main()
Solution
Problem is JSON.stringify. It can't convert object File to string JSON.
It would be simpler to send it as normal FormData (and add other elements to this form).
If you really have to send it as JSON then you may need to use FileReader and function readAsDataURL() to convert file content to Base64. It is popular method to send data (ie. images, data files) as JSON.
const file = formData.get('file_contract');
const reader = new FileReader();
reader.onload = your_function_executed_after_converting;
reader.readAsDataURL(file);
Minimal working example. I removed elements which I didn't need to test this problem.
async function addFile() {
const formData = new FormData(document.forms["DataForm"]);
const file = formData.get('file_contract');
const reader = new FileReader();
reader.onload = () => {
//const data = {'answers': reader.result} // send only file contemt
const data = {'answers':
{
'data': reader.result, // file content as `Base64`
'name': file.name,
'size': file.size,
'type': file.type,
'lastModified': file.lastModified,
}
}
fetch("/cgi-bin/addRecord.py", {
method: "POST",
headers: {
//'Accept': 'application/json, text/plain, */*', // expect JSON or TEXT as response from server
'Content-Type': 'application/json; charset=UTF-8' // inform server that you send JSON
},
body: JSON.stringify(data),
}).then((response) => {
if(response.ok) {
const responseData = response.json();
//TODO: use responseData
}
});
}
reader.readAsDataURL(file); // convert to `Base64` and execute `onload()`
}
Because it sends data as base64 (with prefix application/pdf;base64,) so later in Python you have to remove prefix application/pdf;base64, and decode base64 back to normal bytes.
import base64
file_as_bytes = base64.b64decode(file_as_base64)
I tested problem using framework flask instead of CGI.
Here is full working code which I used to test it.
All code (HTML, JavaScript, Python) is in one file.
import os
from flask import Flask, request, Response
import base64
app = Flask(__name__)
@app.route('/')
def index():
return '''<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8" />
<meta name="viewport" content="width=device-width, initial-scale=1.0" />
<script src="code.js"></script>
<title>test</title>
</head>
<body>
<form method="POST" enctype="multipart/form-data" action="javascript:addFile()" id="DataForm">
<div>
<label for="file_contract">contract:</label>
<input type="file" accept="application/pdf" id="file_contract" name="file_contract" />
</div>
<button type="button" onclick="cancel()">Cancel</button>
<input type="submit" value="Submit" />
</form>
</body>
</html>
'''
@app.route('/code.js')
def code():
text = '''async function addFile() {
const formData = new FormData(document.forms["DataForm"]);
const file = formData.get('file_contract');
const reader = new FileReader();
reader.onload = () => {
//const data = {'answers': reader.result} // send only file contemt
const data = {'answers':
{
'data': reader.result, // file content as `Base64`
'name': file.name,
'size': file.size,
'type': file.type,
'lastModified': file.lastModified,
}
}
fetch("/addRecord", {
method: "POST",
headers: {
//'Accept': 'application/json, text/plain, */*', // expect JSON or TEXT as response from server
'Content-Type': 'application/json; charset=UTF-8' // inform server that you send JSON
},
body: JSON.stringify(data),
}).then((response) => {
if(response.ok) {
const responseData = response.json();
//TODO: use responseData
}
});
}
reader.readAsDataURL(file); // convert to `Base64` and execute `onload()`
}
'''
return Response(text, mimetype='application/javascript')
@app.route('/addRecord', methods=['POST'])
def add_record():
print('args :', request.args)
print('form :', request.form)
if request.is_json:
print('json :', request.json)
print('files:', request.files)
print('data :', request.data[:100]) # crop data to 100 chars - to display only short example
# ----------------------------------------
file_info = request.json['answers']
data = file_info['data'] # file data with prefix `application/pdf;base64,`
# find beginning of data (after prefix `application/pdf;base64,`)
file_start = data.find(',') + 1 # +1 == len(',')
# crop data to remove prefix
file_data = data[file_start:]
file_as_bytes = base64.b64decode(file_data)
print(file_as_bytes[:100]) # crop data to 100 chars - to display only short example
# write data to file on disk (using "bytes mode")
with open(file_info['name'], 'wb') as f_out:
f_out.write(file_as_bytes)
#file_as_text = file_as_bytes.decode("utf-8")
#print(file_as_text[:100]) # crop data to 100 chars - to display only short example
return {'status': 'OK'} # it will send dictionary as JSON
if __name__ == '__main__':
#app.debug = True
#app.run(threaded=False, processes=3)
#app.run(use_reloader=False)
app.run()
- How to fetch data server-side in the latest Next.js? Tried getStaticProps but it's not running and getting undefined
- How can I split a string containing emoji into an array?
- nodemailer Invalid login: 535 Authentication Failed
- Jquery Remove All Event Handlers Inside Element
- Multiple Vimeo video in multiple modals
- How to convert ReadStream into ReadableStream in NodeJS?
- How to create a backdrop for a dialog?
- How can an object have a `.constructor` property if I reset its prototype to an empty object (`{}`)?
- Different way of showing and hiding certain elements
- Is there a good use case for the constructor property in Javascript?
- How to make search in select box?
- My scrollView compnent is not working despite all the online solutions
- Why does my linter warn “Use object destructuring (prefer destructure)”?
- Is it possible to keep scheduled tasks from overlapping in NestJS?
- export html table as word file and change file orientation
- Vis.js callback when the network finishes loading
- Is it safe to use the `Function` constructor to validate JavaScript syntax?
- Equivalent of destructors in JavaScript's object model
- How to slow down scroll to top speed?
- How can I tell if it is an object or an array?
- Bottom of gradient text gets cut off tailwindcss
- How to share state between reducers in redux (how to organize redux)?
- I'm trying to create a password generator
- how to convert a raw readme file to html formated document after geting it from github in angular js
- How to use npm Marked with HighlightJS
- Avoid browser popup blockers
- What does this function do? And what is the odd syntax?
- Next.js server side component request, how to send my cookie token?
- Javascript single quote inside string for exported text file
- Passing in a result of a function to generate readme