Uploading large video file to Google App Engine

Question

I am trying to upload a large video into Google Cloud Storage via Google App Engine.

I followed the principles in this post which refers to uploading a large spreadsheet. Can't upload large files to Python + Flask in GCP App Engine

I have set up chunking using dropzone.js

I have set up an upload in main.py which I had hoped would upload the file chunks into the app's tmp directory and have put in logic which moves the completed file into Google Cloud Storage once all parts are in place.

I get the following error message:

TypeError: write() argument must be str, not bytes

Here is my back end code

from flask import Flask, render_template, request, redirect, url_for
from google.cloud import storage
from flask_dropzone import Dropzone
from werkzeug.utils import secure_filename
import os
import base64


app = Flask(__name__, template_folder='./templates', static_folder="./static")
dropzone = Dropzone(app)
app.config['UPLOAD_PATH'] = '/tmp'

@app.route('/', methods=['GET', 'POST'])

def index():
    return render_template('index.html')


@app.route('/upload', methods=['POST', 'GET'])
def upload():

    if request.method == 'POST':

    upload_file = request.files.get('file')

    tmp_file_path = '/tmp/' + upload_file.filename

    with open(tmp_file_path, 'a') as f:
        f.write(uploaded_file.read())

    chunk_index = int(flask.request.form.get('dzchunkindex')) if 
(flask.request.form.get('dzchunkindex') is not None)  else 0

    chunk_count = int(flask.request.form.get('dztotalchunkcount'))      if (flask.request.form.get('dztotalchunkcount') is not None)  else 1


    if (chunk_index == (chunk_count - 1)):
        print('Saving file to storage')
        print( chunk_count )

        storage_client = storage.Client()

        storage_bucket = storage_client.get_bucket('percy-277618.appspot.com')

        blob = storage_bucket.blob(upload_file.filename)

        blob.upload_from_filename(tmp_file_path, client=storage_client)
        print('Saved to Storage')

        print('Deleting temp file')
        os.remove(tmp_file_path)

              

if __name__ == '__main__':

    app.run(host='127.0.0.1', port=8080, debug=True)

This is my front end code.

<form id="uploadwidget" method="post" action="/upload" class="dropzone"></form>

<script type="text/javascript">

Dropzone.options.uploadwidget = {

       paramName: 'file',
       forceChunking: true,
       timeout: 300000,
       chunking: true,
       url: '/upload',
       chunkSize: 10485760,
       maxFilesize: 1025,

};

</script>

Answer 1

Using uploaded_file.read() produces bytes, not a string. You should open the file in binary mode:

with open(tmp_file_path, 'ab') as f: