Search code examples
djangopython-3.xdjango-storagepyexcel

How to create a file in memory (not an uploaded file) and save to FileField through Django default_storage?

If I was to create a file-like csv object in memory like so:

output_stream = io.StringIO()
sheet = pyexcel.get_sheet(records=data)
sheet.save_to_memory(file_type='csv', stream=output_stream)

What can I do to save the file like object in output_stream to a file on my default_storage backend with Django?

class Example(models.Model):
    model_file = models.FileField(upload_to='', max_length=255, blank=True, null=True)

I've tried something like:

self.model_file.save(filename, ContentFile(output_stream.read()))

But I get the following error:

"TypeError: ('data must be bytes, received', <class 'str'>)"

pyexcel only supports io.StringIO streams for csv type files.

Traceback:

...
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\django\core\files\storage.py", line 49, in save
    return self._save(name, content)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\storages\backends\gcloud.py", line 167, in _save
    content_type=file.mime_type)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\cloud\storage\blob.py", line 1034, in upload_from_file
    size, num_retries, predefined_acl)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\cloud\storage\blob.py", line 947, in _do_upload
    size, num_retries, predefined_acl)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\cloud\storage\blob.py", line 759, in _do_multipart_upload
    transport, data, object_metadata, content_type)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\resumable_media\requests\upload.py", line 94, in transmit
    data, metadata, content_type)
  File "C:\Users\Cole\AppData\Local\Programs\Python\Python36-32\lib\site-packages\google\resumable_media\_upload.py", line 270, in _prepare_request
    raise TypeError(u'`data` must be bytes, received', type(data))
TypeError: ('`data` must be bytes, received', <class 'str'>)
Solution

  • You can read the contents from a StringIO and convert to utf8 encoded bytes like this.

    self.model_file.save(filename, ContentFile(output_stream.getvalue().encode()))

    getvalue() is similar to read(), but you will always get the full content of the StringIO regardless of the current stream position. With read() you might have to rewind using seek(0).