# ! /usr/bin/env python
# -*- coding: utf-8 -*-
# image_upload.py
""" Python 2.7.3
Cherrypy 3.2.2
PostgreSQL 9.1
psycopy2 2.4.5
SQLAlchemy 0.7.10
PIL 1.1.7
"""
I'm trying to save an image and its thumbnail in a SQLAlchemy database, from a client's local file. The upload happens with a HTML form to a CherryPy server. Then the image is processed with the Python Imaging Library (PIL) to get a thumbnail. At last the result should be saved in the SQLAlchemy database, which failed, probably because of the thumbnail.
NOTE: I try to do this without saving the thumbnail temporarily in a file. Since the data is already available in the RAM, I don't like the idea of saving it in a folder, then adding it to the database and finally deleting it from the folder. It just doesn't feel right to do such a thing.
SOLUTION at the end in EDIT 4
A little about the ObjectImage class' table,
which I have to use for this as it is!
It is a requirement! As is Python 2.7
column type attributes
----------------------------------------
id int PRIMARY KEY
object_id int REFERENCES Obeject(id) ON DELETE CASCADE
filename varchar(252)
image bytea
thumbnail bytea
preview boolean
Following is the SQLAlchemy connection to the PostgreSQL database.
It is stored in the CherryPy session as 'session' and retrieved as s1.
Just so nobody gets s1 confused as a CherryPy object.
pg = sqlalchemy.create_engine(
'postgresql://{}:{}@{}:{}/{}'.format(
user, password, server, port, data))
Session = sessionmaker(bind=pg)
cherrypy.session['session'] = Session
Minimalist Python Code:
"""
The variable "image_file"
comes directly from the POSTed dictionary.
image_file = kwargs['image_file']
"""
s1 = cherrypy.session.get('session')
image_entry = {}
img = StringIO.StringIO(image_file.file.read())
image = Image.open(img)
image_entry['image'] = image.copy()
thumb = image.copy()
thumb.thumbnail((30000, 300,), Image.ANTIALIAS)
image_entry['thumbnail'] = thumb.copy()
image_entry['object_id'] = chosen_one
image_entry['filename'] = image_file.filename
image_entry['preview'] = 't'
s1.add(ObjecteImage(**image_entry))
s1.commit() #line 1621
CherryPy Traceback:
File "image_upload.py", line 1621, in store_image
s1.commit()
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 710, in commit
self.transaction.commit()
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 368, in commit
self._prepare_impl()
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 347, in _prepare_impl
self.session.flush()
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 1734, in flush
self._flush(objects)
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 1805, in _flush
flush_context.execute()
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/unitofwork.py", line 331, in execute
rec.execute(self)
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/unitofwork.py", line 475, in execute
uow
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/persistence.py", line 64, in save_obj
table, insert)
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/persistence.py", line 558, in _emit_insert_statements
execute(statement, params)
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1449, in execute
params)
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1584, in _execute_clauseelement
compiled_sql, distilled_params
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/base.py", line 1691, in _execute_context
context)
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/default.py", line 331, in do_execute
cursor.execute(statement, parameters)
TypeError: can't escape instance to binary
In a 2nd attempt I tried to insert at least the originally uploaded file directly, to check for any changes.
Minimalist Python Code:
s1 = cherrypy.session.get('session')
image_entry = {}
img = StringIO.StringIO(image_file.file.read())
image = Image.open(img)
image_entry['image'] = image_file
[ ... the same as above ... ]
s1.add(ObjecteImage(**image_entry))
s1.commit() #line 1621
CherryPy Traceback:
File "image_upload.py", line 1621, in store_image
s1.commit()
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/orm/session.py", line 710, in commit
self.transaction.commit()
[ ... the same as above ... ]
File "/usr/local/lib/python2.7/dist-packages/SQLAlchemy-0.7.10-py2.7-linux-x86_64.egg/sqlalchemy/engine/default.py", line 331, in do_execute
cursor.execute(statement, parameters)
TypeError: can't escape Part to binary
How can I convert the instance from PIL Image or the None from PIL Image thumbnail into a binary?
Or maybe a buffer, bytearray ... I even don't know what I actually need.
Bytearray cast caused:
File "image_upload.py", line 1611, in store_image
image_entry['thumbnail'] = bytearray(thumb.copy())
TypeError: iteration over non-sequence
Buffer cast caused:
File "image_upload.py", line 1611, in store_image
image_entry['thumbnail'] = buffer(thumb.copy())
TypeError: buffer object expected
Didn't find the answer in here:
- Python Imaging Library Handbook (former link)
- CherryPy File Handling
Is there maybe a better tool/library for this?
One that only creates thumbnails?
EDIT 1:
I did a little research on how to save the Image object to a stream.
But first I tried the PIL.Image.tostring() function:
thumb = image.copy()
thumb.thumbnail((30000, 300,), Image.ANTIALIAS)
thumb.tostring()
image_entry['thumbnail'] = thumb.copy()
Then I tried the BytesIO() module.
It caused the UnsupportedOperation: fileno PIL
Since it's a known bug in PIL, I replaced PIL with its fork Pillow and tried again:
thumb = image.copy()
thumb.thumbnail((30000, 300,), Image.ANTIALIAS)
stream = BytesIO()
thumb.save(stream, "JPEG")
image_entry['thumbnail'] = stream.getvalue()
Both got me the SQLAlchemy TypeError: can't escape instance to binary
Of course, like before the error was traced back to:
s1.add(ObjectImage(**image_entry))
s1.commit() #line 1621
At last I replaced BytesIO() with StringIO.StringIO(), but it didn't change a thing.
I guess this is more of a special SQLAlchemy problem.
EDIT 2:
Before I wrongly mentioned the unknown SQLAlchemy TypeError: can't escape Part to binary
Corrected in EDIT 1, it's the SQLAlchemy TypeError: can't escape instance to binary
This only occurred because tried I to save the POSTed value in the database:
"""
The variable "image_file"
comes directly from the POSTed dictionary.
image_file = kwargs['image_file']
Alternative first line:
img = StringIO.StringIO(image_file.file.read())
"""
img = BytesIO(image_file.file.read())
image = Image.open(img)
image_entry['image'] = image_file # the dict value was meant be image.copy()
s1.add(ObjectImage(**image_entry))
s1.commit() #line 1621
EDIT 3:
It seems that I made the mistake, of still trying to insert the full size image as an instance,
while the thumbnail already was "formatted" in the right way.
"""
old version:
"""
img = BytesIO(image_file.file.read())
image = Image.open(img)
image_entry['image'] = image.copy()
"""
new version:
"""
img = BytesIO(image_file.file.read())
image = Image.open(img)
fullsize = image.copy()
stream = BytesIO()
fullsize.save(stream, "JPEG")
image_entry['image'] = stream.getvalue()
"""
from EDIT 1:
"""
thumb = image.copy()
thumb.thumbnail((30000, 300,), Image.ANTIALIAS)
stream = BytesIO()
thumb.save(stream, "JPEG")
image_entry['thumbnail'] = stream.getvalue()
Now at least a very long HEX code is inserted into the table, maybe I can use it.
It seems though, that the image and thumbnail column contain the same HEX code.
EDIT 4:
The image and thumbnail didn't contain the same HEX code, as a SQL-query confirmed later.
Only the first 1179 and last 4 characters were the same, and I had just checked at the beginning.
In between the content was different, as was the length of each entry.
Finally follows the entire code snippet as one.
First the necessary imports:
from io import BytesIO
import base64
import cherrypy
import sqlalchemy
from sqlalchemy.orm import sessionmaker
from PIL import Image
Second the engine and session:
pg = sqlalchemy.create_engine(
'postgresql://{}:{}@{}:{}/{}'.format(
user, password, server, port, data))
Session = sessionmaker(bind=pg)
cherrypy.session['session'] = Session
Third the image and thumbnail uploading code:
s1 = cherrypy.session.get('session')
image_file = kwargs['image_file']
img = BytesIO(image_file.file.read())
image = Image.open(img)
fullsize = image.copy()
stream = BytesIO()
fullsize.save(stream, "JPEG")
image_entry['image'] = stream.getvalue()
thumb = image.copy()
thumb.thumbnail((30000, 300,), Image.ANTIALIAS)
stream = BytesIO()
thumb.save(stream, "JPEG")
image_entry['thumbnail'] = stream.getvalue()
image_entry['sample'] = chosen_one
image_entry['filename'] = image_file.filename
image_entry['preview'] = 't'
s1.add(ObjectImage(**image_entry))
s1.commit() #line 1621
Last a short thumbnail retrieving code for HTML:
s1 = cherrypy.session.get('session')
qry = (s1.query(ObjectImage.id, ObjectImage.filename, ObjectImage.thumbnail).
filter(ObjectImage.preview == 't'))
for rowX in qry:
yield (u'<img src="data:image/jpeg; base64, {}" alt="thumbnail">'.
format(base64.b64encode(rowX.thumbnail)))
For performance reasons, I think about writing an extra function to replace the data URI scheme.
But for now thanks to Neaţu Ovidiu Gabriel, for mentioning the save-to-stream option,
and thanks to the people who provided these resources:
-Python variables and files (simple io.BytesIO() example)
-python Image PIL to binary Hex (mentions the UnsupportedOperation: fileno)
I don't know Cherrypy but my guess is that you are sending as an argument a PIL.Image object to s1 and s1 can't do what it has to do because he doesn't recognize that kind of object.