UnicodeDecodeError when sending the result of a function in CherryPy

Question

Hı All,

I want to send a string variable to web app. Normally, I can send a string without any problem when I create it manually inside the 'switch' part of the code. But when I obtain that string from a function, I get an error. I mean, if counterstring='12.3' , there is no problem. But if counterstring=ReadCounter(), there is a problem.

Here is the actual part of code:

import cherrypy
import webbrowser
import os
import json
import sys

MEDIA_DIR = os.path.join(os.path.abspath("."), u"media")

class AjaxApp(object):
    @cherrypy.expose
    def index(self):
        return open(os.path.join(MEDIA_DIR, u'index.html'))

    @cherrypy.expose
    def switch(self):
        counterstring=ReadCounter()
        return counterstring

config = {'/media':
                {'tools.staticdir.on': True,
                 'tools.staticdir.dir': MEDIA_DIR,
                }
        }

def open_page():
    webbrowser.open("http://127.0.0.1:8080/")
    cherrypy.engine.subscribe('start', open_page)
    cherrypy.tree.mount(AjaxApp(), '/', config=config)
    cherrypy.engine.start()

and the error is :

    ERROR:cherrypy.error.55086800:[27/Jan/2015:02:47:09]  Traceback (most recent call last):
      File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 589, in run
        self.respond(pi)
      File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 690, in respond
        self.handle_error()
      File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 767, in handle_error
        self.error_response()
      File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 401, in set_response
        message=self._message)
      File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 407, in get_error_page
        return get_error_page(*args, **kwargs)
      File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 535, in get_error_page
        return result.encode('utf-8')
    UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 2271: ordinal not in range(128)

ERROR:cherrypy.error.42068624:[29/Jan/2015:09:14:46]  Traceback (most recent call last):
  File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 589, in run
    self.respond(pi)
  File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 690, in respond
    self.handle_error()
  File "C:\Python27\lib\site-packages\cherrypy\_cprequest.py", line 767, in handle_error
    self.error_response()
  File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 402, in set_response
    message=self._message)
  File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 408, in get_error_page
    return get_error_page(*args, **kwargs)
  File "C:\Python27\lib\site-packages\cherrypy\_cperror.py", line 536, in get_error_page
    return result.encode('utf-8')
UnicodeDecodeError: 'ascii' codec can't decode byte 0xce in position 2294: ordinal not in range(128)

I tried many things about encoding/decoding. I added encode/decode functions to strings in app

I have # -- coding: utf-8 -- at the top of my code.

Could you please tell about your suggestions to solve this out?

Thanks in advance.

Answer 1

It's not the problem of your code, as you can see the stack trace ends in get_error_page. It's CherryPy bug (or Python traceback.format_exc()'s bug as it produces quite a mess), I also met some ago. I've filed bug report, #1356, to CherryPy.

So if you can just avoid unicode exception messages. Other the workaround is to set custom error page handler:

#!/usr/bin/env python
# -*- coding: utf-8 -*-


import cherrypy


def errorPage(**kwargs):
  temaplte = cherrypy._cperror._HTTPErrorTemplate
  return temaplte % kwargs


config = {
  'global' : {
    'server.socket_host' : '127.0.0.1',
    'server.socket_port' : 8080,
    'server.thread_pool' : 8,
    # here's the workaround
    'error_page.default' : errorPage
  }
}


class App:

  @cherrypy.expose
  def index(self):
    return '''
      <a href='/errorPageStr'>errorPageStr</a><br/>
      <a href='/errorPageUnicode'>errorPageUnicode</a>
    '''

  @cherrypy.expose
  def errorPageStr(self):
    raise RuntimeError('Şansal Birbaş')

  @cherrypy.expose
  def errorPageUnicode(self):
    raise RuntimeError(u'Şansal Birbaş')


if __name__ == '__main__':
  cherrypy.quickstart(App(), '/', config)

EDIT: To make it easier to understand the answer. Your code, somewhere likely out of the snippet in question, raises an exception with unicode message. CherryPy tries to generate error page for your exception and fails because stack trace in such a case is messy. It doesn't matter whether exception message is UTF-8 bytes or unicode. The code above is just a distilled version.

You need to set custom error handler in your code so you can see your original exception.