Search code examples
djangopython-2.7utf-8weasyprint

Trying to save a PDF string results in UnicodeDecodeError with WeasyPrint

So far this is my code:

from django.template import (Context, Template) # v1.11
from weasyprint import HTML  # v0.42
import codecs

template = Template(codecs.open("/path/to/my/template.html", mode="r", encoding="utf-8").read())
context = Context({})
html = HTML(string=template.render(context))

pdf_file = html.write_pdf()

#with open("/path/to/my/file.pdf", "wb") as f:
#    f.write(self.pdf_file)

Errorstack:

[17/Jan/2019 08:14:13] INFO [handle_correspondence:54] 'utf8' codec can't
decode byte 0xe2 in position 10: invalid continuation byte. You passed in
'%PDF-1.3\n%\xe2\xe3\xcf\xd3\n1 0 obj\n<</Author <> /Creator (cairo 1.14.6
(http://cairographics.org))\n  /Keywords <> /Producer (WeasyPrint 0.42.3
\\(http://weasyprint.org/\\))>>\nendobj\n2 0 obj\n<</Pages 3 0 R /Type
/Catalog>>\nendobj\n3 0 obj\n<</Count 1 /Kids [4 0 R] /Type
/Pages>>\nendobj\n4 0 obj\n<</BleedBox [0 0 595 841] /Contents 5 0 R
/Group\n  <</CS /DeviceRGB /I true /S /Transparency /Type /Group>>
MediaBox\n  [0 0 595 841] /Parent 3 0 R /Resources 6 0 R /TrimBox [0 0 595
841]\n  /Type /Page>>\nendobj\n5 0 obj\n<</Filter /FlateDecode /Length 15
0 R>>\nstream\nx\x9c+\xe4*T\xd0\x0fH,)I-\xcaSH.V\xd0/0U(N\xceS\xd0O4PH/\xe62P0P0\xb54U\xb001T(JUH\xe3\n\x04B\x00\x8bi\r\x89\nendstream\nendobj\n6 0
obj\n<</ExtGState <</a0 <</CA 1 /ca 1>>>> /Pattern <</p5 7 0
R>>>>\nendobj\n7 0 obj\n<</BBox [0 1123 794 2246] /Length 8 0 R /Matrix
[0.75 0 0 0.75 0 -843.5]\n  /PaintType 1 /PatternType 1 /Resources
<</XObject <</x7 9 0 R>>>>\n  /TilingType 1 /XStep 1588 /YStep
2246>>\nstream\n /x7 Do\n \n\nendstream\nendobj\n8 0 obj\n10\nendobj\n9 0
obj\n<</BBox [0 1123 794 2246] /Filter /FlateDecode /Length 10 0 R
/Resources\n  11 0 R /Subtype /Form /Type /XObject>>\nstream\nx\x9c+\xe4\nT(\xe42P0221S0\xb74\xd63\xb3\xb4T\xd05442\xd235R(JU\x08W\xc8\xe3*\xe42T0\x00B\x10\t\x942VH\xce\xe5\xd2O4PH/V\xd0\xaf04Tp\xc9\xe7\n\x04B\x00`\xf0\x10\x11\nendstream\nendobj\n10 0 obj\n77\nendobj\n11 0 obj\n<</ExtGState
<</a0 <</CA 1 /ca 1>>>> /XObject <</x11 12 0 R>>>>\nendobj\n12 0
obj\n<</BBox [0 1123 0 1123] /Filter /FlateDecode /Length 13 0 R
/Resources\n  14 0 R /Subtype /Form /Type /XObject>>\nstream\nx\x9c+\xe4\n
xe4\x02\x00\x02\x92\x00\xd7\nendstream\nendobj\n13 0 obj\n12\nendobj\n14 0
obj\n<<>>\nendobj\n15 0 obj\n58\nendobj\nxref\n0 16\n0000000000 65535
f\r\n0000000015 00000 n\r\n0000000168 00000 n\r\n0000000215 00000
n\r\n0000000270 00000 n\r\n0000000489 00000 n\r\n0000000620 00000
n\r\n0000000697 00000 n\r\n0000000923 00000 n\r\n0000000941 00000
n\r\n0000001165 00000 n\r\n0000001184 00000 n\r\n0000001264 00000
n\r\n0000001422 00000 n\r\n0000001441 00000 n\r\n0000001462 00000
n\r\ntrailer\n\n<</Info 1 0 R /Root 2 0 R /Size 16>>\nstartxref\n1481
n%%EOF\n' (<type 'str'>)

Actually it works via web request (returning the PDF as response) and via shell (manually writting the code). The code is tested and never gaves me problems. The files are saved with correct encoding, and setting the encoding kwarg in HTML doesn't help; also, the mode value of the template is correct, because I've seen other questions whose problem could be that.

However, I was adding a management command to use it periodically (for bigger PDFs I cannot do it via web request because the server's timeout could activate before finishing), and when I try to call it, I only get a UnicodeDecodeError saying 'utf8' codec can't decode byte 0xe2 in position 10: invalid continuation byte.

The PDF (at least from what I see) renders initially with this characters:

%PDF-1.3\n%\xe2\xe3\xcf\xd3\n1 0

which translates into this:

%PDF-1.3
%âãÏÓ
1 0 obj

So the problem is all about the character â. But it's a trap!

Instead, the problem is this line of code:

pdf_file = html.write_pdf()

Changing it to:

html.write_pdf()

Just works as expected!

So my question is: what type of reason could exists for Python to throw an UnicodeDecodeError when trying to assign a variable to a string? I've digged into weasyprint's code in my virtualenv, but I didn't see conversions out there.

Solution

  • So I don't know why, but now suddenly it works. I literally didn't modify anything: I just run the command again and it works.

    I'm not marking the question as answered, as maybe in the future someone could have the same problem as me can try to post a correct one.

    So disturbing.

    EDIT

    So it looks like I'm a very intelligent person who tries to set up the value of self.pdf_file, which is a models.FileField, to the content of the created PDF instead of the file itself.