I try to export a simple table to dbf using dbf module. I have probleme with some special characters. This is what i want to do:
import dbf
table = dbf.Table('temptable', 'name C(30); age N(3,0); birth D')
table.open(mode=dbf.READ_WRITE)
for datum in (
('John Doe', 31, dbf.Date(1979, 9,13)),
('Ethan Furman', 102, dbf.Date(1909, 4, 1)),
('Jan Mężny', 57, dbf.Date(1954, 7, 2)),
):
table.append(datum)
When i do that i get:
UnicodeEncodeError: 'ascii' codec can't encode characters in position 5-6: ordinal not in range(128)
When i try:
'Jan Mężny'.encode('utf-8').strip()
i get:
ValueError: unable to coerce <class 'bytes'>(b'Jan M\xc4\x99\xc5\xbcny') to string
I don't know how to deal with it. Can you help me?
TL;DR You need to provide a codepage for your table:
table = dbf.Table('temptable', 'name C(30); age N(3,0); birth D', codepage='...')
It looks like you are using Python 2, because in Python 3 the str type is unicode.
Your REPL command should be a decode, not encode:
>>> print('Jan Mężny'.decode('utf-8').strip())
Jan Mężny
For your dbf Table you need to provide a code page (aka encoding) as the default 'ascii' cannot handle anything above chr(127). The available code pages are:
0x00 : ('ascii', "plain ol' ascii"),
0x01 : ('cp437', 'U.S. MS-DOS'),
0x02 : ('cp850', 'International MS-DOS'),
0x03 : ('cp1252', 'Windows ANSI'),
0x04 : ('mac_roman', 'Standard Macintosh'),
0x08 : ('cp865', 'Danish OEM'),
0x09 : ('cp437', 'Dutch OEM'),
0x0A : ('cp850', 'Dutch OEM (secondary)'),
0x0B : ('cp437', 'Finnish OEM'),
0x0D : ('cp437', 'French OEM'),
0x0E : ('cp850', 'French OEM (secondary)'),
0x0F : ('cp437', 'German OEM'),
0x10 : ('cp850', 'German OEM (secondary)'),
0x11 : ('cp437', 'Italian OEM'),
0x12 : ('cp850', 'Italian OEM (secondary)'),
0x13 : ('cp932', 'Japanese Shift-JIS'),
0x14 : ('cp850', 'Spanish OEM (secondary)'),
0x15 : ('cp437', 'Swedish OEM'),
0x16 : ('cp850', 'Swedish OEM (secondary)'),
0x17 : ('cp865', 'Norwegian OEM'),
0x18 : ('cp437', 'Spanish OEM'),
0x19 : ('cp437', 'English OEM (Britain)'),
0x1A : ('cp850', 'English OEM (Britain) (secondary)'),
0x1B : ('cp437', 'English OEM (U.S.)'),
0x1C : ('cp863', 'French OEM (Canada)'),
0x1D : ('cp850', 'French OEM (secondary)'),
0x1F : ('cp852', 'Czech OEM'),
0x22 : ('cp852', 'Hungarian OEM'),
0x23 : ('cp852', 'Polish OEM'),
0x24 : ('cp860', 'Portugese OEM'),
0x25 : ('cp850', 'Potugese OEM (secondary)'),
0x26 : ('cp866', 'Russian OEM'),
0x37 : ('cp850', 'English OEM (U.S.) (secondary)'),
0x40 : ('cp852', 'Romanian OEM'),
0x4D : ('cp936', 'Chinese GBK (PRC)'),
0x4E : ('cp949', 'Korean (ANSI/OEM)'),
0x4F : ('cp950', 'Chinese Big 5 (Taiwan)'),
0x50 : ('cp874', 'Thai (ANSI/OEM)'),
0x57 : ('cp1252', 'ANSI'),
0x58 : ('cp1252', 'Western European ANSI'),
0x59 : ('cp1252', 'Spanish ANSI'),
0x64 : ('cp852', 'Eastern European MS-DOS'),
0x65 : ('cp866', 'Russian MS-DOS'),
0x66 : ('cp865', 'Nordic MS-DOS'),
0x67 : ('cp861', 'Icelandic MS-DOS'),
0x68 : (None, 'Kamenicky (Czech) MS-DOS'),
0x69 : (None, 'Mazovia (Polish) MS-DOS'),
0x6a : ('cp737', 'Greek MS-DOS (437G)'),
0x6b : ('cp857', 'Turkish MS-DOS'),
0x78 : ('cp950', 'Traditional Chinese (Hong Kong SAR, Taiwan) Windows'),
0x79 : ('cp949', 'Korean Windows'),
0x7a : ('cp936', 'Chinese Simplified (PRC, Singapore) Windows'),
0x7b : ('cp932', 'Japanese Windows'),
0x7c : ('cp874', 'Thai Windows'),
0x7d : ('cp1255', 'Hebrew Windows'),
0x7e : ('cp1256', 'Arabic Windows'),
0xc8 : ('cp1250', 'Eastern European Windows'),
0xc9 : ('cp1251', 'Russian Windows'),
0xca : ('cp1254', 'Turkish Windows'),
0xcb : ('cp1253', 'Greek Windows'),
0x96 : ('mac_cyrillic', 'Russian Macintosh'),
0x97 : ('mac_latin2', 'Macintosh EE'),
0x98 : ('mac_greek', 'Greek Macintosh'),
0xf0 : ('utf8', '8-bit unicode'),
Two things to note:
1) use the first item of the tuple, not the hex code; so, for example, codepage='cp1252' for ANSI.
2) the last entry ('utf8') is non-standard -- only use it if you will not be sharing your tables with other programs; also, give yourself a few extra bytes to store your data in, because
>>> len(u'Jan Mężny')
9
but
>>> len(u'Jan M\u0119\u017cny'.encode('utf-8'))
11