Could you explain in detail what the difference is between byte string and Unicode string in Python. I have read this:
Byte code is simply the converted source code into arrays of bytes
Does it mean that Python has its own coding/encoding format? Or does it use the operation system settings? I don't understand. Could you please explain? Thank you!
No, Python does not use its own encoding - it will use any encoding that it has access to and that you specify.
A character in a str
represents one Unicode character. However, to represent more than 256 characters, individual Unicode encodings use more than one byte per character to represent many characters.
bytes
objects give you access to the underlying bytes. str
objects have the encode
method that takes a string representing an encoding and returns the bytes
object that represents the string in that encoding. bytes
objects have the decode
method that takes a string representing an encoding and returns the str
that results from interpreting the byte
as a string encoded in the the given encoding.
For example:
>>> a = "αά".encode('utf-8')
>>> a
b'\xce\xb1\xce\xac'
>>> a.decode('utf-8')
'αά'
We can see that UTF-8 is using four bytes, \xce
, \xb1
, \xce
, and \xac
, to represent two characters.
Related reading:
Python Unicode Howto (from the official documentation)
The Absolute Minimum Every Software Developer Absolutely, Positively Must Know About Unicode and Character Sets (No Excuses!) by Joel Spolsky
Pragmatic Unicode by Ned Batchelder