Search code examples
python-3.xtextunicodenewlinefile-writing

Python 3: Issue writing new lines along side unicode to text file

I ran into an issue when writing the header of a text file in python 3. I have a header that contains unicode AND new line characters. The following is a minimum working example:

with open('my_log.txt', 'wb') as my_file:
    str_1 = '\u2588\u2588\u2588\u2588\u2588\n\u2588\u2588\u2588\u2588\u2588'
    str_2 = 'regular ascii\nregular ascii'
    my_file.write(str_1.encode('utf8'))
    my_file.write(bytes(str_2, 'UTF-8'))

The above works, except the output file does not have the new lines (it basically looks like I replaced '\n' with ''). Like the following:

████████regular asciiregular ascii

I was expecting:

████
████
regular ascii
regular ascii

I have tried replacing '\n' with u'\u000A' and other characters based on similar questions - but I get the same result.

An additional, and probably related, question: I know I am making my life harder with the above encoding and byte methods. Still getting used to unicode in py3 so any advice regarding that would be great, thanks!

EDIT Based on Ignacio's response and some more research: The following seems to produce the desired results (basically converting from '\n' to '\r\n' and ensuring the encoding is correct on all the lines):

with open('my_log.txt', 'wb') as my_file:
    str_1 = '\u2588\u2588\u2588\u2588\u2588\r\n\u2588\u2588\u2588\u2588\u2588'
    str_2 = '\r\nregular ascii\r\nregular ascii'
    my_file.write(str_1.encode('utf8'))
    my_file.write(str_2.encode('utf8'))
Solution

  • 'wb'

    The file is open in binary mode. As such \n isn't translated into the native newline format. If you open the file in a text editor that doesn't treat LF as a line break character then all the text will appear on a single line in the editor. Either open the file in text mode with an appropriate encoding or translate the newlines manually before writing.