Search code examples
command-lineiconv

failed to use iconv to change encoding in shell

I have a file to convert its encoding. In terminal, I have :

$file -i *

and it turns out text/plain; charset=us-ascii

Then I have this command line :

$ iconv -f us-ascii -t UTF-8 oldfile.txt > newfile.txt

It turns out a message: iconv: illegal input sequence at position 2575661

I searched and found that this suggests there are invalid sequences (for instance this page https://spin.atomicobject.com/2011/07/13/some-useful-iconv-functionality/). I am wondering what would be the most reliable way to convert the encoding to the most appropriate encoding to sqlite, and how to find the invalid sequences? Thank you!!

Solution

  • Would it help the process if you typed $ iconv -c -f us-ascii -t UTF-8 oldfile.txt > newfile.txt

    ...otherwise try: $ iconv -f US-ASCII -t UTF-8 oldfile.txt > newfile.txt and with the -c-flag.