How to handle iOS string characters parsing in node (Japanese characters)?

Question

I'm running into issues uploading an ios strings file (english -> japanese) to a node server for parsing.

The file is UTF-16LE but when parsed as a string the character encoding loses characters. This may have something to do with express using utf8 to read in the request file data which malforms the file data.

When the file is loaded in atom/sublime w/ utf16 encoding it works great When the file is loaded in utf8 things break down.

Any help would be awesome.

Answer 1

After doing some research and digging.

Utilizing npm module iconv-lite to parse the file buffer one should:

1) parse the buffer as utf16le

2) down convert to utf8

3) conver to a string.

  if (encoding === 'utf-16le') {
    str = iconv.decode(buffer, 'utf16le');
    body = iconv.encode(str, 'utf8').toString();
  } else if (encoding === 'utf-16be') {
    str = iconv.decode(buffer, 'utf16be');
    body = iconv.encode(str, 'utf8').toString();
  } else {
    body = Buffer.concat(file.data).toString();
  }