Search code examples
javascriptline-endings

Converting csv to json adds '\r' characters


I have a method that converts csv to json data:

Here is the code:

    CSV2JSON(): void {
      const lines = this.csvData.split('\n');
      const result = [];
      const headers = lines[0].split(',');
      console.log(headers);
      for (let i = 1; i < lines.length - 1; i++ ) {
       const obj = {};
       const currentline = lines[i].split(',');

       for (let j = 0; j < headers.length; j++) {
        obj[headers[j]] = currentline[j];
       }

      result.push(obj);

   }

    console.log(JSON.stringify(result)); // JSON
}

This is giving me:

[{"name":"test1","code z\r":"0.030\r"}] 

As you can see there's '\r' which shouldn't be there.

How can I get rid of those \r or stop them appearing in the json data?


Solution

  • Your problem is with line-endings, not Angular. Different operating systems have different standards on how to encode the "new line" character; this is a common annoying issue when parsing human-readable strings (such as the CSV format). You read about this in detail on Wikipedia, for example.

    The \r you're getting probably means that the original file had \n\r for line endings. When you do a split at \n, the right-hand part will retain the leading \r. To remove the \r as well, you'll have to split by \n\r instead, treating the whole group as a new line.

    const s = `foo\n\rbar`
    console.log(s.split('\n'))    // bad
    console.log(s.split('\n\r'))  // good

    But if you do this blindly, you're going to miss the cases when a new line is simply \n.

    const s = `foo\nbar`
    console.log(s.split('\n'))    // good
    console.log(s.split('\n\r'))  // bad

    To handle both, you could firstly replace all instance of \n\r with just \n, and then split.

    const s = `foo\n\rbar\nbaz`
    console.log(s.replace(/\n\r/g, '\n').split('\n'))

    However, with this code, you're going to miss other operating systems which use only \r for the new line. You can either handle this case as well, or use an existing library for parsing CSV. There's probably a few to choose from on the NPM repository.