Node: Express: How to handle application/octet-stream;charset=;UTF-8 response?

Question

I have a node-express application. There, I'm trying to make a call to an API which is responding an raw xlsx object as

'Content-Type' : 'application/octet-stream;charset=;UTF-8'

Code how I'm calling the API:

var unirest = require("unirest");

var reqClient = unirest("POST", "https://api.application.com/getExcel");
reqClient.headers({ "Authorization": "Bearer " + req.session.passport.user.token, 
"content-type": req.headers['content-type'], "application/json"});
reqClient.type("json");
reqClient.send(JSON.stringify(requestbody));
reqClient.end(function(res) {
if (res.error) throw new Error(res.error);
console.log(res.body);
});

Now 2 things I'm trying to do actually with this data.

1. Write that into an excel file. Below is the code how I'm trying it:

    let data = res.body // res is the response coming from the API
    let buf = Buffer.from(data);
    excelfile = fs.createWriteStream("result.xlsx");
    excelfile.write(buf);
    excelfile.end();

2. Trying to send it to UI where the excelfile will get created. Below is my code for that:

    let data = res.body // res is the response coming from the API
    let buf = Buffer.from(data);
    response.write(buf); //response is the response to the request to ui
    response.end();

So in both the cases the file is coming corrupted.

But the API response is prefect because, when it's directly getting consumed by the UI, xlsx file is generating properly.

Answer 1

When dealing with binary data, you have to set the encoding to null

reqClient.encoding(null)

reqClient.encoding(null)
reqClient.end(function(res) {
    if (res.error) {
       return response.status(500).send('error');
    }

    // res.body is now a buffer
    response.write(res.body);
    response.end();
});

Otherwise, the data is converted to UTF-8 and you can't convert from UTF-8 back to binary and get the same data, which is what you were doing:

Buffer.from(res.body)

The recommended approach is to use streams directly, I don't see a way of doing this in a simple way in unirest, I recommend using request or got, so you can .pipe directly to a file or express res