I want store web log that using morgan and express to json file

Question

I want store JSON file each like form

[
 {
   "remote-addr" : "127.0.0.1",
   "date"  : " 2018.07.28"
 }
] 

and i use this code

var format=json(
    ':remote-addr:date'
);

app.use(logger({
    format:format,
    stream: fs.createWriteStream('log.json')
}));  

i use this code and get

{"remote-addr":"::ffff:127.0.0.1","date":"Sat, 28 Jul 2018 04:38:41 GMT"}
{"remote-addr":"::ffff:127.0.0.1","date":"Sat, 28 Jul 2018 04:38:41 GMT"}
{"remote-addr":"::ffff:127.0.0.1","date":"Sat, 28 Jul 2018 04:38:42 GMT"}
{"remote-addr":"::ffff:127.0.0.1","date":"Sat, 28 Jul 2018 04:38:48 GMT"}

this is json file but there no [ ] and ,

how to get json file??

Answer 1

Technically Morgan does not allow you to do this, because it's entire purpose is to write one standard access.log line per request (credits to Douglas Wilson for pointing that out).

Yes, you can hack around the single log line, like you did, which makes it a valid JSON line. However, in order to make your log.json file to be a valid JSON as well, the only way I can think of is to implement some kind of post-processing to the file.

Here's how the post-processing will look like: First, you need to read line by line the file. Then, create your valid JSON. Finally - save it in a separate file (or override the log.json anyways).

Here's how I did it. Input: your current log.json file:

{"remote-addr":"::ffff:127.0.0.1","date":"Sat, 28 Jul 2018 04:38:41 GMT"}
{"remote-addr":"::ffff:127.0.0.1","date":"Sat, 28 Jul 2018 04:38:41 GMT"}
{"remote-addr":"::ffff:127.0.0.1","date":"Sat, 28 Jul 2018 04:38:42 GMT"}
{"remote-addr":"::ffff:127.0.0.1","date":"Sat, 28 Jul 2018 04:38:48 GMT"}

The post-processing script I wrote:

const fs = require('fs');
//  Since Node.js v0.12 and as of Node.js v4.0.0, there is a stable
// readline core module. That's the easiest way to read lines from a file,
// without any external modules. Credits: https://stackoverflow.com/a/32599033/1333836
const readline = require('readline');

const lineReader = readline.createInterface({
    input: fs.createReadStream('log.json')
});

const realJSON = [];
lineReader.on('line', function (line) {
    realJSON.push(JSON.parse(line));
});

lineReader.on('close', function () {
    // final-log.json is the post-processed, valid JSON file
    fs.writeFile('final-log.json', JSON.stringify(realJSON), 'utf8', () => {
        console.log('Done!');
    });
});

Result: the final-log.json file, which is a valid JSON (I validated it with jsonlint, all good).

[{
    "remote-addr": "::ffff:127.0.0.1",
    "date": "Sat, 28 Jul 2018 04:38:41 GMT"
}, {
    "remote-addr": "::ffff:127.0.0.1",
    "date": "Sat, 28 Jul 2018 04:38:41 GMT"
}, {
    "remote-addr": "::ffff:127.0.0.1",
    "date": "Sat, 28 Jul 2018 04:38:42 GMT"
}, {
    "remote-addr": "::ffff:127.0.0.1",
    "date": "Sat, 28 Jul 2018 04:38:48 GMT"
}]