Search code examples
javascriptnode.jsfs

Fast method to write in a TXT file?


I made two scripts to make a file and write in it. The Problem is both are slow (based on Disk Speed) Is there a better Method to write to a file?

let codes = [];
function makeid(length) {
   var result           = '';
   var characters       = 'ABCDEFGHIJKLMNOPQRSTUVWXYZabcdefghijklmnopqrstuvwxyz0123456789';
   var charactersLength = characters.length;
   for ( var i = 0; i < length; i++ ) {
      result += characters.charAt(Math.floor(Math.random() * charactersLength));
   }
   return result;

};

for (let i = 0; 1000000 > i; i++)
{
   codes.push(`${makeid(20)}`);
}
fs.writeFile(`./codes/${makeid(5)}.txt`, codes.join('\n'), (err) => {
   if (err)
   {
       message.reply("There was an error processing your request");
       throw err;
   }
}) 
/* Second script */
let stream = fs.createWriteStream(`./codes/${makeid(5)}.txt`);
for (let i = 0; 1000000 > i; i++)
{
   stream.write(`${makeid(20)}`);
}
stream.end();

Both Codes take 5-15 seconds. I'm looking for a faster method to make it in < 5 seconds. The file is saved in a txt. Thanks!


Solution

  • Buffer

    Instead of creating an array of strings that you then join, you can create Buffer. This is faster because Node.js does not do any conversion, but just writes the bytes in the buffer to the disk.

    // improved first method
    
    const codes = Buffer.allocUnsafe(1000000 * 20);
    
    for (let i = 0; 1000000 > i; i++) {
      codes.write(`${makeid(20)}`, i * 20, 20);
    }
    
    fs.writeFileSync(`./${makeid(5)}.txt`, codes);
    

    On my machine this takes 970.012ms instead of 1907.401ms it took before which is 50% less.