Search code examples
javascriptnode.jsexpressfswinston

Is fs.watch() broken in Node v13 or am I doing something wrong?


I've been stuck trying to set up a file watcher that detects when new data is added to a file. The general sequence is 1.) a client connects to server. 2.) server logs to a file. 3.) fs.watch() detects the file has changed and runs a function, in this case a simple console.log('New log entry')

Everything seems to work except fs.watch() does not detect when new messages are added to the log file. However if I click on the log file in VScode it seems to trigger it. Is this a bug in newer versions of Node or am I doing something wrong here?

I realize I could use fs.watchFile() but I was hoping to avoid the overhead of polling...

// src/index.js
const path = require('path');
const express = require('express');
const app = express();
const server = require('http').createServer(app);
const io = require('socket.io')(server);
const logger = require('./logger');
const fs = require('fs');

fs.watch('./logs/combined.log', (event) => {
  if (event === 'change') {
    console.log('New log entry');
  }
});

app.use(express.static(path.join(__dirname, '../public')));
app.get('/', function(req, res, next) {
  res.sendFile(path.join(__dirname, '../public', 'index.html'));
});

io.on('connection', function(socket) {
  logger.info('a user connected');
});

const PORT = process.env.PORT || 8888;

server.listen(PORT, () => {
  logger.info(`Listening at http://localhost:${port}`);
});

-

// src/logger.js
const winston = require('winston');

const logger = winston.createLogger({
  level: 'info',
  format: winston.format.json(),
  transports: [
    new winston.transports.File({ filename: 'logs/combined.log' })
  ]
});

module.exports = logger

-

// public/index.html
<script src="/socket.io/socket.io.js"></script>
<script>
  var socket = io.connect('http://localhost:8888');
</script>

-

// simplified test
const logger = require('./logger');
const fs = require('fs');

fs.watch('./logs/combined.log', event => {
  if (event === 'change') {
    console.log('log file has updated');
  }
});

function intervalFunc() {
  logger.info('new log message');
}
setInterval(intervalFunc, 5000);

Solution

  • Update:

    So it looks like Winston comes with built-in listeners. These listeners act similar to a file watcher, and allow you to hook into certain events. One of the events you can hook into is called 'data' - this lets you listen for when data is written to file, and run a callback function.

    This provides similar behavior to file.watch.

    They allow you to listen for the following events:

    • close
    • data
    • end
    • error
    • readable

    I tested this on Windows and it works perfectly.

    This is the test file I am using:

    const winston = require('winston');
    
    const { createLogger, format: { json }, transports: { File } } = winston;
    
    const logger = createLogger({
      level: 'info',
      format: json(),
      transports: [
        new File({ filename: './log.txt' })
      ],  
    });
    
    /**
     * Added a built in Winston listener
     */
    logger.addListener('data', chunk => {
        console.log('\r\n[winston listener] we have logged some data:\n', chunk)
    })
    
    /**
     * Their documentation seems to use the `.on` method.
     * 
     * This appears to provide the same type of behavior that `.addListener` does.
     * 
     * I'm not sure which is best, or recommended, `.addListener` or `.on` - you
     * may want to dig deeper into that.
     */
    logger.on('data', chunk => {
        console.log('\r\n[winston on "data"] we have logged some data:\n', chunk);
    });
    
    setInterval(() => {
      logger.info('new log message');
    }, 5000);
    

    so I've been giving this some thought and at first I believed this was happening because Winston uses write streams to update files..

    Then I went ahead and tried to replicate your issue, but I cannot reproduce it.

    Each time the log file gets updated, a message is written to console (meaning, fs.watch is working.

    Are you using Windows? What version of Node are you on?

    This is the test file I am using:

    const winston = require('winston');
    const fs = require('fs');
    
    const { createLogger, format: { json }, transports: { File } } = winston;
    
    const logger = createLogger({
      level: 'info',
      format: json(),
      transports: [
        new File({ filename: './log.txt' })
      ]
    });
    
    fs.watch('./log.txt', event => { 
      if (event === 'change') {
        console.log('log file has updated');
      }
    });
    
    setInterval(() => {
      logger.info('new log message');
    }, 5000);