Search code examples
javascriptnode.jscronfsnightmare

Not able to get result of Nightmare from cron execution


I've been working on some automation process with Nightmare and cron. After Nightmare runs, it gets the value of the result and appends it to a file in my local computer (no network issues detected).

var Nightmare = require('nightmare');
var fs = require('fs');
var filePath = `${process.env['HOME']}/Documents/ResultCron.txt`;

Nightmare()
    .goto('http://mywebsite.com/form')
    .type('input[id="email"]', '[email protected]')
    .click('#submit-button')
    .wait('.formSubmitted')
    .evaluate(() => document.querySelector('.formSubmitted').value)
    .end()
    .then(result => {
      console.log(`The form submitted is:\n${result}`)
      fs.appendFileSync(filePath, `\n${result}: ${new Date()}`);
    })
    .catch(error => {
      console.error(error)
      fs.appendFileSync(filePath, `\nERROR: ${error}: ${new Date()}`);
    })

When I run this with node /home/user/Documents/nightmareForm.js it's working perfectly and writting in the file. However, when executed from the cron it doesn't append anything to the file.

I've added the config to the crontab as following:

crontab -e

# It will be executed everyday at 13:00
0 13 * * * node /home/user/Documents/nightmareForm.js

And if I do grep nightmareForm.js /var/log/syslog -C 10 I can see it has been executed: Apr 24 13:00:00 PC CRON[1223]: (root) CMD (node /home/user/Documents/nightmareForm.js)

Continuing the investigation, the file has this rights (Ubuntu 18.10): -rwxr-xr-x so it should be executable and the created file has the rights: -rw-rw-rw-

Update: After the ideas of @ponury-kostek I verified that the script can write in the file, but it never reaches the then() or catch() block

Update2: I tried to catch the logs from the crontab but it doesn't display any error: 0 13 * * * /usr/local/bin/node /home/user/Documents/nightmareForm.js >> /home/user/Documents/nightmareForm.log 2>&1

Any ideas why from crontab the script is not able to reach the then() or catch() blocks? Thank you


Solution

  • I had the same Problem.

    what did the job for me was using xvfb-run to start the node process.

    xvfb-run node myscript.js
    

    Actually i made shellscript (go.sh) that only contained 3 lines:

    #!/bin/bash
    cd /home/me/nodestuff/spkfetch
    xvfb-run node fetch_umsaetze.js
    

    And added this to the crontab:

    * * * * *  cd /home/norbert/nodestuff/spkfetch && sh go.sh  >> /home/norbert/nodestuff/spkfetch/log/log.txt  2>&1
    

    I never tried to put the xvfb-run call directly into the crontab, but i think its possibly an issue of the script not being allowed to start an X process(window) or something like that. Maybe its worth to try to put it directly into the cron command.

    Here is the page that led me to this solution:
    https://www.nickhart.co.uk/2019/08/09/scraping-with-nightmarejs-getting-started/#run_on_server