Search code examples
web-scrapingtimersystemdnixos

Run a script with systemd timers on Nixos


I have a small shellscript scrape.sh that scrapes a website and puts the resulting data into a new directory:

website='test.com'
dir_name="datev_data/$(date -u +'%Y-%m-%dT%H:%M:%S')"
mkdir $dirname
wget --directory-prefix="$dir_name" "$website"

(I don't realy care where the data ends up as long as it is gets a new directory every time and I can get to the data. I therefore put it into my home directory /home/kaligule for now.)

Running this script by hand works fine, so now I want to run this script every hour on my nixos-server. Therefore I put the following into my config there (Inspired by this post):

    systemd.services.test_systemd_timers = {
      serviceConfig.Type = "oneshot";
      script = ''
        echo "Will start scraping now."
        {pkgs.bash}/bin/bash /home/kaligule/scrape.sh
        echo "Done scraping."
      '';
    };

    systemd.timers.test_systemd_timers = {
      wantedBy = [ "timers.target" ];
      partOf = [ "test_systemd_timers.service" ];
      timerConfig.OnCalendar = [ "*-*-* *:00:00" ];
    };

Then I test it out:

sudo nixos-rebuild switch # everything is fine
sudo systemctl start test_systemd_timers # run it immediatelly for testing

I get:

Job for test_systemd_timers.service failed because the control process exited with error code.
See "systemctl status test_systemd_timers.service" and "journalctl -xe" for details.

The first suggested command gives me this:

● test_systemd_timers.service
     Loaded: loaded (/nix/store/f8348svxpnn6qx08adrv5s7ksc2zy1sk-unit-test_systemd_timers.service/test_systemd_timers.service; linked; vendor preset: enabled)
     Active: failed (Result: exit-code) since Fri 2021-04-02 14:50:02 CEST; 2min 36s ago
TriggeredBy: ● test_systemd_timers.timer
    Process: 5686 ExecStart=/nix/store/4smyxxxlhnnmw8l6l3nnfjyvmg0wxcwh-unit-script-test_systemd_timers-start/bin/test_systemd_timers-start (code=exited, status=127)
   Main PID: 5686 (code=exited, status=127)
         IP: 0B in, 0B out
        CPU: 11ms

Apr 02 14:50:02 regulus systemd[1]: Starting test_systemd_timers.service...
Apr 02 14:50:02 regulus test_systemd_timers-start[5686]: Will start scraping now.
Apr 02 14:50:02 regulus test_systemd_timers-start[5687]: /nix/store/4smyxxxlhnnmw8l6l3nnfjyvmg0wxcwh-unit-script-test_systemd_timers-start/bin/test_systemd_timers-start: line 3: {pkgs.bash}/bin/bash: No such file or directory
Apr 02 14:50:02 regulus systemd[1]: test_systemd_timers.service: Main process exited, code=exited, status=127/n/a
Apr 02 14:50:02 regulus systemd[1]: test_systemd_timers.service: Failed with result 'exit-code'.
Apr 02 14:50:02 regulus systemd[1]: Failed to start test_systemd_timers.service.

The second suggested command gives me:

Apr 02 14:54:42 regulus systemd[1]: Starting test_systemd_timers.service...
░░ Subject: A start job for unit test_systemd_timers.service has begun execution
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit test_systemd_timers.service has begun execution.
░░ 
░░ The job identifier is 34454.
Apr 02 14:54:42 regulus test_systemd_timers-start[5734]: Will start scraping now.
Apr 02 14:54:42 regulus test_systemd_timers-start[5735]: /nix/store/4smyxxxlhnnmw8l6l3nnfjyvmg0wxcwh-unit-script-test_systemd_timers-start/bin/test_systemd_timers-start: line 3: {pkgs.bash}/bin/bash: No such file or directory
Apr 02 14:54:42 regulus systemd[1]: test_systemd_timers.service: Main process exited, code=exited, status=127/n/a
░░ Subject: Unit process exited
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ An ExecStart= process belonging to unit test_systemd_timers.service has exited.
░░ 
░░ The process' exit code is 'exited' and its exit status is 127.
Apr 02 14:54:42 regulus systemd[1]: test_systemd_timers.service: Failed with result 'exit-code'.
░░ Subject: Unit failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ The unit test_systemd_timers.service has entered the 'failed' state with result 'exit-code'.
Apr 02 14:54:42 regulus systemd[1]: Failed to start test_systemd_timers.service.
░░ Subject: A start job for unit test_systemd_timers.service has failed
░░ Defined-By: systemd
░░ Support: https://lists.freedesktop.org/mailman/listinfo/systemd-devel
░░ 
░░ A start job for unit test_systemd_timers.service has finished with a failure.
░░ 
░░ The job identifier is 34454 and the job result is failed.
Apr 02 14:54:42 regulus sudo[5731]: pam_unix(sudo:session): session closed for user root

So because I got the Will start scraping now. in the logs I think the job started but wasn't able to run the script. My questions are:

  • where (and with which permissions) should I put my script?
  • how should I addapt my config so that the script runs as I described?

(Of course I am thankfull for every feedback to my approach.)


Solution

  • Your problem is that, in your script, {pkgs.bash} should be ${pkgs.bash}; without the $, you won't get variable interpolation.