Search code examples
datastaxsystemd

Datastax & systemd

I have configured a systemd Unit for DataStax Enterprise 4.8.5:

### /etc/systemd/system/dse1.service

[Unit]
Description=DataStax Enterprise

[Service]
User=cassandra
ExecStart=/opt/dse/dse1/bin/dse cassandra -k
ExecStop=/opt/dse/dse1/bin/dse cassandra-stop

when I execute sudo systemctl start dse1, if i immediately do a status afterwards, i get:

● dse1.service - DataStax Enterprise 1
   Loaded: loaded (/etc/systemd/system/dse1.service; static; vendor preset: disabled)
   Active: active (running) since Wed 2016-03-23 13:47:57 EDT; 1s ago
 Main PID: 31699 (cassandra)
   CGroup: /system.slice/dse1.service
           ├─31699 /bin/sh /opt/dse/dse1/resources/cassandra/bin/cassandra -k -Djava.library.path=:/opt/dse/dse1/resources/hadoop/native...
           ├─31894 /bin/java -cp :/opt/dse/dse1/lib/dse-core-4.8.5.jar:/opt/dse/dse1/lib/dse-hadoop-4.8.5.jar:/opt/dse/dse1/lib/dse-hive...
           └─31895 grep -q Error: Exception thrown by the agent : java.lang.NullPointerException

If I then wait a few seconds and try again, I get:

● dse1.service - DataStax Enterprise 1
   Loaded: loaded (/etc/systemd/system/dse1.service; static; vendor preset: disabled)
   Active: inactive (dead)

Mar 23 13:34:28 pspldsea01p.fleet.ad systemd[1]: Started DataStax Enterprise 1.
Mar 23 13:34:28 pspldsea01p.fleet.ad systemd[1]: Starting DataStax Enterprise 1...
Mar 23 13:38:33 pspldsea01p.fleet.ad systemd[1]: Started DataStax Enterprise 1.
Mar 23 13:38:33 pspldsea01p.fleet.ad systemd[1]: Starting DataStax Enterprise 1...
Mar 23 13:47:41 pspldsea01p.fleet.ad systemd[1]: Started DataStax Enterprise 1.
Mar 23 13:47:41 pspldsea01p.fleet.ad systemd[1]: Starting DataStax Enterprise 1...
Mar 23 13:47:44 pspldsea01p.fleet.ad dse[31267]: nodetool: Failed to connect to '127.0.0.1:7199' - ConnectException: 'Connection refused'.
Mar 23 13:47:57 pspldsea01p.fleet.ad systemd[1]: Started DataStax Enterprise 1.
Mar 23 13:47:57 pspldsea01p.fleet.ad systemd[1]: Starting DataStax Enterprise 1...
Mar 23 13:48:01 pspldsea01p.fleet.ad dse[32004]: nodetool: Failed to connect to '127.0.0.1:7199' - ConnectException: 'Connection refused'.
Hint: Some lines were ellipsized, use -l to show in full.

If I just execute /opt/dse/dse1/bin/dse cassandra -k as the cassandra user, it works fine.

I can't seem to find any additional logging in the normal logging locations or with sudo journalctl -u dse1

Any ideas? Thanks!

Solution

  • It is unfortunate that DataStax Enterprise doesn't come with a systemd service file to be able to use systemctl. However, it does come with an init script. Full documentation is available at the docs

    Basically you have two options. The first one is to use the init.d directly, by starting the service:

    sudo service dse start

    I'm , however, too used to systemctl now to go back to that. So this is my systemd service file

    [Unit]
Description=DataStax Enterprise
After=network.target

[Service]
PIDFile=/var/run/dse/dse.pid
ExecStart=/etc/init.d/dse start
ExecStop=/etc/init.d/dse stop
SuccessExitStatus=143
TimeoutSec=300

[Install]
WantedBy=multi-user.target

    The init script has many configuration options. For the sake of simplicity, it may be wise to just use those directly in the script. For example, you specify the user in your systemd service file. That was giving me problems until I noticed that the user is already specified in the script. No need to duplicate options.

    The SucessExitStatus=143 option is a common configuration for Java applications.

    You may have to adapt the location of the script if you didn't install DSE with your package manager