Search code examples
apache-sparkdatastax-enterprisespark-jobserver

Schedule Automatic Spark Jobs in Spark Job server every hour


In DataStax Enterprise Edition 4.8 , Spark Jobserver 0.5.2 has been specially compiled against the supported version of Apache Spark 1.4.1.1. Spark job will read data from Cassandra and write summarized data into another table in same Keyspace.

Is there any feature in Spark Job server to schedule jobs to run automatically after every hour?


Solution

  • Unfortunately Spark Jobserver doesn't have any feature to schedule jobs right now.

    Maybe you could use a cron task on linux to automatically run the curl command to send a request to Jobserver.