Search code examples
pythonuser-defined-functionsinfluxdbkapacitor

Python cron job vs Kapacitor UDF

I need to perform some background advanced calculations on my data after it is collected in InfluxDb which is stored on the edge server, which means I have limited resources for the calculations. Also I cannot block the data collection while I do calculations. I am weighing using Kapacitor UDF streams vs custom Python scripts.

Please note I need to make the scripts configurable so that I can easily deply them to different environments with different sensors

Solution

  • It probably makes little difference, in general, especially for 'simple' usecases, though I lean towards standalone python scripts. (It may be better to use Kapacitor if you can cover your usecase using the kapacitor language for this instead of python based UDFs, but I found it insufficient since I needed to retrieve additional data from other databases)

    • Standalone python scripts could be a bit lighter, since you don't need to run the Kapacitor service.
    • Standalone python scripts could be a bit more configurable. Kapacitor is also pretty configurable but you'd have to spend a bit of time learning how to use it.
    • Standalone python scripts could be a bit more stable. I've experimented with python UDF's a couple of years ago and found them unstable and buggy. While this may have improved by now, you'd still be relying on this being supported and if go is not your language you might have trouble debugging and fixing issues yourself.