I am trying to run some some python/other language modules modules/workflows/workloads and collect their CPU, Mem, I/O etc resource usage with Grahite, Carbon, CollectD, StatsD. I have read the documentation (see:here) about creating tagged series but what I can't seem to find how to tag specific moudles. For example, I have two modules
First Module
def firstModule:
# Initialize a list
primes = []
for possiblePrime in range(2, 21):
# Assume number is prime until shown it is not.
isPrime = True
for num in range(2, possiblePrime):
if possiblePrime % num == 0:
isPrime = False
if isPrime:
primes.append(possiblePrime)
Second Module
def secondModule:
# Initialize a list
primes = []
for possiblePrime in range(2, 21):
# Assume number is prime until shown it is not.
isPrime = True
for num in range(2, possiblePrime):
if possiblePrime % num == 0:
isPrime = False
break
if isPrime:
primes.append(possiblePrime)
Here I want to call the two modules and then tag resource usage metrics so that I can send it to the whisper database as shown in code below: How do I achieve that?
firstModule() # assign some tag say A
secondModule() # assign some tag say B
Although I ended up not using a tagged series, I have solved the problem above by writing a bash script and using the render api in graphite. Here is the walk around solution:
#!/bin/bash
#Remove previous records. Be careful not to remove useful files!
rm cpu.json, mem.json
#clear variables for storing time
unset start
unset finish
#log start time of process using absolute time, if using relative time check doc at https://graphite.readthedocs.io/en/latest/render_api.html for rules
start=$(date '+%H:%M_%Y%m%d')
#call first module
firstModule()
#sleep 1 minute or more because you will get an error that start time and end time are equal
sleep 100
#call second module
secondModule()
#log finish date using absolute time, if using relative time check doc at https://graphite.readthedocs.io/en/latest/render_api.html) for rules
finish=$(date '+%H:%M_%Y%m%d')
#collect data using curl here cpu usage and memusage, adjust to fit your setup and Voila!
curl "http://host/render?target=carbon.agents.*.cpuUsage&width=500&height=300&from={$start}&until={$finish}&format=json" > cpu.json
sudo curl "http://host/render?target=collectdlocalhost.memory.memory-used&width=500&height=300&from={$start}&until={$finish}&format=json" > mem.json