Search code examples
google-cloud-platformmetrics

get "logging.googleapis.com/billing/monthly_bytes_ingested" project info by gcloud or python


I'd like to reach the following information via cli w/ gcloud or a script (bash/python/javascript).

logging.googleapis.com/billing/monthly_bytes_ingested

which can be found in the console under Monitoring > Metrics Explorer > Logs-Based Metric > Monthly Log bytes ingested (pictured below)

While any answer would be great, I'd also like to know what you searched to find it. I've read over docs for most of the day in both monitoring and logs and I don't seem to find clear programmatic approach to this data. The closest I found was in the API here: https://cloud.google.com/monitoring/api/ref_v3/rest/v3/projects.timeSeries/list

enter image description here


Solution

  • This answer extends the original question and answers it. Following the directions above, I use jq to sort the top value. So I get just the sum of the monthly log ingestion.

    Why? It seems to be what I want when calling a metric called monthly_bytes_ingested is just the aggregated number. I do realize that this metric is under the timeseries area and so it is consistent with the timeseries notion to give rolling results. But, when I want a monthly total, do I want rolling or simply the greatest amount?

    referenced docs

    query.json

    { "query": "fetch global
    | metric 'logging.googleapis.com/billing/monthly_bytes_ingested'
    | group_by 3h,
        [value_monthly_bytes_ingested_mean: mean(value.monthly_bytes_ingested)]
    | every 3h
    | group_by [],
        [value_monthly_bytes_ingested_mean_aggregate:
           aggregate(value_monthly_bytes_ingested_mean)] | within 4w" }
    

    I worked out the mql in the console

    curl + jq

    curl -d @query.json -H "Authorization: Bearer $TOKEN" \                                                                                                 [14:38:23]
    --header "Content-Type: application/json" -X POST \
    https://monitoring.googleapis.com/v3/projects/${GCP_PROJECT}/timeSeries:query | jq '.timeSeriesData[].pointData | sort_by(.doubleValue) | .[-1]'
    

    sample result

    {
      "values": [
        {
          "doubleValue": 39625036610082.664
        }
      ],
      "timeInterval": {
        "startTime": "2022-03-23T23:38:23.771566Z",
        "endTime": "2022-03-23T23:38:23.771566Z"
      }
    }