How to send Metrics via Scrapy to Prometheus?

Question

I'm new to Prometheus.

I need too send the numbers of scraped items to Prometheus and show them on a graph in Grafana.

I installed Prometheus, Scrapy-Prometheus, and Grafana .

In Scrapy setting I add

STATS_CLASS = 'scrapy_prometheus.PrometheusStatsCollector'
# Prometheus pushgateway host
PROMETHEUS_PUSHGATEWAY = 'http://0.0.0.0:9090'   
# Metric name prefix
PROMETHEUS_METRIC_PREFIX = 'scrapy_prometheus'   
# Timeout for pushing metrics to pushgateway
PROMETHEUS_PUSH_TIMEOUT = 5  
# Method to use when pushing metrics
PROMETHEUS_PUSH_METHOD = 'POST'  # default
PROMETHEUS_SUPPRESS_TYPE_CHECK = False
# job label value, applied to all metrics.
PROMETHEUS_JOB = 'scrapy' 
PROMETHEUS_GROUPING_KEY = {'instance': 'localhost'}

I configurate the setting of Grafana like this

In Documentation we have

stat foo: 67 whill produce metric

scrapy_prometheus_foo{instance="...",job="scrapy",spider="..."} 67

My question is how can I send the number of scraped items as metrics to Prometheus?

Answer 1

From reading the docs it seems that scrapy-prometheus just sends all stats in:

<prefix><stats name with / replaced by _>

Default prefix is scrapy_prometheus and the stat you are looking for items scraped is item_scraped_count, so you should be looking for scrapy_prometheus_item_scraped_count stat and it should be sent by default.