How to combine Grafana/Prometheus series after relabeling

I have Prometheus configured to scrape metrics from an ec2 instance. At some point I wrote an extra relabeling rule that populates the new ec2_vpc_id value.

- job_name: ec2
  scrape_interval: 15s
  scrape_timeout: 10s
  metrics_path: /metrics

  relabel_configs:
  - source_labels: [__meta_ec2_vpc_id]     <-- newly added
    separator: ;
    regex: (.*)
    target_label: ec2_vpc_id
    replacement: $1
    action: replace
  ec2_sd_configs:
  - endpoint: ""
    region: us-west-1
    port: 1234

My graph in Grafana now shows 2 distinct series before and after the relabeling.

node_load1{instance="$node",job="$job",ec2_vpc_id=~".*"}

The yellow series is my old metric without ec2_vpc_id. The green series is the new metric with ec2_vpc_id. Both of them are of the same host.

It looks like Grafana automatically separates them intro 2 distinct series. How do I combine them into one?

Solution

Prometheus is the one who makes distinction between this series.

Prometheus considers two timeseries different if they have different label set.

To avoid this distinction you need to use query, that will remove label ec2_vpc_id from consideration.

For example, this can be done with

sum without(ec2_vpc_id) (node_load1{instance="$node",job="$job"})