I created a python script to send about 1.5k to 1.7k metrics every hour to Prometheus using Pushgateway.

Now I wanted to display in Grafana with only unique values.

When I query for last 3 months, it is taking more than 30 minutes to load and it is getting time out but it is showing unique value when I search for 1 day.

I use this PromQl - webex_active_users{Instance="webex_instance", EmailID=~"$Email"}

And I added Group by Transform data on EmailID

Could you please help me to load all unique data in Table based on date range I select in Grafana?

I created a python script to send about 1.5k to 1.7k metrics every hour to Prometheus using Pushgateway.

Now I wanted to display in Grafana with only unique values.

When I query for last 3 months, it is taking more than 30 minutes to load and it is getting time out but it is showing unique value when I search for 1 day.

I use this PromQl - webex_active_users{Instance="webex_instance", EmailID=~"$Email"}

And I added Group by Transform data on EmailID

Could you please help me to load all unique data in Table based on date range I select in Grafana?

• prometheus
• grafana
• promql
• prometheus-pushgateway

• 1 It sounds like you are using wrong tool for the job. Are you aware that "The Pushgateway never fets series pushed to it and will expose them to Prometheus forever"? – markalex Commented Mar 15 at 18:36

1 Answer 1

The following PromQL query should return unique values for the EmailID label on the selected time range in Grafana (the query must run in instant mode ):

count(
  group(
    last_over_time(
      webex_active_users{
        Instance="webex_instance", EmailID=~"$Email"
      }[$__range]
    )
  ) by (EmailID)
)

The query selects all the time series matching the webex_active_users{Instance="webex_instance", EmailID=~"$Email"} series selector on the selected time range $__range, with the help of last_over_time function, then returns an arbitrary time series with the help of group function per every unique EmailID label values, and then counts the number of returned time series per unique EmailID value with the count function.

P.S. It is likely you use incorrect tool (Prometheus) for calculating this type of stats. I'd recommend you reading Prometheus key concepts in order to understand better the data model behind Prometheus. It is likely you hit high cardinality issue because of improper usage of Prometheus. I suppose you need an analytical database such as ClickHouse for efficient storing and querying structured events (aka wide events ) instead of Prometheus.