Unable to drop distinct matching metrics ?

[RJNY]

I expected this to work:

      - match: "airflow_local(.+)"
        match_type: regex
        action: drop
        name: "dropped"

      - match: "(.+)_task_mem_usage_(.*)"
        match_type: regex
        action: drop
        name: "dropped"

      - match: "(.+)_task_cpu_usage_(.*)"
        match_type: regex
        action: drop
        name: "dropped"

But only this works:

      - match: "."
        match_type: regex
        action: drop
        name: "dropped"

Is dropping on a single-match basis not possible? I've only been able to get the catch-all to work.

[pedro-stanaka]

wrote a small unit test in exporter_test.go and it passes, so it should be working.

did you configure a catchall for metrics to be kept at the end of the config?

func TestDropSpecificMetric(t *testing.T) {
	config := `
mappings:
- match: metric(.*)drop
  action: drop
  match_type: regex
  name: "dropped_metric"

- match: metric.to.keep
  name: "kept_metric"
`
	testMapper := &mapper.MetricMapper{}
	err := testMapper.InitFromYAMLString(config)
	if err != nil {
		t.Fatalf("Config load error: %s %s", config, err)
	}

	events := make(chan event.Events)
	defer close(events)

	// It's important to use a new registry for this test to avoid interference from other tests.
	reg := prometheus.NewRegistry()
	exporterEventsActions := prometheus.NewCounterVec(
		prometheus.CounterOpts{
			Name: "statsd_exporter_events_actions_total",
			Help: "The total number of StatsD events by action.",
		},
		[]string{"action"},
	)
	reg.MustRegister(exporterEventsActions)

	go func() {
		ex := NewExporter(reg, testMapper, promslog.NewNopLogger(), exporterEventsActions, eventsUnmapped, errorEventStats, eventStats, conflictingEventStats, metricsCount)
		ex.Listen(events)
	}()

	ev := event.Events{
		&event.CounterEvent{
			CMetricName: "metric.to.drop",
			CValue:      1,
		},
		&event.CounterEvent{
			CMetricName: "metric.to.keep",
			CValue:      1,
		},
	}

	events <- ev
	events <- event.Events{} // Send empty event to ensure prior events are processed

	metrics, err := reg.Gather()
	if err != nil {
		t.Fatalf("Cannot gather from registry: %v", err)
	}

	droppedMetricValue := getFloat64(metrics, "dropped_metric", prometheus.Labels{})
	if droppedMetricValue != nil {
		t.Errorf("Metric 'dropped_metric' was found, but it should have been dropped. Value: %f", *droppedMetricValue)
	}

	keptMetricValue := getFloat64(metrics, "kept_metric", prometheus.Labels{})
	if keptMetricValue == nil {
		t.Errorf("Metric 'kept_metric' was not found, but it should have been present.")
	} else if *keptMetricValue != 1 {
		t.Errorf("Metric 'kept_metric' has value %f, expected 1", *keptMetricValue)
	}

	// Check that the drop action was recorded
	var dropActionMetric dto.Metric
	err = exporterEventsActions.WithLabelValues("drop").Write(&dropActionMetric)
	if err != nil {
		t.Fatalf("Error writing drop action metric: %v", err)
	}
	if dropActionMetric.Counter.GetValue() != 1 {
		t.Errorf("Expected drop action count to be 1, got %f", dropActionMetric.Counter.GetValue())
	}
}

[RJNY]

Strange.

These are the metrics I was trying to catch:

airflow_local_task_job_task_exit_2563685_hello_world_print_hello_1
airflow_task_mem_usage_hello_world_print_hello_100
airflow_task_cpu_usage_hello_world_print_hello_100

and this is was my initial remap that didn't work.

statsd:
  mappingConfig: |-
    mappings:
      {re-mappings}
      ...
      
      - match: "airflow_local(.+)"
        match_type: regex
        action: drop
        name: "dropped"

      - match: "(.+)_task_mem_usage_(.*)"
        match_type: regex
        action: drop
        name: "dropped"

      - match: "(.+)_task_cpu_usage_(.*)"
        match_type: regex
        action: drop
        name: "dropped"

did you configure a catchall for metrics to be kept at the end of the config?

Not at first no. I currently have only a catch all in place right now, because we're generating so many metrics that our collector is throwing "body too large" errors.

Alternatively (while off topic of the issue) I would have liked to have remapped only those metrics
airflow_local_task_job_task_exit_2563685_hello_world_print_hello_1 would become task_exit{dag_id: hello_world, task_id: print_hello} 1, but because they're all underscored, it's difficult to ensure I'm mapping it correctly, so I thought it best to drop it.