Customize error message when hitting limits

[friedrichg]

Is your feature request related to a problem? Please describe.

Prometheus/Remote write agents that get throttled will get a typical error message like this in the logs

server returned HTTP status 400 Bad Request: user=xxx: per-user series limit of 150000 exceeded, please contact administrator to raise it (local limit: 0 global limit: 150000 actual local limit: 30000)

Describe the solution you'd like
It should be possible to customize error messages for limits or other problems

Ideally we should make the part "please contact administrator to raise it" customizable. So admin can provide more exact info on where to get help.

Describe alternatives you've considered
None

Additional context
None

[friedrichg]

cortex/pkg/ingester/limiter.go

Line 134 in 2c37922

return fmt.Errorf("per-metric series limit of %d exceeded, please contact administrator to raise it (local limit: %d global limit: %d actual local limit: %d)",

[kumanik]

Hi, I would love to give this issue a go.

[friedrichg]

go ahead @kumanik please

Make it an -ingester flag like

cortex/pkg/ingester/ingester.go

Line 146 in b3cab52

f.StringVar(&cfg.IgnoreSeriesLimitForMetricNames, "ingester.ignore-series-limit-for-metric-names", "", "Comma-separated list of metric names, for which -ingester.max-series-per-metric and -ingester.max-global-series-per-metric limits will be ignored. Does not affect max-series-per-user or max-global-series-per-metric limits.")