Skip to content

Default instance limits #33

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
merged 1 commit into from
Aug 25, 2023
Merged

Default instance limits #33

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
1 commit
Select commit
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
2 changes: 2 additions & 0 deletions CHANGELOG.md
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,6 +1,8 @@
# Changelog

## master / unreleased
* [CHANGE] Add default instance max series for ingesters
* [CHANGE] Add default instance max inflight pushes for distributors
* [CHANGE] Remove mem-ballast from distributor and querier.
* [CHANGE] Increase cpu requests for querier to 2.
* [CHANGE] Configure default GOMAXPROCS and GOMEMLIMIT for all cortex modules
Expand Down
13 changes: 6 additions & 7 deletions cortex/config.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -395,13 +395,12 @@
ingester_stream_chunks_when_using_blocks: true,

// Ingester limits are put directly into runtime config, if not null. Available limits:
// ingester_instance_limits: {
// max_inflight_push_requests: 0, // Max inflight push requests per ingester. 0 = no limit.
// max_ingestion_rate: 0, // Max ingestion rate (samples/second) per ingester. 0 = no limit.
// max_series: 0, // Max number of series per ingester. 0 = no limit.
// max_tenants: 0, // Max number of tenants per ingester. 0 = no limit.
// },
ingester_instance_limits: null,
ingester_instance_limits: {
// max_inflight_push_requests: 0, // Max inflight push requests per ingester. 0 = no limit.
// max_ingestion_rate: 0, // Max ingestion rate (samples/second) per ingester. 0 = no limit.
max_series: 4.8e+6, // Max number of series per ingester. 0 = no limit. 4.8 million is closely tied to 15Gb in requests per ingester
// max_tenants: 0, // Max number of tenants per ingester. 0 = no limit.
},
},

local configMap = $.core.v1.configMap,
Expand Down
1 change: 1 addition & 0 deletions cortex/distributor.libsonnet
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,6 +31,7 @@
// Do not extend the replication set on unhealthy (or LEAVING) ingester when "unregister on shutdown"
// is set to false.
'distributor.extend-writes': $._config.unregister_ingesters_on_shutdown,
'distributor.instance-limits.max-inflight-push-requests': 60, //60 is very conservative to protect the distributor from OOMs
},

distributor_env_map:: {
Expand Down