Skip to content

Commit 731c145

Browse files
jmcarpnschad
jmcarp
and
nschad
authored
Optionally pass cortex config as configmap. (#280)
* Optionally pass cortex config as configmap. Fixes #235. Signed-off-by: Josh Carp <[email protected]> * rework format for guide section Signed-off-by: ShuzZzle <[email protected]> * fix config volume name Signed-off-by: ShuzZzle <[email protected]> Co-authored-by: ShuzZzle <[email protected]>
1 parent 69f8dfb commit 731c145

20 files changed

+294
-145
lines changed

‎CHANGELOG.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,6 +1,7 @@
11
# Changelog
22

33
## master / unreleased
4+
* [FEATURE] Optionally manage cortex config as configmap. #280
45

56
## 1.1.0 / 2021-12-01
67

‎README.md

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -754,5 +754,6 @@ Kubernetes: `^1.19.0-0`
754754
| table_manager.&ZeroWidthSpace;terminationGracePeriodSeconds | int | `180` | |
755755
| table_manager.&ZeroWidthSpace;tolerations | list | `[]` | |
756756
| tags.&ZeroWidthSpace;blocks-storage-memcached | bool | `false` | Set to true to enable block storage memcached caching |
757+
| useConfigMap | bool | `false` | |
757758
| useExternalConfig | bool | `false` | |
758759

‎ci/test-configmap-values.yaml

Lines changed: 206 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,206 @@
1+
ingress:
2+
enabled: true
3+
4+
useConfigMap: true
5+
6+
config:
7+
ingester:
8+
lifecycler:
9+
join_after: 30s
10+
final_sleep: 30s
11+
tokens_file_path: "/data/tokens"
12+
ring:
13+
replication_factor: 1
14+
kvstore:
15+
store: "memberlist"
16+
17+
# See https://github.com/cortexproject/cortex/blob/master/docs/configuration/config-file-reference.md#storage_config
18+
storage:
19+
engine: blocks
20+
blocks_storage:
21+
backend: "filesystem"
22+
tsdb:
23+
dir: "/data/tsdb"
24+
bucket_store:
25+
sync_dir: "/data/tsdb-sync"
26+
filesystem:
27+
dir: "/data/store"
28+
29+
distributor:
30+
ring:
31+
kvstore:
32+
store: memberlist
33+
34+
ruler:
35+
ring:
36+
kvstore:
37+
store: memberlist
38+
39+
ruler_storage:
40+
backend: "filesystem"
41+
filesystem:
42+
dir: "/data/store"
43+
44+
alertmanager_storage:
45+
backend: "filesystem"
46+
filesystem:
47+
dir: "/data/store"
48+
49+
memberlist:
50+
join_members:
51+
- '{{ include "cortex.fullname" $ }}-memberlist'
52+
53+
ingester:
54+
replicas: 1
55+
autoscaling:
56+
enabled: true
57+
minReplicas: 1
58+
statefulSet:
59+
enabled: false
60+
extraVolumes:
61+
- name: tmp-test
62+
emptyDir: {}
63+
extraVolumeMounts:
64+
- name: tmp-test
65+
mountPath: /tmp-test
66+
compactor:
67+
replicas: 1
68+
extraVolumes:
69+
- name: tmp-test
70+
emptyDir: {}
71+
extraVolumeMounts:
72+
- name: tmp-test
73+
mountPath: /tmp-test
74+
distributor:
75+
replicas: 1
76+
autoscaling:
77+
enabled: true
78+
minReplicas: 1
79+
extraVolumes:
80+
- name: tmp-test
81+
emptyDir: {}
82+
extraVolumeMounts:
83+
- name: tmp-test
84+
mountPath: /tmp-test
85+
querier:
86+
replicas: 1
87+
autoscaling:
88+
enabled: true
89+
extraVolumes:
90+
- name: tmp-test
91+
emptyDir: {}
92+
extraVolumeMounts:
93+
- name: tmp-test
94+
mountPath: /tmp-test
95+
query_frontend:
96+
replicas: 1
97+
extraVolumes:
98+
- name: tmp-test
99+
emptyDir: {}
100+
extraVolumeMounts:
101+
- name: tmp-test
102+
mountPath: /tmp-test
103+
nginx:
104+
replicas: 1
105+
autoscaling:
106+
enabled: true
107+
extraVolumes:
108+
- name: tmp-test
109+
emptyDir: {}
110+
extraVolumeMounts:
111+
- name: tmp-test
112+
mountPath: /tmp-test
113+
config:
114+
httpSnippet: |-
115+
# http snippet
116+
mainSnippet: |-
117+
# main snippet
118+
serverSnippet: |-
119+
# server snippet
120+
runtimeconfigmap:
121+
annotations:
122+
foo: bar
123+
alertmanager:
124+
replicas: 1
125+
statefulSet:
126+
enabled: false
127+
extraVolumes:
128+
- name: tmp-test
129+
emptyDir: {}
130+
extraVolumeMounts:
131+
- name: tmp-test
132+
mountPath: /tmp-test
133+
ruler:
134+
replicas: 1
135+
enabled: true
136+
extraVolumes:
137+
- name: tmp-test
138+
emptyDir: {}
139+
extraVolumeMounts:
140+
- name: tmp-test
141+
mountPath: /tmp-test
142+
store_gateway:
143+
replicas: 1
144+
extraVolumes:
145+
- name: tmp-test
146+
emptyDir: {}
147+
extraVolumeMounts:
148+
- name: tmp-test
149+
mountPath: /tmp-test
150+
151+
tags:
152+
blocks-storage-memcached: true
153+
memcached:
154+
enabled: true
155+
architecture: standalone
156+
replicaCount: 1
157+
resources:
158+
# https://github.com/helm/helm/issues/9136
159+
requests:
160+
cpu: 0
161+
memory: 0
162+
memcached-index-read:
163+
enabled: true
164+
architecture: standalone
165+
replicaCount: 1
166+
resources:
167+
requests:
168+
cpu: 0
169+
memory: 0
170+
memcached-index-write:
171+
enabled: true
172+
architecture: standalone
173+
replicaCount: 1
174+
resources:
175+
requests:
176+
cpu: 0
177+
memory: 0
178+
memcached-frontend:
179+
enabled: true
180+
architecture: standalone
181+
replicaCount: 1
182+
resources:
183+
requests:
184+
cpu: 0
185+
memory: 0
186+
memcached-blocks:
187+
architecture: standalone
188+
replicaCount: 1
189+
resources:
190+
requests:
191+
cpu: 0
192+
memory: 0
193+
memcached-blocks-index:
194+
architecture: standalone
195+
replicaCount: 1
196+
resources:
197+
requests:
198+
cpu: 0
199+
memory: 0
200+
memcached-blocks-metadata:
201+
architecture: standalone
202+
replicaCount: 1
203+
resources:
204+
requests:
205+
cpu: 0
206+
memory: 0

‎docs/guides/configure_configmap.markdown

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,18 @@
1+
---
2+
layout: page
3+
title: Configure with configmap
4+
parent: Guides
5+
has_children: true
6+
has_toc: false
7+
---
8+
# Configuring using configmap
9+
10+
By default, cortex configuration is managed using a *Secret*. To use a *ConfigMap* instead, set **useConfigMap**:
11+
12+
{% raw %}
13+
```yaml
14+
useConfigMap: true
15+
```
16+
{% endraw %}
17+
18+
Note: if **useConfigMap** is enabled, sensitive values should be stored in separate secrets, then referenced in the config [using environment variables](https://cortexmetrics.io/docs/configuration/configuration-file/#use-environment-variables-in-the-configuration). Use *extraVolumes*, *extraVolumeMounts*, *extraEnvs*, and *extraArgs* to add environment variables from secrets.

‎templates/_helpers.tpl

Lines changed: 30 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -123,3 +123,33 @@ policy/v1
123123
policy/v1beta1
124124
{{- end -}}
125125
{{- end -}}
126+
127+
{{/*
128+
Get checksum of config secret or configMap
129+
*/}}
130+
{{- define "cortex.configChecksum" -}}
131+
{{- if .Values.useExternalConfig -}}
132+
{{- .Values.externalConfigVersion -}}
133+
{{- else if .Values.useConfigMap -}}
134+
{{- include (print $.Template.BasePath "/configmap.yaml") . | sha256sum -}}
135+
{{- else -}}
136+
{{- include (print $.Template.BasePath "/secret.yaml") . | sha256sum -}}
137+
{{- end -}}
138+
{{- end -}}
139+
140+
{{/*
141+
Get volume of config secret of configMap
142+
*/}}
143+
{{- define "cortex.configVolume" -}}
144+
- name: config
145+
{{- if .Values.useExternalConfig }}
146+
secret:
147+
secretName: {{ .Values.externalConfigSecretName }}
148+
{{- else if .Values.useConfigMap }}
149+
configMap:
150+
name: {{ template "cortex.fullname" . }}-config
151+
{{- else }}
152+
secret:
153+
secretName: {{ template "cortex.fullname" . }}
154+
{{- end }}
155+
{{- end -}}

‎templates/alertmanager/alertmanager-dep.yaml

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -26,11 +26,7 @@ spec:
2626
{{- toYaml . | nindent 8 }}
2727
{{- end }}
2828
annotations:
29-
{{- if .Values.useExternalConfig }}
30-
checksum/config: {{ .Values.externalConfigVersion }}
31-
{{- else }}
32-
checksum/config: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
33-
{{- end }}
29+
checksum/config: {{ include "cortex.configChecksum" . }}
3430
{{- with .Values.alertmanager.podAnnotations }}
3531
{{- toYaml . | nindent 8 }}
3632
{{- end }}
@@ -157,13 +153,7 @@ spec:
157153
{{- toYaml .Values.alertmanager.tolerations | nindent 8 }}
158154
terminationGracePeriodSeconds: {{ .Values.alertmanager.terminationGracePeriodSeconds }}
159155
volumes:
160-
- name: config
161-
secret:
162-
{{- if .Values.useExternalConfig }}
163-
secretName: {{ .Values.externalConfigSecretName }}
164-
{{- else }}
165-
secretName: {{ template "cortex.fullname" . }}
166-
{{- end }}
156+
{{- include "cortex.configVolume" . | nindent 8 }}
167157
- name: runtime-config
168158
configMap:
169159
name: {{ template "cortex.fullname" . }}-runtime-config

‎templates/alertmanager/alertmanager-statefulset.yaml

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,7 @@ spec:
4949
{{- toYaml . | nindent 8 }}
5050
{{- end }}
5151
annotations:
52-
{{- if .Values.useExternalConfig }}
53-
checksum/config: {{ .Values.externalConfigVersion }}
54-
{{- else }}
55-
checksum/config: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
56-
{{- end }}
52+
checksum/config: {{ include "cortex.configChecksum" . }}
5753
{{- with .Values.alertmanager.podAnnotations }}
5854
{{- toYaml . | nindent 8 }}
5955
{{- end }}
@@ -81,13 +77,7 @@ spec:
8177
{{- toYaml .Values.alertmanager.tolerations | nindent 8 }}
8278
terminationGracePeriodSeconds: {{ .Values.alertmanager.terminationGracePeriodSeconds }}
8379
volumes:
84-
- name: config
85-
secret:
86-
{{- if .Values.useExternalConfig }}
87-
secretName: {{ .Values.externalConfigSecretName }}
88-
{{- else }}
89-
secretName: {{ template "cortex.fullname" . }}
90-
{{- end }}
80+
{{- include "cortex.configVolume" . | nindent 8 }}
9181
- name: runtime-config
9282
configMap:
9383
name: {{ template "cortex.fullname" . }}-runtime-config

‎templates/compactor/compactor-statefulset.yaml

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -49,11 +49,7 @@ spec:
4949
{{- toYaml . | nindent 8 }}
5050
{{- end }}
5151
annotations:
52-
{{- if .Values.useExternalConfig }}
53-
checksum/config: {{ .Values.externalConfigVersion }}
54-
{{- else }}
55-
checksum/config: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
56-
{{- end }}
52+
checksum/config: {{ include "cortex.configChecksum" . }}
5753
{{- with .Values.compactor.podAnnotations }}
5854
{{- toYaml . | nindent 8 }}
5955
{{- end }}
@@ -81,13 +77,7 @@ spec:
8177
{{- toYaml .Values.compactor.tolerations | nindent 8 }}
8278
terminationGracePeriodSeconds: {{ .Values.compactor.terminationGracePeriodSeconds }}
8379
volumes:
84-
- name: config
85-
secret:
86-
{{- if .Values.useExternalConfig }}
87-
secretName: {{ .Values.externalConfigSecretName }}
88-
{{- else }}
89-
secretName: {{ template "cortex.fullname" . }}
90-
{{- end }}
80+
{{- include "cortex.configVolume" . | nindent 8 }}
9181
- name: runtime-config
9282
configMap:
9383
name: {{ template "cortex.fullname" . }}-runtime-config

‎templates/configmap.yaml

Lines changed: 12 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,12 @@
1+
{{- if (and (not .Values.useExternalConfig) (.Values.useConfigMap)) }}
2+
apiVersion: v1
3+
kind: ConfigMap
4+
metadata:
5+
name: {{ template "cortex.fullname" . }}-config
6+
namespace: {{ .Release.Namespace }}
7+
labels:
8+
{{- include "cortex.labels" . | nindent 4 }}
9+
data:
10+
cortex.yaml: |
11+
{{- tpl (toYaml .Values.config) . | nindent 4 }}
12+
{{- end }}

‎templates/configs/configs-dep.yaml

Lines changed: 2 additions & 12 deletions
Original file line numberDiff line numberDiff line change
@@ -23,11 +23,7 @@ spec:
2323
{{- toYaml . | nindent 8 }}
2424
{{- end }}
2525
annotations:
26-
{{- if .Values.useExternalConfig }}
27-
checksum/config: {{ .Values.externalConfigVersion }}
28-
{{- else }}
29-
checksum/config: {{ include (print $.Template.BasePath "/secret.yaml") . | sha256sum }}
30-
{{- end }}
26+
checksum/config: {{ include "cortex.configChecksum" . }}
3127
{{- with .Values.configs.podAnnotations }}
3228
{{- toYaml . | nindent 8 }}
3329
{{- end }}
@@ -110,13 +106,7 @@ spec:
110106
{{- toYaml .Values.configs.tolerations | nindent 8 }}
111107
terminationGracePeriodSeconds: {{ .Values.configs.terminationGracePeriodSeconds }}
112108
volumes:
113-
- name: config
114-
secret:
115-
{{- if .Values.useExternalConfig }}
116-
secretName: {{ .Values.externalConfigSecretName }}
117-
{{- else }}
118-
secretName: {{ template "cortex.fullname" . }}
119-
{{- end }}
109+
{{- include "cortex.configVolume" . | nindent 8 }}
120110
{{- if .Values.configsdb_postgresql.enabled }}
121111
- name: postgres-password
122112
secret:

0 commit comments

Comments
 (0)