Description
Hello,
It has been almost a week that I am trying to deploy Trino in my kubernetes cluster and connect it to my Metabase instance in another cluster under the same VPN.
Initially have deployed Trino in my kubernetes cluster using the following custom values, for my case I need to reach trino using the ingress at trino.myorg.com, and I use AWS Route53 to redirect from this domain to my kubernetes master IP:
catalogs:
glue: |-
connector.name=hive
hive.metastore=glue
hive.metastore.glue.region=eu-central-1
fs.native-s3.enabled=true
s3.region=eu-central-1
s3.path-style-access=true
additionalConfigProperties:
- http-server.process-forwarded=true
env:
- name: AWS_ACCESS_KEY_ID
valueFrom:
secretKeyRef:
name: trino-aws-credentials
key: AWS_ACCESS_KEY_ID
- name: AWS_SECRET_ACCESS_KEY
valueFrom:
secretKeyRef:
name: trino-aws-credentials
key: AWS_SECRET_ACCESS_KEY
- name: AWS_REGION
value: eu-central-1
service:
type: ClusterIP
ingress:
enabled: true
className: contour
hosts:
- host: trino.myorg.com
paths:
- path: /
pathType: Prefix
So far so good I am able to reach the UI at trino.myorg.com and I can query my database from the Metabase machine connecting like this via shell: trino --server trino.myorg.com --user metabase --password
But when I try to add the database in Metabase like this:
I get as error: Authentication failed: Unauthorized
If I enable SSL instead, I get: Error executing query
At this point I tried setting up the authorization by adding the following to values.yaml:
server:
config:
https:
enabled: true
port: 8443
keystore:
path: /etc/trino/keystore/keystore.jks
authenticationType: PASSWORD
additionalConfigProperties:
- internal-communication.shared-secret=${ENV:INTERNAL_COMMUNICATION_SECRET}
- http-server.https.keystore.key=${ENV:JKS_PASSWORD}
- http-server.process-forwarded=true
auth:
passwordAuthSecret: trino-password-authentication
env:
- name: INTERNAL_COMMUNICATION_SECRET
valueFrom:
secretKeyRef:
name: trino-generic-secrets
key: INTERNAL_COMMUNICATION_SECRET
- name: JKS_PASSWORD
valueFrom:
secretKeyRef:
name: trino-generic-secrets
key: JKS_PASSWORD
secretMounts:
- name: trino-keystore
secretName: trino-keystore
path: /etc/trino/keystore
Now I can see from the trino-coordinator logs that the authentication settings are loaded:
2024-11-27T14:32:03.590Z INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION
2024-11-27T14:32:03.590Z INFO main Bootstrap tpcds.split-count ---- ---- Number of split to be created. If not specified the number of splits is computed as 'tpcds.splits-per-node * <number of active nodes>'
2024-11-27T14:32:03.590Z INFO main Bootstrap tpcds.splits-per-node 32 4
2024-11-27T14:32:03.590Z INFO main Bootstrap tpcds.with-no-sexism false false
2024-11-27T14:32:03.741Z INFO main io.trino.connector.StaticCatalogManager -- Added catalog tpcds using connector tpcds --
2024-11-27T14:32:03.745Z INFO main io.trino.security.AccessControlManager Using system access control: default
2024-11-27T14:32:03.745Z INFO main io.trino.server.security.PasswordAuthenticatorManager -- Loading password authenticator --
2024-11-27T14:32:03.821Z INFO main org.hibernate.validator.internal.util.Version HV000001: Hibernate Validator 8.0.1.Final
2024-11-27T14:32:03.968Z INFO main Bootstrap PROPERTY DEFAULT RUNTIME DESCRIPTION
2024-11-27T14:32:03.968Z INFO main Bootstrap file.auth-token-cache.max-size 1000 1000 Max number of cached authenticated passwords
2024-11-27T14:32:03.968Z INFO main Bootstrap file.password-file ---- /etc/trino/auth/password.db Location of the file that provides user names and passwords
2024-11-27T14:32:03.968Z INFO main Bootstrap file.refresh-period 5.00s 5.00s How often to reload the password file
2024-11-27T14:32:04.018Z INFO main io.trino.server.security.PasswordAuthenticatorManager -- Loaded password authenticator file --
2024-11-27T14:32:04.050Z INFO main io.trino.server.Server Server startup completed in 9.70s
2024-11-27T14:32:04.051Z INFO main io.trino.server.Server ======== SERVER STARTED ========
But the trino-worker pods fail to start, due to this error:
+ launcher_opts=(--etc-dir /etc/trino)
+ grep -s -q node.id /etc/trino/node.properties
+ launcher_opts+=("-Dnode.id=${HOSTNAME}")
+ exec /usr/lib/trino/bin/launcher run --etc-dir /etc/trino -Dnode.id=trino-worker-6d4d59b679-twxw2
# WARNING: Unable to attach Serviceability Agent. You can try again with escalated privileges. Two options: a) use -Djol.tryWithSudo=true to try with sudo; b) echo 0 | sudo tee /proc/sys/kernel/yama/ptrace_scope
Nov 27, 2024 2:35:10 PM io.airlift.log.Logger info
INFO: Java version: 23.0.1
2024-11-27T14:35:10.408Z INFO main io.airlift.log.Logging Logging to stderr
2024-11-27T14:35:10.409Z INFO main Bootstrap Loading secrets plugins
2024-11-27T14:35:10.409Z INFO main Bootstrap Loading configuration
2024-11-27T14:35:10.486Z INFO main org.hibernate.validator.internal.util.Version HV000001: Hibernate Validator 8.0.1.Final
2024-11-27T14:35:10.598Z INFO main Bootstrap Initializing logging
2024-11-27T14:35:11.317Z ERROR main io.trino.server.Server Configuration is invalid
==========
Errors:
1) Configuration property 'http-server.https.keystore.key' was not used
==========
And in addition I am not able to reach the UI at http://trino.myorg.com cause it says Web Interface is Disabled.
To create the keystore I used this command:
keytool -genkeypair \
-alias trino \
-keyalg RSA \
-keysize 2048 \
-keystore keystore.jks \
-validity 365 \
-dname "CN=trino.myorg.com, OU=Something, O=Org, L=City, ST=State, C=AA" \
-ext "SAN=dns:trino.myorg.com"
To generate the internal communication shared secret I used: openssl rand 512 | base64
I have read almost the entire Trino documentation but I am not able to find a solution to connect Trino on kubernetes with my Metabase, please help me 😭