Skip to content

Skaffold 1.28.1 doesn't override kubecontext via SKAFFOLD_KUBE_CONTEXT #6316

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Closed
artemkoru opened this issue Jul 28, 2021 · 3 comments · Fixed by #6331
Closed

Skaffold 1.28.1 doesn't override kubecontext via SKAFFOLD_KUBE_CONTEXT #6316

artemkoru opened this issue Jul 28, 2021 · 3 comments · Fixed by #6331
Labels
area/config area/env-vars kind/bug Something isn't working priority/p2 May take a couple of releases

Comments

@artemkoru
Copy link
Contributor

artemkoru commented Jul 28, 2021 •
edited
Loading

Expected behavior

When I set the SKAFFOLD_KUBE_CONTEXT environment variable, I expect that the value deploy.kubeContext in skaffold.yaml to be ignored and skaffold will deploy my application to the kubecontext that is set in the SKAFFOLD_KUBE_CONTEXT variable. This works fine in versions <= 1.26.0.

Actual behavior

Skaffold ignores SKAFFOLD_KUBE_CONTEXT environment variable and uses the value deploy.kubeContext from skaffold.yaml.

Information

apiVersion: skaffold/v2beta7
kind: Config
metadata:
  name: skaffod-kubecontext-mismatch
deploy:
  kubeContext: kubecontext-1
  helm:
    flags:
      install:
      - "--timeout=5m"
      upgrade:
      - "--timeout=5m"
    releases:
      - name: "skaffod-kubecontext-mismatch"
        chartPath: charts/web-app
        wait: true
        setValueTemplates:
          image.name: nginx
        valuesFiles:
          - helm/values.yaml

Steps to reproduce the behavior

  1. Clone git repo: https://github.com/artemkoru/skaffod-kubecontext-mismatch
  2. Run in console: export SKAFFOLD_KUBE_CONTEXT=kubecontext-2;skaffold deploy
  3. Make sure skaffold deployed your app in the wrong kubecontext (kubecontext-1)

Logs:

➜  skaffod-kubecontext-mismatch skaffold deploy -vdebug                                                                                          
INFO[0000] Activated kube-context "kubecontext-2"          
INFO[0000] Skaffold &{Version:v1.28.1 ConfigVersion:skaffold/v2beta19 GitVersion: GitCommit:7b855e136dc0f9cc5544ed2808c9d5feb767118c BuildDate:2021-07-20T17:51:49Z GoVersion:go1.14.14 Compiler:gc Platform:darwin/amd64 User:} 
INFO[0000] Loaded Skaffold defaults from "/Users/dddddd/.skaffold/config" 
DEBU[0000] config version out of date: upgrading to latest "skaffold/v2beta19" 
DEBU[0000] parsed 1 configs from configuration file /Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch/skaffold.yaml 
DEBU[0000] Defaulting build type to local build         
INFO[0000] Using kubectl context: kubecontext-2            
DEBU[0000] Running command: [minikube version --output=json] 
DEBU[0000] Command output: [{"commit":"15cede53bdc5fe242228853e737333b09d4336b5","minikubeVersion":"v1.19.0"}
] 
DEBU[0000] setting Docker user agent to skaffold-v1.28.1 
DEBU[0000] Using builder: local                         
DEBU[0000] push value not present in NewBuilder, defaulting to true because cluster.PushImages is true 
INFO[0000] build concurrency first set to 0 parsed from *local.Builder[0] 
INFO[0000] final build concurrency value is 0           
DEBU[0000] Running command: [helm version --client]     
DEBU[0000] Command output: [version.BuildInfo{Version:"v3.5.4", GitCommit:"1b5edb69df3d3a08df77c9902dc17af864ff05d1", GitTreeState:"dirty", GoVersion:"go1.16.3"}
] 
Tags used in deployment:
Starting deploy...
DEBU[0000] getting client config for kubeContext: `kubecontext-2` 
INFO[0001] Deploying with helm v3.5.4 ...               
DEBU[0001] Executing template &{envTemplate 0xc0008f2300 0xc000944e00  } with environment map[COLORTERM:truecolor COMMAND_MODE:unix2003 GITHUB_ACTOR:dddddd GIT_ASKPASS:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass.sh GOPATH:/Users/dddddd/go HOME:/Users/dddddd JAVA_HOME:/usr/local/opt/openjdk KUBECONFIG: *** LANG:ru_RU.UTF-8 LESS:-R LOGNAME:dddddd LSCOLORS:Gxfxcxdxbxegedabagacad LaunchInstanceID:28AFA06D-9E36-40F6-94D2-ADABC557A5CD OLDPWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch ORIGINAL_XDG_CURRENT_DESKTOP:undefined PAGER:less PATH:/Users/dddddd/go/bin:/usr/local/opt/openjdk@11/bin:/usr/local/opt/openjdk/bin:/usr/local/opt/libpq/bin:/usr/local/opt/helm@2/bin:/Users/dddddd/yandex-cloud/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Library/Apple/usr/bin PWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch SECURITYSESSIONID:186aa SHELL:/bin/zsh SHLVL:1 SKAFFOLD_KUBE_CONTEXT:kubecontext-2 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.9fathf7KVU/Listeners TERM:xterm-256color TERM_PROGRAM:vscode TERM_PROGRAM_VERSION:1.58.0 TMPDIR:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/ USER:dddddd VSCODE_GIT_ASKPASS_MAIN:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass-main.js VSCODE_GIT_ASKPASS_NODE:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer) VSCODE_GIT_IPC_HANDLE:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/vscode-git-a1038f512b.sock XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/dddddd/.oh-my-zsh _:/usr/local/bin/skaffold __CFBundleIdentifier:com.microsoft.VSCode __CF_USER_TEXT_ENCODING:0x1F5:0x7:0x31] 
DEBU[0001] Executing template &{envTemplate 0xc0008f2400 0xc000945680  } with environment map[COLORTERM:truecolor COMMAND_MODE:unix2003 GITHUB_ACTOR:dddddd GIT_ASKPASS:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass.sh GOPATH:/Users/dddddd/go HOME:/Users/dddddd JAVA_HOME:/usr/local/opt/openjdk KUBECONFIG: *** LANG:ru_RU.UTF-8 LESS:-R LOGNAME:dddddd LSCOLORS:Gxfxcxdxbxegedabagacad LaunchInstanceID:28AFA06D-9E36-40F6-94D2-ADABC557A5CD OLDPWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch ORIGINAL_XDG_CURRENT_DESKTOP:undefined PAGER:less PATH:/Users/dddddd/go/bin:/usr/local/opt/openjdk@11/bin:/usr/local/opt/openjdk/bin:/usr/local/opt/libpq/bin:/usr/local/opt/helm@2/bin:/Users/dddddd/yandex-cloud/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Library/Apple/usr/bin PWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch SECURITYSESSIONID:186aa SHELL:/bin/zsh SHLVL:1 SKAFFOLD_KUBE_CONTEXT:kubecontext-2 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.9fathf7KVU/Listeners TERM:xterm-256color TERM_PROGRAM:vscode TERM_PROGRAM_VERSION:1.58.0 TMPDIR:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/ USER:dddddd VSCODE_GIT_ASKPASS_MAIN:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass-main.js VSCODE_GIT_ASKPASS_NODE:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer) VSCODE_GIT_IPC_HANDLE:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/vscode-git-a1038f512b.sock XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/dddddd/.oh-my-zsh _:/usr/local/bin/skaffold __CFBundleIdentifier:com.microsoft.VSCode __CF_USER_TEXT_ENCODING:0x1F5:0x7:0x31] 
DEBU[0001] Running command: [helm --kube-context kubecontext-1 get all skaffod-kubecontext-mismatch] 
Helm release skaffod-kubecontext-mismatch not installed. Installing...
INFO[0002] Building helm dependencies...                
DEBU[0002] Running command: [helm --kube-context kubecontext-1 dep build charts/web-app] 
DEBU[0002] EnvVarMap: map[]                             
DEBU[0002] Executing template &{envTemplate 0xc0008f2500 0xc000945dc0  } with environment map[COLORTERM:truecolor COMMAND_MODE:unix2003 GITHUB_ACTOR:dddddd GIT_ASKPASS:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass.sh GOPATH:/Users/dddddd/go HOME:/Users/dddddd JAVA_HOME:/usr/local/opt/openjdk KUBECONFIG: *** LANG:ru_RU.UTF-8 LESS:-R LOGNAME:dddddd LSCOLORS:Gxfxcxdxbxegedabagacad LaunchInstanceID:28AFA06D-9E36-40F6-94D2-ADABC557A5CD OLDPWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch ORIGINAL_XDG_CURRENT_DESKTOP:undefined PAGER:less PATH:/Users/dddddd/go/bin:/usr/local/opt/openjdk@11/bin:/usr/local/opt/openjdk/bin:/usr/local/opt/libpq/bin:/usr/local/opt/helm@2/bin:/Users/dddddd/yandex-cloud/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Library/Apple/usr/bin PWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch SECURITYSESSIONID:186aa SHELL:/bin/zsh SHLVL:1 SKAFFOLD_KUBE_CONTEXT:kubecontext-2 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.9fathf7KVU/Listeners TERM:xterm-256color TERM_PROGRAM:vscode TERM_PROGRAM_VERSION:1.58.0 TMPDIR:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/ USER:dddddd VSCODE_GIT_ASKPASS_MAIN:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass-main.js VSCODE_GIT_ASKPASS_NODE:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer) VSCODE_GIT_IPC_HANDLE:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/vscode-git-a1038f512b.sock XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/dddddd/.oh-my-zsh _:/usr/local/bin/skaffold __CFBundleIdentifier:com.microsoft.VSCode __CF_USER_TEXT_ENCODING:0x1F5:0x7:0x31] 
DEBU[0002] Executing template &{envTemplate 0xc0008f2600 0xc000eec0c0  } with environment map[COLORTERM:truecolor COMMAND_MODE:unix2003 GITHUB_ACTOR:dddddd GIT_ASKPASS:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass.sh GOPATH:/Users/dddddd/go HOME:/Users/dddddd JAVA_HOME:/usr/local/opt/openjdk KUBECONFIG: *** LANG:ru_RU.UTF-8 LESS:-R LOGNAME:dddddd LSCOLORS:Gxfxcxdxbxegedabagacad LaunchInstanceID:28AFA06D-9E36-40F6-94D2-ADABC557A5CD OLDPWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch ORIGINAL_XDG_CURRENT_DESKTOP:undefined PAGER:less PATH:/Users/dddddd/go/bin:/usr/local/opt/openjdk@11/bin:/usr/local/opt/openjdk/bin:/usr/local/opt/libpq/bin:/usr/local/opt/helm@2/bin:/Users/dddddd/yandex-cloud/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Library/Apple/usr/bin PWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch SECURITYSESSIONID:186aa SHELL:/bin/zsh SHLVL:1 SKAFFOLD_KUBE_CONTEXT:kubecontext-2 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.9fathf7KVU/Listeners TERM:xterm-256color TERM_PROGRAM:vscode TERM_PROGRAM_VERSION:1.58.0 TMPDIR:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/ USER:dddddd VSCODE_GIT_ASKPASS_MAIN:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass-main.js VSCODE_GIT_ASKPASS_NODE:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer) VSCODE_GIT_IPC_HANDLE:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/vscode-git-a1038f512b.sock XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/dddddd/.oh-my-zsh _:/usr/local/bin/skaffold __CFBundleIdentifier:com.microsoft.VSCode __CF_USER_TEXT_ENCODING:0x1F5:0x7:0x31] 
DEBU[0002] Executing template &{envTemplate 0xc0008f2800 0xc000eec1c0  } with environment map[COLORTERM:truecolor COMMAND_MODE:unix2003 GITHUB_ACTOR:dddddd GIT_ASKPASS:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass.sh GOPATH:/Users/dddddd/go HOME:/Users/dddddd JAVA_HOME:/usr/local/opt/openjdk KUBECONFIG: *** LANG:ru_RU.UTF-8 LESS:-R LOGNAME:dddddd LSCOLORS:Gxfxcxdxbxegedabagacad LaunchInstanceID:28AFA06D-9E36-40F6-94D2-ADABC557A5CD OLDPWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch ORIGINAL_XDG_CURRENT_DESKTOP:undefined PAGER:less PATH:/Users/dddddd/go/bin:/usr/local/opt/openjdk@11/bin:/usr/local/opt/openjdk/bin:/usr/local/opt/libpq/bin:/usr/local/opt/helm@2/bin:/Users/dddddd/yandex-cloud/bin:/usr/local/bin:/usr/bin:/bin:/usr/sbin:/sbin:/usr/local/go/bin:/Library/Apple/usr/bin PWD:/Users/dddddd/projects/asdasd/skaffod-kubecontext-mismatch SECURITYSESSIONID:186aa SHELL:/bin/zsh SHLVL:1 SKAFFOLD_KUBE_CONTEXT:kubecontext-2 SSH_AUTH_SOCK:/private/tmp/com.apple.launchd.9fathf7KVU/Listeners TERM:xterm-256color TERM_PROGRAM:vscode TERM_PROGRAM_VERSION:1.58.0 TMPDIR:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/ USER:dddddd VSCODE_GIT_ASKPASS_MAIN:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Resources/app/extensions/git/dist/askpass-main.js VSCODE_GIT_ASKPASS_NODE:/Users/dddddd/Applications/Visual Studio Code.app/Contents/Frameworks/Code Helper (Renderer).app/Contents/MacOS/Code Helper (Renderer) VSCODE_GIT_IPC_HANDLE:/var/folders/fj/bh3y3kx10klcp15gbkf422vh0000gn/T/vscode-git-a1038f512b.sock XPC_FLAGS:0x0 XPC_SERVICE_NAME:0 ZSH:/Users/dddddd/.oh-my-zsh _:/usr/local/bin/skaffold __CFBundleIdentifier:com.microsoft.VSCode __CF_USER_TEXT_ENCODING:0x1F5:0x7:0x31] 
DEBU[0002] Running command: [helm --kube-context kubecontext-1 install skaffod-kubecontext-mismatch --timeout=5m charts/web-app --set image.name=nginx -f helm/values.yaml --wait] 
NAME: skaffod-kubecontext-mismatch
LAST DEPLOYED: Wed Jul 28 18:39:13 2021
NAMESPACE: default
STATUS: deployed
REVISION: 1
TEST SUITE: None
NOTES:
1. Get the application URL by running these commands:
  export POD_NAME=$(kubectl get pods --namespace default -l "app.kubernetes.io/name=skaffold-kubecontext-mismatch,app.kubernetes.io/instance=skaffod-kubecontext-mismatch" -o jsonpath="{.items[0].metadata.name}")
  echo "Visit http://127.0.0.1:8080 to use your application"
  kubectl --namespace default port-forward $POD_NAME 8080:80
DEBU[0025] Running command: [helm --kube-context kubecontext-1 get all skaffod-kubecontext-mismatch --template {{.Release.Manifest}}] 
DEBU[0026] ---
# Source: web-app/templates/service.yaml
apiVersion: v1
kind: Service
metadata:
  name: skaffod-kubecontext-mismatch-skaffold-kubecontext-mismatch
  labels:
    helm.sh/chart: web-app-0.1.8
    app.kubernetes.io/name: skaffold-kubecontext-mismatch
    app.kubernetes.io/instance: skaffod-kubecontext-mismatch
    app.kubernetes.io/version: "1.0.0"
    app.kubernetes.io/managed-by: Helm
  annotations:
    
spec:
  type: ClusterIP
  ports:
    - port: 80
      targetPort: 80
      protocol: TCP
      name: http
  selector:
    app.kubernetes.io/name: skaffold-kubecontext-mismatch
    app.kubernetes.io/instance: skaffod-kubecontext-mismatch
---
# Source: web-app/templates/deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  name: skaffod-kubecontext-mismatch-skaffold-kubecontext-mismatch
  labels:
    helm.sh/chart: web-app-0.1.8
    app.kubernetes.io/name: skaffold-kubecontext-mismatch
    app.kubernetes.io/instance: skaffod-kubecontext-mismatch
    app.kubernetes.io/version: "1.0.0"
    app.kubernetes.io/managed-by: Helm
    app.kubernetes.io/component: web-app
  annotations:
    
spec:
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/name: skaffold-kubecontext-mismatch
      app.kubernetes.io/instance: skaffod-kubecontext-mismatch
  template:
    metadata:
      annotations:
      labels:
        helm.sh/chart: web-app-0.1.8
        app.kubernetes.io/name: skaffold-kubecontext-mismatch
        app.kubernetes.io/instance: skaffod-kubecontext-mismatch
        app.kubernetes.io/version: "1.0.0"
        app.kubernetes.io/managed-by: Helm
        app.kubernetes.io/component: web-app
    spec:
      securityContext:
        {}
      containers:
        - name: skaffold-kubecontext-mismatch
          securityContext:
            {}
          image: "nginx"
          imagePullPolicy: IfNotPresent
          env:
          ports:
            - name: "http"
              containerPort: 80
              protocol: "TCP"
          livenessProbe:
            httpGet:
              path: /
              port: http
          readinessProbe:
            httpGet:
              path: /
              port: http
      affinity:
        podAntiAffinity:
          preferredDuringSchedulingIgnoredDuringExecution:
            - podAffinityTerm:
                labelSelector:
                  matchLabels:
                    app.kubernetes.io/name: skaffold-kubecontext-mismatch
                    app.kubernetes.io/instance: skaffod-kubecontext-mismatch
                    app.kubernetes.io/component: web-app
                namespaces:
                  - "default"
                topologyKey: kubernetes.io/hostname
              weight: 1 
DEBU[0026] found deployed object: &Service{ObjectMeta:{skaffod-kubecontext-mismatch-skaffold-kubecontext-mismatch      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app.kubernetes.io/instance:skaffod-kubecontext-mismatch app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:skaffold-kubecontext-mismatch app.kubernetes.io/version:1.0.0 helm.sh/chart:web-app-0.1.8] map[] [] []  []},Spec:ServiceSpec{Ports:[]ServicePort{ServicePort{Name:http,Protocol:TCP,Port:80,TargetPort:{0 80 },NodePort:0,AppProtocol:nil,},},Selector:map[string]string{app.kubernetes.io/instance: skaffod-kubecontext-mismatch,app.kubernetes.io/name: skaffold-kubecontext-mismatch,},ClusterIP:,Type:ClusterIP,ExternalIPs:[],SessionAffinity:,LoadBalancerIP:,LoadBalancerSourceRanges:[],ExternalName:,ExternalTrafficPolicy:,HealthCheckNodePort:0,PublishNotReadyAddresses:false,SessionAffinityConfig:nil,IPFamily:nil,TopologyKeys:[],},Status:ServiceStatus{LoadBalancer:LoadBalancerStatus{Ingress:[]LoadBalancerIngress{},},},} 
DEBU[0026] found deployed object: &Deployment{ObjectMeta:{skaffod-kubecontext-mismatch-skaffold-kubecontext-mismatch      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app.kubernetes.io/component:web-app app.kubernetes.io/instance:skaffod-kubecontext-mismatch app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:skaffold-kubecontext-mismatch app.kubernetes.io/version:1.0.0 helm.sh/chart:web-app-0.1.8] map[] [] []  []},Spec:DeploymentSpec{Replicas:*1,Selector:&v1.LabelSelector{MatchLabels:map[string]string{app.kubernetes.io/instance: skaffod-kubecontext-mismatch,app.kubernetes.io/name: skaffold-kubecontext-mismatch,},MatchExpressions:[]LabelSelectorRequirement{},},Template:{{      0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[app.kubernetes.io/component:web-app app.kubernetes.io/instance:skaffod-kubecontext-mismatch app.kubernetes.io/managed-by:Helm app.kubernetes.io/name:skaffold-kubecontext-mismatch app.kubernetes.io/version:1.0.0 helm.sh/chart:web-app-0.1.8] map[] [] []  []} {[] [] [{skaffold-kubecontext-mismatch nginx [] []  [{http 0 80 TCP }] [] [] {map[] map[]} [] [] Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/,Port:{1 0 http},Host:,Scheme:,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:0,PeriodSeconds:0,SuccessThreshold:0,FailureThreshold:0,} &Probe{Handler:Handler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/,Port:{1 0 http},Host:,Scheme:,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,},InitialDelaySeconds:0,TimeoutSeconds:0,PeriodSeconds:0,SuccessThreshold:0,FailureThreshold:0,} nil nil   IfNotPresent &SecurityContext{Capabilities:nil,Privileged:nil,SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,ReadOnlyRootFilesystem:nil,AllowPrivilegeEscalation:nil,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,} false false false}] []  <nil> <nil>  map[]   <nil>  false false false <nil> &PodSecurityContext{SELinuxOptions:nil,RunAsUser:nil,RunAsNonRoot:nil,SupplementalGroups:[],FSGroup:nil,RunAsGroup:nil,Sysctls:[]Sysctl{},WindowsOptions:nil,FSGroupChangePolicy:nil,SeccompProfile:nil,} []   &Affinity{NodeAffinity:nil,PodAffinity:nil,PodAntiAffinity:&PodAntiAffinity{RequiredDuringSchedulingIgnoredDuringExecution:[]PodAffinityTerm{},PreferredDuringSchedulingIgnoredDuringExecution:[]WeightedPodAffinityTerm{WeightedPodAffinityTerm{Weight:1,PodAffinityTerm:PodAffinityTerm{LabelSelector:&v1.LabelSelector{MatchLabels:map[string]string{app.kubernetes.io/component: web-app,app.kubernetes.io/instance: skaffod-kubecontext-mismatch,app.kubernetes.io/name: skaffold-kubecontext-mismatch,},MatchExpressions:[]LabelSelectorRequirement{},},Namespaces:[default],TopologyKey:kubernetes.io/hostname,},},},},}  [] []  <nil> nil [] <nil> <nil> <nil> map[] [] <nil>}},Strategy:DeploymentStrategy{Type:,RollingUpdate:nil,},MinReadySeconds:0,RevisionHistoryLimit:nil,Paused:false,ProgressDeadlineSeconds:nil,},Status:DeploymentStatus{ObservedGeneration:0,Replicas:0,UpdatedReplicas:0,AvailableReplicas:0,UnavailableReplicas:0,Conditions:[]DeploymentCondition{},ReadyReplicas:0,CollisionCount:nil,},} 
INFO[0026] Deploy completed in 25.555 seconds           
Waiting for deployments to stabilize...
DEBU[0026] getting client config for kubeContext: `kubecontext-2` 
Deployments stabilized in 88.996089ms
DEBU[0026] getting client config for kubeContext: `kubecontext-2`

Thanks!
@artemkoru artemkoru changed the title Skaffold 1.28.1 doesn't override kubecontext in skaffold.yaml Skaffold 1.28.1 doesn't override kubecontext via SKAFFOLD_KUBE_CONTEXT Jul 28, 2021
@aaron-prindle aaron-prindle added area/env-vars area/config kind/bug Something isn't working priority/p2 May take a couple of releases labels Jul 28, 2021
@aaron-prindle
Copy link
Contributor

aaron-prindle commented Jul 28, 2021 •
edited
Loading

@artemkoru thanks for the detailed report here. I was able to reproduce the bug mentioned for v1.28.1 and you are correct this is a recent regression (v1.26.0 did work properly here). Additionally I wanted to clarify, the skaffold.yaml file in the repo should have kubeContext: kubecontext-1 set for the repro case (currently I think it is a typo but it says kubeContext: kubecontext-2. As is, kubecontext-1 is not set anywhere in the config in the example above).

NOTE: gke_aprindle-gke-experiments_us-central1-c_cluster-1 & gke_aprindle-gke-experiments_us-central1-c_cluster-2 map to kubecontext-1 & kubecontext-2 in the above repro example @artemkoru posted

v1.26.0 - log snippets with gke_aprindle-gke*, skaffold deploys to SKAFFOLD_KUBE_CONTEXT value gke_aprindle-gke-experiments_us-central1-c_cluster-2

time="2021-07-28T12:54:12-07:00" level=info msg="Activated kube-context \"gke_aprindle-gke-experiments_us-central1-c_cluster-2\""
time="2021-07-28T12:54:12-07:00" level=info msg="Using kubectl context: gke_aprindle-gke-experiments_us-central1-c_cluster-2"
time="2021-07-28T12:54:13-07:00" level=debug msg="Running command: [helm --kube-context gke_aprindle-gke-experiments_us-central1-c_cluster-2 get all skaffod-kubecontext-mismatch]"
time="2021-07-28T12:54:14-07:00" level=debug msg="Running command: [helm --kube-context gke_aprindle-gke-experiments_us-central1-c_cluster-2 dep build charts/web-app]"
time="2021-07-28T12:54:19-07:00" level=debug msg="Running command: [helm --kube-context gke_aprindle-gke-experiments_us-central1-c_cluster-2 get all skaffod-kubecontext-mismatch --template {{.Release.Manifest}}]"
time="2021-07-28T12:54:20-07:00" level=debug msg="getting client config for kubeContext: `gke_aprindle-gke-experiments_us-central1-c_cluster-2`"
time="2021-07-28T12:54:20-07:00" level=debug msg="getting client config for kubeContext: `gke_aprindle-gke-experiments_us-central1-c_cluster-2`"
time="2021-07-28T12:54:20-07:00" level=debug msg="getting client config for kubeContext: `gke_aprindle-gke-experiments_us-central1-c_cluster-2`"
time="2021-07-28T12:54:21-07:00" level=debug msg="Running command: [kubectl --context gke_aprindle-gke-experiments_us-central1-c_cluster-2 rollout status deployment skaffod-kubecontext-mismatch-skaffold-kubecontext-mismatch --namespace default --watch=false]"
time="2021-07-28T12:54:21-07:00" level=debug msg="getting client config for kubeContext: `gke_aprindle-gke-experiments_us-central1-c_cluster-2`"

v1.28.1 - log snippets with gke_aprindle-gke*, skaffold ignores SKAFFOLD_KUBE_CONTEXT value gke_aprindle-gke-experiments_us-central1-c_cluster-2 and deploys to gke_aprindle-gke-experiments_us-central1-c_cluster-1

time="2021-07-28T12:50:00-07:00" level=info msg="Activated kube-context \"gke_aprindle-gke-experiments_us-central1-c_cluster-2\""
time="2021-07-28T12:50:00-07:00" level=info msg="Using kubectl context: gke_aprindle-gke-experiments_us-central1-c_cluster-2"
time="2021-07-28T12:50:01-07:00" level=debug msg="Running command: [helm --kube-context gke_aprindle-gke-experiments_us-central1-c_cluster-1 get all skaffod-kubecontext-mismatch]"
time="2021-07-28T12:50:02-07:00" level=debug msg="Running command: [helm --kube-context gke_aprindle-gke-experiments_us-central1-c_cluster-1 dep build charts/web-app]"
time="2021-07-28T12:50:02-07:00" level=debug msg="Running command: [helm --kube-context gke_aprindle-gke-experiments_us-central1-c_cluster-1 upgrade skaffod-kubecontext-mismatch --timeout=5m charts/web-app --set image.name=nginx -f helm/values.yaml --wait]"
time="2021-07-28T12:50:06-07:00" level=debug msg="Running command: [helm --kube-context gke_aprindle-gke-experiments_us-central1-c_cluster-1 get all skaffod-kubecontext-mismatch --template {{.Release.Manifest}}]"
time="2021-07-28T12:50:06-07:00" level=debug msg="getting client config for kubeContext: `gke_aprindle-gke-experiments_us-central1-c_cluster-2`"
time="2021-07-28T12:50:06-07:00" level=debug msg="getting client config for kubeContext: `gke_aprindle-gke-experiments_us-central1-c_cluster-2`"
time="2021-07-28T12:50:08-07:00" level=debug msg="getting client config for kubeContext: `gke_aprindle-gke-experiments_us-central1-c_cluster-2`"
time="2021-07-28T12:50:08-07:00" level=debug msg="getting client config for kubeContext: `gke_aprindle-gke-experiments_us-central1-c_cluster-2`"

From the above logs for v1.28.1 you can see that both kubecontexts gke_aprindle-gke-experiments_us-central1-c_cluster-1 & gke_aprindle-gke-experiments_us-central1-c_cluster2 appear which is a bug. It seems that v1.27.0+ the executed helm command always uses the value set in the config - kubeContext: gke_aprindle-gke-experiments_us-central1-c_cluster-1 while the other skaffold config/libs seems to store and use the value set in the env var - SKAFFOLD_KUBE_CONTEXT=gke_aprindle-gke-experiments_us-central1-c_cluster-2

Still poking around to see what the exact issue is and what caused it. Between v1.26.0 and v1.28.1 these PRs seem like possible culprits:
#6024

@gsquared94 any idea what the issue might be here? Just wanted to run this past you as I start poking around in case the change above might be related.

@artemkoru
Copy link
Contributor Author

Additionally I wanted to clarify, the skaffold.yaml file in the repo should have kubeContext: kubecontext-1 set for the repro case (currently I think it is a typo but it says kubeContext: kubecontext-2. As is, kubecontext-1 is not set anywhere in the config in the example above).

@aaron-prindle You are right, I made a mistake. Fixed! Thanks for your comment.

@gsquared94 gsquared94 mentioned this issue Jul 29, 2021
Merged
@gsquared94
Copy link
Contributor

@aaron-prindle you're correct, #6024 is the culprit. Raised #6331 to fix this.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area/config area/env-vars kind/bug Something isn't working priority/p2 May take a couple of releases
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

fix: kubeContext override via flag gsquared94/skaffold
3 participants
@aaron-prindle @artemkoru @gsquared94