feat: support infra deployment in the gateway namespace

[cnvergence]

What type of PR is this?

feat: support infra deployment in the gateway namespace

What this PR does / why we need it:

Enable deploying Infra envoy proxies in the namespace of related Gateway resources.

• Support GatewayNamespaceMode deploy mode in the gateway config map.
• Adjust infra resources to reflect non-default namespace.
• Set up the auth between xds-server and infra envoy proxy based on sTLS and JWT Auth, set up proper TLS credentials and interceptor in gRPC server.
• Update the related helm chart to set up necessary permissions for envoy gateway ServiceAccount.

JWT token are validated using k8s TokenReview API .

Which issue(s) this PR fixes:

Fixes #2629

Release Notes: Yes

[codecov]

Codecov Report

Attention: Patch coverage is 50.00000% with 11 lines in your changes missing coverage. Please review.

Project coverage is 65.15%. Comparing base (68a2713) to head (bbddde9).
Report is 4 commits behind head on main.

Files with missing lines	Patch %	Lines
api/v1alpha1/envoygateway_helpers.go	0.00%	5 Missing ⚠️
internal/gatewayapi/runner/runner.go	33.33%	2 Missing ⚠️
internal/gatewayapi/securitypolicy.go	33.33%	2 Missing ⚠️
internal/cmd/certgen.go	50.00%	1 Missing ⚠️
internal/extension/registry/extension_manager.go	0.00%	1 Missing ⚠️

❌ Your patch status has failed because the patch coverage (50.00%) is below the target coverage (60.00%). You can increase the patch coverage or adjust the target coverage.

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #5137      +/-   ##
==========================================
- Coverage   65.41%   65.15%   -0.26%     
==========================================
  Files         222      224       +2     
  Lines       35643    35857     +214     
==========================================
+ Hits        23315    23364      +49     
- Misses      10882    11048     +166     
+ Partials     1446     1445       -1     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:

• ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.

[zhaohuabing]

Hi @cnvergence Just checking in - any updates on this PR? Would love to see it land in 1.4.0.

[cnvergence]

hey @zhaohuabing, getting back to this after Kubecon, I need to fix connection between control-plane and proxies, if everything will work again, I think it should be ready for 1.4.0 :)

[zhaohuabing]

hey @zhaohuabing, getting back to this after Kubecon, I need to fix connection between control-plane and proxies, if everything will work again, I think it should be ready for 1.4.0 :)

@cnvergence a quick way to solve this issue is to copy the envoy secret into each namespace that contains a Gateway resource. While not ideal, this would unblock some key use cases -- such as using EG as an Ambient Waypoint proxy, and allow us to get this PR into v1.4.0.

This is not the most optimal solution, since all Gateway infras share the same client cert. As the next step, we can introduce logic in Envoy Gateway to automatically generate a unique cert secret per Gateway. This can be handled in a follow-up PR.

Would love your input on this, @envoyproxy/gateway-maintainers

[arkodg]

hey @zhaohuabing, getting back to this after Kubecon, I need to fix connection between control-plane and proxies, if everything will work again, I think it should be ready for 1.4.0 :)

@cnvergence a quick way to solve this issue is to copy the envoy secret into each namespace that contains a Gateway resource. While not ideal, this would unblock some key use cases -- such as using EG as an Ambient Waypoint proxy, and allow us to get this PR into v1.4.0.

This is not the most optimal solution, since all Gateway infras share the same client cert. As the next step, we can introduce logic in Envoy Gateway to automatically generate a unique cert secret per Gateway. This can be handled in a follow-up PR.

Would love your input on this, @envoyproxy/gateway-maintainers

-1 to this, app teams have will have access to this, and can take advantage

[cnvergence]

@zhaohuabing
Hey, thanks for the hint, we have discussed a few solutions:

• generating certs per every Gateway
• sharing EG cert per every infra proxy
• moving to sTLS with JWT validation

While we can introduce more options as the followup to this, we wanted to do it initially with the option 3.
This way we need only to share the CA cert and set up JWT validation with serviceaccount token for envoy proxies.

[zhaohuabing]

To move this forward, I agree that we can start without client cert and use the JWT token for client-side validation for the namespaced mode, but please keep mTLS for the current mode.

We eventually will need mTLS for stronger security posture and compliance reasons. We may end up introducing a shim to exchange the JWT token for a client cert, and then establish the mTLS XDS connection for Envoy. This solution is similar to the approach taken by Istio's pilot agent.

If @arkodg also feels this is the right approach, we can go ahead.

[cnvergence]

Yes, mTLS by default was always planned, the JWT method would only be used for the gatewayNamespacedMode.

[cnvergence]

/retest

[arkodg]

are we validating that these token belongs to an envoy created by EG ?

[cnvergence]

Just by the service account, I was thinking about it, we will need to have an exact name of the envoy pod

[arkodg]

Can we leverage the IR keys in the snapshot cache which map to the fleet deployment name and namespace?

[zhaohuabing]

Tracked in this issue: #5863

[cnvergence]

I have added the basic check for deployment in snapshot cache, we will need to improve it and make it work with merged gateways mode

[cnvergence]

Thanks for the review, I will tackle open issues as the followups

[zhaohuabing]

LGTM. Thanks for the patience!