Create a routing extension interface

[kfswain]

What would you like to be added:
An extension interface that allows the deployment of a custom extension/load balancing solution. Standardizing a general interface would allow best practices to be baked in, and help build an eco system around tooling supporting this interface.

Why this is needed:
This is applicable in scenarios where a workload may benefit from a bespoke routing algorithm as opposed to a simple round robin. We do something like this in https://github.com/kubernetes-sigs/gateway-api-inference-extension, and would love to help collaborate on a more general solution that can work for all gateway users.

The way we do this is via ext-proc and the original dst cluster feature original dst host. We rename the header here, and populate that header here. This currently uses many envoy specific features, and is all in our experimental API. As we mature this product we are open to finding a solution that works for all. So we would love to find a way to integrate this into Gateway proper, or at the very least ensure we are aligned with yall on what the final solution should be! Thanks!

[shaneutt]

I think this sounds great as its been a consistent ask over the years, here's some of the prior art:

• Explicit LoadBalancing Configurations for HTTPBackendRefs #992
• Add support for Load Balancing Policy #1778

Note: Implementations have historically been pushed to "do this themselves via policy attachment". Notably the different parts of policy attachment are largely in provisional and experimental at the time of writing. This is mostly meant as an FYI and doesn't preclude the notion of having a discussion and seeing what comes of it, but needs to be taken into account.

I'm in favor, and looking forward to the discussion. ☕

[arkodg]

brainstorming here, I prefer if this was part of a loadBalancer setting

loadBalancer:
  type: External
  external:
    refs:
    - kind: Service
      name: foo  

linking Envoy Gateway's loadBalancer setting here https://gateway.envoyproxy.io/docs/api/extension_types/#loadbalancer

[youngnick]

So we add a loadBalancer stanza to HTTPRoute somewhere? In the Route rule, or backendRef, or both? Note that each of those options have subtly different behaviors.

For example, if you add a loadbalancer stanza at the backendRef, that implies that you're choosing the LB algo for choosing between endpoints in that backend pool, while if you add it at the Route rule level, then you're choosing a backend pool out of multiple options.

[howardjohn]

cc @mikemorris - I think this ends up overlapping a lot with the discussions of a broader "BackendPolicy" with things like retry budget, load balancing, etc

[danehans]

It would be helpful to get input from non-Envoy Gateway implementations.

[shaneutt]

It would be helpful to get input from non-Envoy Gateway implementations.

Yes indeed.

/cc @mlavacca @nik-netlox @sjberman @guicassolato

Check this out and let us know if you have any thoughts?

[sjberman]

To @howardjohn's point, seems like there is potential overlap here with the BackendLBPolicy?

I know the community is trying to move away from too many policies, but attaching a loadBalancer config to a Service is much easier than having to add it to every backendRef in every route that refers to that Service (as an example).

The configuration itself would have to be fairly generic since proxies support different methods and such.

For reference, the NGINX upstream module contains the various LB algorithms and related directives that are supported by NGINX Open Source (NGINX Plus supports even more).

[youngnick]

To be honest, I would rather see this sort of extension done inside a custom backend type rather than being added directly to the Route objects. That way, opting in to this feature means opting in to that backend type. Could that backend type be included in Gateway API? Maybe if we all agree. But that's much easier and safer than adding more fields that we're not sure about into a GA type (HTTPRoute).

[mlavacca]

This proposal makes a lot of sense to me, thanks for bringing this up here.

In Kong, multiple algorithms of load balancing are available through the upstream API.

I agree that this feature should ideally be an extension of some sort (maybe a policy). Embedding it directly into the HTTPRoute looks risky and inefficient from the configuration point of view.

[youngnick]

I think that using a Policy here may add too much complexity, I would prefer to see us try something else first.

[howardjohn]

Maybe a good first step would be to collect use cases and prior art. That should help guide the correct location for the API.