Skip to content

Commit 5182ef2

Browse files
tssuryatssurya
committed
NPEP: Iron out Egress Support API Design
Signed-off-by: Surya Seetharaman <[email protected]>
1 parent 95ae2f8 commit 5182ef2

File tree

2 files changed

+189
-6
lines changed

2 files changed

+189
-6
lines changed

‎mkdocs.yml

+2-2
Original file line numberDiff line numberDiff line change
@@ -62,11 +62,11 @@ nav:
6262
- Enhancement Proposals:
6363
- Overview: enhancements.md
6464
- Template: npeps/npep-95.md
65-
- Provisional:
66-
- npeps/npep-126-egress-traffic-control.md
65+
- Provisional:
6766
- npeps/npep-122.md
6867
- Implementable:
6968
- npeps/npep-137-conformance-profiles.md
69+
- npeps/npep-126-egress-traffic-control.md
7070
# - Experimental:
7171
# - Standard:
7272
# - Declined:

‎npeps/npep-126-egress-traffic-control.md

+187-4
Original file line numberDiff line numberDiff line change
@@ -1,7 +1,7 @@
11
# NPEP-126: Add northbound traffic support in (B)ANP API
22

33
* Issue: [#126](https://github.com/kubernetes-sigs/network-policy-api/issues/126)
4-
* Status: Provisional
4+
* Status: Implementable
55

66
## TLDR
77

@@ -76,13 +76,196 @@ selected cluster workloads to k8s-apiservers for securing the server.
7676

7777
## API
7878

79-
(... details, can point to PR with changes)
79+
Proof of Concept for the API design details can be found here:
8080

81+
* https://github.com/kubernetes-sigs/network-policy-api/pull/143
82+
* https://github.com/kubernetes-sigs/network-policy-api/pull/185
83+
84+
### Implementing egress traffic control towards cluster nodes
85+
86+
This NPEP proposes to add a new type of `AdminNetworkPolicyEgressPeer` called `Nodes`
87+
to be able to explicitly select nodes (based on the node's labels) in the cluster.
88+
This ensures that if the list of IPs on a node OR list of nodes change, the users
89+
don't need to manually intervene to include those new IPs. The label selectors will
90+
take care of this automatically. Note that the nodeIPs that this type of peer matches
91+
on are the IPs present in `Node.Status.Addresses` field of the node.
92+
93+
```
94+
// AdminNetworkPolicyEgressPeer defines a peer to allow traffic to.
95+
// Exactly one of the selector pointers must be set for a given peer. If a
96+
// consumer observes none of its fields are set, they must assume an unknown
97+
// option has been specified and fail closed.
98+
// +kubebuilder:validation:MaxProperties=1
99+
// +kubebuilder:validation:MinProperties=1
100+
type AdminNetworkPolicyEgressPeer struct {
101+
<snipped>
102+
// Nodes defines a way to select a set of nodes in
103+
// in the cluster. This field follows standard label selector
104+
// semantics; if present but empty, it selects all Nodes.
105+
//
106+
// Support: Core
107+
//
108+
// +optional
109+
Nodes *metav1.LabelSelector `json:"nodes,omitempty"`
110+
}
111+
```
112+
113+
Note that `AdminNetworkPolicyPeer` will be changed to
114+
`AdminNetworkPolicyEgressPeer` and `AdminNetworkPolicyIngressPeer` since ingress and
115+
egress peers have started to diverge at this point and it is easy to
116+
maintain it with two sets of peer definitions.
117+
This ensures nodes can be referred to only as "egress peers".
118+
119+
Example: Admin wants to deny egress traffic from tenants who don't have
120+
`restricted`, `confidential` or `internal` level security clearance
121+
to control-plane nodes at 443 and 6443 ports in the cluster
122+
123+
```
124+
apiVersion: policy.networking.k8s.io/v1alpha1
125+
kind: AdminNetworkPolicy
126+
metadata:
127+
name: node-as-egress-peer
128+
spec:
129+
priority: 55
130+
subject:
131+
namespaces:
132+
matchExpressions:
133+
- {key: security, operator: notIn, values: [restricted, confidential, internal]}
134+
egress:
135+
- name: "deny-all-egress-to-kapi-server"
136+
action: "Deny"
137+
to:
138+
- nodes:
139+
matchLabels:
140+
node-role.kubernetes.io/control-plane:
141+
ports:
142+
- portNumber:
143+
protocol: TCP
144+
port: 443
145+
- portNumber:
146+
protocol: TCP
147+
port: 6443
148+
```
149+
150+
### Implementing egress traffic control towards CIDRs
151+
152+
This NPEP proposes to add a new type of `AdminNetworkPolicyEgressPeer` called `Networks`
153+
to be able to select destination CIDRs. This is provided to be able to select entities
154+
outside the cluster that cannot be selected using the other peer types.
155+
This peer type will not be supported in `AdminNetworkPolicyIngressPeer`.
156+
157+
```
158+
// AdminNetworkPolicyEgressPeer defines a peer to allow traffic to.
159+
// Exactly one of the selector pointers must be set for a given peer. If a
160+
// consumer observes none of its fields are set, they must assume an unknown
161+
// option has been specified and fail closed.
162+
// +kubebuilder:validation:MaxProperties=1
163+
// +kubebuilder:validation:MinProperties=1
164+
type AdminNetworkPolicyEgressPeer struct {
165+
<snipped>
166+
// Networks defines a way to select peers via CIDR blocks. This is
167+
// intended for representing entities that live outside the cluster,
168+
// which can't be selected by pods and namespaces peers, but note
169+
// that cluster-internal traffic will be checked against the rule as
170+
// well, so if you Allow or Deny traffic to `"0.0.0.0/0"`, that will allow
171+
// or deny all IPv4 pod-to-pod traffic as well. If you don't want that,
172+
// add a rule that Passes all pod traffic before the Networks rule.
173+
//
174+
// Support: Core
175+
//
176+
// +optional
177+
// +kubebuilder:validation:MinItems=1
178+
// +kubebuilder:validation:MaxItems=100
179+
Networks []string `json:"networks,omitempty" validate:"omitempty,dive,cidr"`
180+
}
181+
```
182+
183+
Note: It is recommended to use `networks` to select a set of CIDR range destinations
184+
that represent entities outside the cluster. If a user puts a podCIDR, nodeCIDR,
185+
serviceCIDR or other intra-cluster networks, it will work but it is better to use
186+
namespaces, pods, nodes peers to express such entities. Not all implementations can
187+
correctly define the boundary between "internal" and "external" destinations with respect
188+
to a Kubernetes cluster which is why this field is generic enough to select any CIDR
189+
destination.
190+
191+
Example: Let's define ANP and BANP that refer to some CIDR networks:
192+
```
193+
apiVersion: policy.networking.k8s.io/v1alpha1
194+
kind: AdminNetworkPolicy
195+
metadata:
196+
name: network-as-egress-peer
197+
spec:
198+
priority: 70
199+
subject:
200+
namespaces: {}
201+
egress:
202+
- name: "deny-egress-to-external-dns-servers"
203+
action: "Deny"
204+
to:
205+
- networks:
206+
- 194.0.2.0/24
207+
- 205.0.113.15/32
208+
- 199.51.100.10/32
209+
ports:
210+
- portNumber:
211+
protocol: UDP
212+
port: 53
213+
- name: "allow-all-egress-to-intranet"
214+
action: "Allow"
215+
to:
216+
- networks:
217+
- 192.0.2.0/24
218+
- 203.0.113.0/24
219+
- 198.51.100.0/24
220+
- name: "allow-all-intra-cluster-traffic"
221+
action: "Allow"
222+
to:
223+
- networks:
224+
- POD_CIDR
225+
- NODE_CIDR
226+
- SERVICE_CIDR
227+
- name: "pass-all-egress-to-internet"
228+
action: "Pass"
229+
to:
230+
- networks:
231+
- 0.0.0.0/0
232+
---
233+
apiVersion: policy.networking.k8s.io/v1alpha1
234+
kind: BaselineAdminNetworkPolicy
235+
metadata:
236+
name: default
237+
spec:
238+
subject:
239+
namespaces: {}
240+
egress:
241+
- name: "deny-all-egress-to-internet"
242+
action: "Deny"
243+
to:
244+
- networks:
245+
- 0.0.0.0/0
246+
```
247+
This allows admins to specify rules that define:
248+
249+
* all pods cannot talk to company's intranet DNS servers.
250+
* all pods can talk to rest of the company's intranet.
251+
* all pods can talk to other pods, nodes, services.
252+
* all pods cannot talk to internet (using last ANP Pass rule + BANP guardrail rule)
81253

82254
## Alternatives
83255

84-
(List other design alternatives and why we did not go in that
85-
direction)
256+
* Instead of adding CIDR peer directly into the main object, we can
257+
define a new object called `NetworkSet` and use selectors or
258+
name of that object to be referred to from AdminNetworkPolicy and
259+
BaselineAdminNetworkPolicy objects. This is particularly useful
260+
if CIDR ranges are prone to changes versus the current model is
261+
is better if the set of CIDRs are mostly a constant and are only referred
262+
to from one or two egress rules. It increases readability. However the
263+
drawback is if the CIDRs do change, then one has to ensure to update all
264+
the relevant ANPs and BANP accordingly. In order to see whether we need
265+
a new object to be able to define CIDRs in addition to the in-line peer,
266+
we have another NPEP where that is being discussed
267+
https://github.com/kubernetes-sigs/network-policy-api/pull/183. The scope
268+
of this NPEP is limited to inline CIDR peers.
86269

87270
## References
88271

0 commit comments

Comments
 (0)