开头语
写在前面:如有问题,以你为准,
目前24年应届生,各位大佬轻喷,部分资料与图片来自网络
内容较长,页面右上角目录方便跳转
概述
- 网络策略指的是 Pod 间的网络隔离策略,默认情况下是互通的。
- Pod 之间的互通是通过如下三个标识符的组合来辨识的:
- ② 被允许的名称空间。
- ③ IP 组块(例外:与 Pod 运行所在的节点的通信总是被允许的, 无论 Pod 或节点的 IP 地址)
如果命名空间没有配置任何NetworkPolicy,则允许所有流量进站和出站
网络策略(Network Policy):是一个K8s资源,用于限制Pod出入流量,提供Pod级别和Namespace级别网络访问控制。
- 应用程序间的访问控制,例如项目A不能访问项目B的Pod
- 开发环境命名空间不能访问测试环境命名空间Pod
- 当Pod暴露到外部时,需要做Pod白名单
- 多租户网络环境隔离
匹配策略
当有一个命名空间有多个策略使用,只要其中一个策略是放通,其他策略都是拒绝,那也是放通
只有允许策略,没有拒绝策略,(可以通过使用一个默认拒绝所有的策略和一个只开放策略来实现)
Pod 隔离和非隔离
apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: networkpol-01namespace: defaultspec:podSelector: # Pod 选择器matchLabels:app: nginx # 选中的 Pod 就被隔离起来了policyTypes: # 策略类型- Ingress # Ingress 入站规则、Egress 出站规则- Egressingress: # 入站白名单,什么能访问我- from:- podSelector: # 选中的 Pod 可以访问 spec.matchLabels 中筛选的 PodmatchLabels:access: granted #有这个标签的pod才能访问被隔离的pod(上面的nginx)ports: # 开放端口和协议- protocol: TCPport: 80egress: # 出站白名单,我能访问什么- to:- podSelector: # spec.matchLabels 中筛选的 Pod 能访问的 PodmatchLabels:app: tomcat- namespaceSelector:matchLabels:kubernetes.io/metadata.name: dev # spec.matchLabels 中筛选的 Pod 能访问 dev 命名空间下的所有ports:- protocol: TCPport: 8080
Namespace 隔离和非隔离
apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: allow-port-from-namespacenamespace: my-appspec:podSelector: {}policyTypes:- Ingressingress:- from:- namespaceSelector:matchLabels:kubernetes.io/metadata.name: echo# 拥有这个标签的namespace 可以访问my-app命名空间ports:- protocol: TCPport: 9000
拒绝操作
入站 yaml
# 默认拒绝所有入站流量apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: default-deny-ingressnamespace: defaultspec:podSelector: {}policyTypes:- Ingress
# 允许所有入站流量apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: allow-all-ingressnamespace: defaultspec:podSelector: {}policyTypes:- Ingressingress:- {}
出站 yaml
# 默认拒绝所有出站流量apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: default-deny-egressspec:podSelector: {}policyTypes:- Egress
# 允许所有出站流量apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: allow-all-egressspec:podSelector: {}policyTypes:- Egressegress:- {}
默认拒绝所有入站和所有出站流量
apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: default-deny-allspec:podSelector: {} # 匹配本命名空间所有podpolicyTypes:- Ingress- Egress# ingress 和 Egress 没有指定规则,则不允许任何流量进出pod
整个 yaml 解析
apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: test-network-policynamespace: default # 配置NetworkPolicy的命名空间spec:podSelector:matchLabels:role: db # 在上面ns定义的命名空间下,符合这个labels的pod,才会被添加networkpolicypolicyTypes: # 定义类型- Ingress #(入站)- Egress #(出站)ingress: # 入站的规则- from: # 源地址,请求来自那个地址- ipBlock: # ip地址cidr: 172.17.0.0/16except: # 除了这个以外- 172.17.1.0/24- namespaceSelector: # namespace 符合有下面这个标签的即可通过 (default命名空间下的pod)matchLabels:project: myproject# kubernetes.io/metadata.name: test 指定某个命名空间下的所有pod- podSelector: # 和上面隔离的命名空间的同一个命名空间下的pod,且有一下标签matchLabels:role: frontendports: # 开放的协议和端口- protocol: TCPport: 6379egress: # 出站规则- to: # 类似于上面的 from- ipBlock: # 来自那个地址cidr: 10.0.0.0/24- podSelector: # spec.matchLabels 中筛选的 Pod 能访问的 PodmatchLabels:app: tomcat- namespaceSelector:matchLabels:kubernetes.io/metadata.name: dev # spec.matchLabels 中筛选的 Pod 能访问 dev 命名空间下的所有ports: # 开放协议端口- protocol: TCPport: 5978
- 隔离 default 名字空间下 role=db 的 Pod (如果它们不是已经被隔离的话)。
- (Ingress 规则)允许以下 Pod 连接到 default 名字空间下的带有 role=db 标签的所有 Pod 的 6379 TCP 端口:
-
- default 名字空间下带有 role=frontend 标签的所有 Pod
- 带有 project=myproject 标签的所有名字空间中的 Pod
- IP 地址范围为 172.17.0.0–172.17.0.255 和 172.17.2.0–172.17.255.255 (即,除了 172.17.1.0/24 之外的所有 172.17.0.0/16)
- (Egress 规则)允许 default 名字空间中任何带有标签 role=db 的 Pod 到 CIDR 10.0.0.0/24 下 5978 TCP 端口的连接。
多个from或同一个from多个规则
or 的逻辑
ingress:- from:- namespaceSelector:matchLabels:kubernetes.io/metadata.name: default- podSelector:matchLabels:app: client1
and
ingress:- from:- namespaceSelector:matchLabels:kubernetes.io/metadata.name: default- from:- podSelector:matchLabels:app: client1
[root@master cks]# kubectl describe networkpolicy -n test ns-app-allow-appName: ns-app-allow-appNamespace: testCreated on: 2023-11-01 07:00:50 -0400 EDTLabels: <none>Annotations: <none>Spec:PodSelector: app=webAllowing ingress traffic:To Port: <any> (traffic allowed to all ports)From:NamespaceSelector: kubernetes.io/metadata.name=defaultFrom:PodSelector: run=client1Not affecting egress trafficPolicy Types: Ingress
and 的逻辑
ingress:- from:- namespaceSelector:matchLabels:kubernetes.io/metadata.name: defaultpodSelector:matchLabels:app: client1
[root@master cks]# kubectl describe networkpolicy -n test ns-app-allow-appName: ns-app-allow-appNamespace: testCreated on: 2023-11-01 07:00:50 -0400 EDTLabels: <none>Annotations: <none>Spec:PodSelector: app=webAllowing ingress traffic:To Port: <any> (traffic allowed to all ports)From:NamespaceSelector: kubernetes.io/metadata.name=defaultPodSelector: run=client1Not affecting egress trafficPolicy Types: Ingress
实操
命名空间级别
限制入站
[root@master cks]# kubectl create ns testnamespace/test created[root@master cks]# kubectl get nsNAME STATUS AGEcalico-apiserver Active 2d2hcalico-system Active 2d2hdefault Active 271dingress-nginx Active 256dkube-node-lease Active 271dkube-public Active 271dkube-system Active 271dkubernetes-dashboard Active 246dstudy Active 231dtest Active 5stigera-operator Active 2d2h[root@master cks]# kubectl run busybox -n test --image=busybox:1.30 -- sleep 12hpod/busybox created[root@master cks]# kubectl run nginx -n test --image=nginx:1.17.1pod/nginx created[root@master cks]# kubectl get pod -n testNAME READY STATUS RESTARTS AGEbusybox 1/1 Running 0 54snginx 1/1 Running 0 4s
[root@master cks]# kubectl run busybox --image=busybox:1.30 -- sleep 12hpod/busybox created[root@master cks]# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESbusybox 1/1 Running 0 33s 10.244.104.36 node2 <none> <none>
进入 busybox pod 里面进行ping(default)
[root@master cks]# kubectl get podNAME READY STATUS RESTARTS AGEbusybox 1/1 Running 0 2m46s[root@master cks]# kubectl get pod -n testNAME READY STATUS RESTARTS AGEbusybox 1/1 Running 0 5m47snginx 1/1 Running 0 4m57s[root@master cks]# kubectl get pod -n test -owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESbusybox 1/1 Running 0 5m54s 10.244.104.34 node2 <none> <none>nginx 1/1 Running 0 5m4s 10.244.104.35 node2 <none> <none>[root@master cks]# kubectl exec -it busybox /bin/shkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.104.35PING 10.244.104.35 (10.244.104.35): 56 data bytes64 bytes from 10.244.104.35: seq=0 ttl=63 time=0.124 ms^C--- 10.244.104.35 ping statistics ---1 packets transmitted, 1 packets received, 0% packet lossround-trip min/avg/max = 0.124/0.124/0.124 ms/ # ping 10.244.104.34PING 10.244.104.34 (10.244.104.34): 56 data bytes64 bytes from 10.244.104.34: seq=0 ttl=63 time=0.093 ms64 bytes from 10.244.104.34: seq=1 ttl=63 time=0.076 ms^C--- 10.244.104.34 ping statistics ---2 packets transmitted, 2 packets received, 0% packet lossround-trip min/avg/max = 0.076/0.084/0.093 ms/ #
编写 networkpolicy,让其他所有命名空间访问不了test命名空间下的pod
# 默认拒绝所有入站流量apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: test-deny-ingressnamespace: testspec:podSelector: {}policyTypes:- Ingress
[root@master cks]# kubectl create -f np-test-web.yaml[root@master cks]# kubectl get NetworkPolicy -n testNAME POD-SELECTOR AGEtest-deny-ingress <none> 2m7s[root@master cks]# kubectl describe NetworkPolicy -n testName: test-deny-ingressNamespace: testCreated on: 2023-11-01 04:00:34 -0400 EDTLabels: <none>Annotations: <none>Spec:PodSelector: <none> (Allowing the specific traffic to all pods in this namespace)Allowing ingress traffic:<none> (Selected pods are isolated for ingress connectivity)Not affecting egress trafficPolicy Types: Ingress
# defualt 下的 busybox[root@master cks]# kubectl exec -it busybox /bin/shkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.104.35PING 10.244.104.35 (10.244.104.35): 56 data bytes
# test 下的 busybox[root@master cks]# kubectl get pod -owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESbusybox 1/1 Running 0 12m 10.244.104.36 node2 <none> <none>[root@master cks]# kubectl exec -it busybox /bin/sh -n testkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping www.baidu.comPING www.baidu.com (14.119.104.254): 56 data bytes64 bytes from 14.119.104.254: seq=0 ttl=127 time=7.583 ms64 bytes from 14.119.104.254: seq=1 ttl=127 time=6.822 ms/ # ping 10.244.104.36PING 10.244.104.36 (10.244.104.36): 56 data bytes64 bytes from 10.244.104.36: seq=0 ttl=63 time=0.071 ms64 bytes from 10.244.104.36: seq=1 ttl=63 time=0.074 ms
结果表明:test 下的 pod 没办法被defualt的pod 访问,但是它可以访问外部
限制出站
# 默认拒绝所有入站流量apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: test-deny-ingressnamespace: testspec:podSelector: {}policyTypes:- Ingress- Egress
[root@master cks]# kubectl apply -f np-test-web.yaml[root@master cks]# kubectl describe NetworkPolicy -n testName: test-deny-ingressNamespace: testCreated on: 2023-11-01 04:00:34 -0400 EDTLabels: <none>Annotations: <none>Spec:PodSelector: <none> (Allowing the specific traffic to all pods in this namespace)Allowing ingress traffic:<none> (Selected pods are isolated for ingress connectivity)Allowing egress traffic:<none> (Selected pods are isolated for egress connectivity)Policy Types: Ingress, Egress[root@master cks]# kubectl exec -it busybox /bin/sh -n testkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping www.baidu.com^C/ # ping 10.244.104.36PING 10.244.104.36 (10.244.104.36): 56 data bytes^C--- 10.244.104.36 ping statistics ---3 packets transmitted, 0 packets received, 100% packet loss
pod 级别
限制入站
kubectl run nginx -n test --image=nginx:1.17.1 -l="app=web"[root@master cks]# kubectl get pod -n test --show-labelsNAME READY STATUS RESTARTS AGE LABELSbusybox 1/1 Running 0 30m run=busyboxnginx 1/1 Running 0 37s app=web
apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: test-deny-ingressnamespace: testspec:podSelector:matchLabels:app: webpolicyTypes:- Ingressingress:- from:- namespaceSelector:matchLabels:kubernetes.io/metadata.name: test# - podSelector: {} # 匹配本命名空间所有pod
[root@master cks]# kubectl apply -f np-test-web.yamlnetworkpolicy.networking.k8s.io/test-deny-ingress changed
[root@master cks]# kubectl get pod -n test -owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESbusybox 1/1 Running 0 34m 10.244.104.34 node2 <none> <none>nginx 1/1 Running 0 4m27s 10.244.104.37 node2 <none> <none>[root@master cks]# kubectl exec -it busybox /bin/sh -n testkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.104.37PING 10.244.104.37 (10.244.104.37): 56 data bytes64 bytes from 10.244.104.37: seq=0 ttl=63 time=0.148 ms64 bytes from 10.244.104.37: seq=1 ttl=63 time=0.105 ms^C--- 10.244.104.37 ping statistics ---2 packets transmitted, 2 packets received, 0% packet lossround-trip min/avg/max = 0.105/0.126/0.148 ms/ # exitcommand terminated with exit code 127[root@master cks]# kubectl exec -it busybox /bin/shkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.104.37PING 10.244.104.37 (10.244.104.37): 56 data bytes^C--- 10.244.104.37 ping statistics ---3 packets transmitted, 0 packets received, 100% packet loss
其他案例
(双策略)允许其他命名空间访问指定命名空间的指定 pod
编写一个拒绝的yaml:只有本命名空间可以访问web pod
# 只允许 test 命名空间可以相互访问,拒绝其他命名空间的访问apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: test-deny-ingressnamespace: testspec:podSelector: {}policyTypes:- Ingressingress:- from:- namespaceSelector:matchLabels:kubernetes.io/metadata.name: test# 或者不指定ingress部分,默认是拒绝
# 只允许test下带标签 app: web 的 pod 可以被其他命名空间访问apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: test-allow-ingressnamespace: testspec:podSelector:matchLabels:app: webpolicyTypes:- Ingressingress:- from:- namespaceSelector:{}
[root@master cks]# kubectl get NetworkPolicy -n testNAME POD-SELECTOR AGEtest-allow-ingress app=web 8stest-deny-ingress <none> 56m[root@master cks]# kubectl get pod -n test -owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESbusybox 1/1 Running 0 68m 10.244.104.34 node2 <none> <none>nginx 1/1 Running 0 38m 10.244.104.37 node2 <none> <none>[root@master cks]# kubectl exec -it busybox /bin/sh -n testkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.104.37PING 10.244.104.37 (10.244.104.37): 56 data bytes64 bytes from 10.244.104.37: seq=0 ttl=63 time=0.098 ms# default 命名空间访问效果
[root@master cks]# kubectl exec -it busybox /bin/shkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.104.34PING 10.244.104.34 (10.244.104.34): 56 data bytes--- 10.244.104.34 ping statistics ---2 packets transmitted, 0 packets received, 100% packet loss/ # ping 10.244.104.37PING 10.244.104.37 (10.244.104.37): 56 data bytes64 bytes from 10.244.104.37: seq=0 ttl=63 time=0.118 ms
pod之间调用
同一命名空间下pod之间的调用
# 只允许 test 命名空间中的带有run:client1 可以访问apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: ns-app-allow-appnamespace: testspec:podSelector:matchLabels:run: webpolicyTypes:- Ingressingress:- from:- podSelector:matchLabels:run: client1
[root@master cks]# kubectl create -f np-test-web-client1.yamlnetworkpolicy.networking.k8s.io/ns-app-allow-app created[root@master cks]# kubectl get NetworkPolicy -n testNAME POD-SELECTOR AGEns-app-allow-app run=web 5stest-allow-ingress app=web 2m26stest-deny-ingress <none> 2m33s[root@master cks]# kubectl get pod -n test -owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESbusybox 1/1 Running 0 156m 10.244.104.34 node2 <none> <none>client1 1/1 Running 0 2m20s 10.244.166.157 node1 <none> <none>nginx 1/1 Running 0 126m 10.244.104.37 node2 <none> <none>[root@master cks]# kubectl exec -it busybox /bin/sh -n testkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.166.157PING 10.244.166.157 (10.244.166.157): 56 data bytes[root@master cks]# kubectl exec -it client1 /bin/sh -n testkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.166.157PING 10.244.166.157 (10.244.166.157): 56 data bytes64 bytes from 10.244.166.157: seq=0 ttl=64 time=0.703 ms64 bytes from 10.244.166.157: seq=1 ttl=64 time=0.037 ms
不同命名空间下pod之间的调用
# 只允许 default 命名空间中的带有run:client1 可以访问apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: ns-app-allow-appnamespace: testspec:podSelector:matchLabels:run: webpolicyTypes:- Ingressingress:- from:- namespaceSelector:matchLabels:name: default podSelector:matchLabels:run: client1
[root@master cks]# kubectl get pod -n test -owideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESbusybox 1/1 Running 0 175m 10.244.104.34 node2 <none> <none>client1 1/1 Running 0 21m 10.244.166.157 node1 <none> <none>nginx 1/1 Running 0 145m 10.244.104.37 node2 <none> <none>[root@master cks]# kubectl exec -it client1 /bin/sh -n testkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.104.37PING 10.244.104.37 (10.244.104.37): 56 data bytes--- 10.244.104.37 ping statistics ---2 packets transmitted, 0 packets received, 100% packet loss[root@master cks]# kubectl exec -it client1 /bin/shkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead./ # ping 10.244.104.37PING 10.244.104.37 (10.244.104.37): 56 data bytes64 bytes from 10.244.104.37: seq=0 ttl=63 time=0.128 ms64 bytes from 10.244.104.37: seq=1 ttl=63 time=0.090 ms
所有命名空间的指定pod
# 允许所有命名空间中的带有run:client1 可以访问apiVersion: networking.k8s.io/v1kind: NetworkPolicymetadata:name: ns-app-allow-appnamespace: testspec:podSelector:matchLabels:run: webpolicyTypes:- Ingressingress:- from:- namespaceSelector: {}podSelector:matchLabels:run: client1