Istio 环境
kubernetes version: v1.18.2
istio version: v1.10.0
运行之后 istio-operator 的日志就抛出下面错误,而且会一直重启
# kubectl get iop -A
NAMESPACE NAME REVISION STATUS AGE
istio-system iop-pro-cluster-8410-1-10-0 iop-1-10-0 ERROR 1h
# kubectl get pod -n istio-system
NAME READY STATUS RESTARTS AGE
istio-operator-iop-1-10-0-759f564df6-6rvbj 1/1 Running 19 1h
failed to update resource with server-side apply for obj ConfigMap/istio-system/istio-sidecar-injector-iop-1-10-0: 415: Unsupported Media Type
failed to update resource with server-side apply for obj MutatingWebhookConfiguration//istio-sidecar-injector-iop-1-10-0: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Deployment/istio-system/istiod-iop-1-10-0: 415: Unsupported Media Type
failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istiod-iop-1-10-0: 415: Unsupported Media Type
failed to update resource with server-side apply for obj HorizontalPodAutoscaler/istio-system/istiod-iop-1-10-0: 415: Unsupported Media Type
2021-08-31T09:42:12.369429Z info installer using server side apply to update obj: ServiceAccount/istio-system/istio-ingressgateway-service-account
2021-08-31T09:42:12.376827Z error installer failed to update resource with server-side apply for obj ServiceAccount/istio-system/istio-ingressgateway-service-account: 415: Unsupported Media Type
2021-08-31T09:42:12.382257Z info installer using server side apply to update obj: Deployment/istio-system/istio-ingressgateway
2021-08-31T09:42:12.384672Z error installer failed to update resource with server-side apply for obj Deployment/istio-system/istio-ingressgateway: 415: Unsupported Media Type
2021-08-31T09:42:12.385266Z info installer using server side apply to update obj: PodDisruptionBudget/istio-system/istio-ingressgateway
2021-08-31T09:42:12.386973Z error installer failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istio-ingressgateway: 415: Unsupported Media Type
2021-08-31T09:42:12.387721Z info installer using server side apply to update obj: Role/istio-system/istio-ingressgateway-sds
2021-08-31T09:42:12.392418Z error installer failed to update resource with server-side apply for obj Role/istio-system/istio-ingressgateway-sds: 415: Unsupported Media Type
2021-08-31T09:42:12.393067Z info installer using server side apply to update obj: RoleBinding/istio-system/istio-ingressgateway-sds
2021-08-31T09:42:12.398916Z error installer failed to update resource with server-side apply for obj RoleBinding/istio-system/istio-ingressgateway-sds: 415: Unsupported Media Type
2021-08-31T09:42:12.401822Z info installer using server side apply to update obj: HorizontalPodAutoscaler/istio-system/istio-ingressgateway
2021-08-31T09:42:12.403878Z error installer failed to update resource with server-side apply for obj HorizontalPodAutoscaler/istio-system/istio-ingressgateway: 415: Unsupported Media Type
2021-08-31T09:42:12.404578Z info installer using server side apply to update obj: Service/istio-system/istio-ingressgateway
2021-08-31T09:42:12.405936Z error installer failed to update resource with server-side apply for obj Service/istio-system/istio-ingressgateway: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Service/istio-system/istiod-iop-1-10-0: 415: Unsupported Media Type
✘ Ingress gateways encountered an error: failed to update resource with server-side apply for obj ServiceAccount/istio-system/istio-ingressgateway-service-account: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Deployment/istio-system/istio-ingressgateway: 415: Unsupported Media Type
failed to update resource with server-side apply for obj PodDisruptionBudget/istio-system/istio-ingressgateway: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Role/istio-system/istio-ingressgateway-sds: 415: Unsupported Media Type
failed to update resource with server-side apply for obj RoleBinding/istio-system/istio-ingressgateway-sds: 415: Unsupported Media Type
failed to update resource with server-side apply for obj HorizontalPodAutoscaler/istio-system/istio-ingressgateway: 415: Unsupported Media Type
failed to update resource with server-side apply for obj Service/istio-system/istio-ingressgateway: 415: Unsupported Media Type
2021-08-31T09:46:44.671636Z error klog error retrieving resource lock istio-system/istio-operator-lock-iop-1-10-0: etcdserver: request timed out
解决问题
仔细看就可以发现错误日志抛出的内容:failed to update resource with server-side apply for obj
服务端更新
搜索了一下 server side 找到 k8s 官方的文档:服务端更新
注意这句话,这个特新是可以禁用的。这里我就想到了 istio-operator 为什么会出现这个问题。
在使用 v1.18.2 版本的 k8s 时,我注意到所有的资源都添加了 managedfields
字段并且配置非常多,看着也比较麻烦。所以就使用 --feature-gates ServerSideApply=false
禁用了 managedfields
问题现在比较清晰了:由于使用 --feature-gates ServerSideApply=false
禁用 managedfields
导致 istio-operator 内部使用了 server side apply 而更新失败,修复的话就直接把这个参数从 api-server 的启动中去掉就可以了
如果是 kubeadm 部署的 k8s,kube-apiserver 配置在下面默认路径中或者更新参数
/etc/kubernetes/manifests/kube-apiserver.yaml
让 istio-operator 重启或者稍等 1 分钟左右
NAMESPACE NAME REVISION STATUS AGE
istio-system iop-pro-cluster-8410-1-10-0 iop-1-10-0 HEALTHY 2h