kubernetes集群证书期限修改（三）

kubernetes集群证书期限修改

1查看证书截止时间

查看证书截止时间，该命令显示 /etc/kubernetes/pki 文件夹中的客户端证书以及 kubeadm 使用的 KUBECONFIG 文件中嵌入的客户端证书的到期时间/剩余时间。
新版本命令：kubeadm certs check-expiration
旧版命令：kubeadm alpha certs check-expiration

[root@k8s-master01 ~]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 May 09, 2025 17:44 UTC   312d            ca                      no      
apiserver                  May 09, 2025 17:44 UTC   312d            ca                      no      
apiserver-etcd-client      May 09, 2025 17:44 UTC   312d            etcd-ca                 no      
apiserver-kubelet-client   May 09, 2025 17:44 UTC   312d            ca                      no      
controller-manager.conf    May 09, 2025 17:44 UTC   312d            ca                      no      
etcd-healthcheck-client    May 09, 2025 17:44 UTC   312d            etcd-ca                 no      
etcd-peer                  May 09, 2025 17:44 UTC   312d            etcd-ca                 no      
etcd-server                May 09, 2025 17:44 UTC   312d            etcd-ca                 no      
front-proxy-client         May 09, 2025 17:44 UTC   312d            front-proxy-ca          no      
scheduler.conf             May 09, 2025 17:44 UTC   312d            ca                      no      CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      May 07, 2034 17:44 UTC   9y              no      
etcd-ca                 May 07, 2034 17:44 UTC   9y              no      
front-proxy-ca          May 07, 2034 17:44 UTC   9y              no

2 命令renew使用

renew:Renew certificates for a Kubernetes cluster（更新 Kubernetes 集群的证书，注：不涉及 CA 根证书）。使用的 renew 命令，来进行重新生成集群证书。
查看重新生成证书下有哪些命令：

#kubeadm certs renew -h[root@k8s-master01 ~]# kubeadm certs renew -h
This command is not meant to be run on its own. See list of available subcommands.
Usage:kubeadm certs renew [flags]kubeadm certs renew [command]
Available Commands:admin.conf               Renew the certificate embedded in the kubeconfig file for the admin to use and for kubeadm itselfall                      Renew all available certificatesapiserver                Renew the certificate for serving the Kubernetes APIapiserver-etcd-client    Renew the certificate the apiserver uses to access etcdapiserver-kubelet-client Renew the certificate for the API server to connect to kubeletcontroller-manager.conf  Renew the certificate embedded in the kubeconfig file for the controller manager to useetcd-healthcheck-client  Renew the certificate for liveness probes to healthcheck etcdetcd-peer                Renew the certificate for etcd nodes to communicate with each otheretcd-server              Renew the certificate for serving etcdfront-proxy-client       Renew the certificate for the front proxy clientscheduler.conf           Renew the certificate embedded in the kubeconfig file for the scheduler manager to use
Flags:-h, --help   help for renew
Global Flags:--add-dir-header           If true, adds the file directory to the header of the log messages--log-file string          If non-empty, use this log file (no effect when -logtostderr=true)--log-file-max-size uint   Defines the maximum size a log file can grow to (no effect when -logtostderr=true). Unit is megabytes. If the value is 0, the maximum file size is unlimited. (default 1800)--one-output               If true, only write logs to their native severity level (vs also writing to each lower severity level; no effect when -logtostderr=true)--rootfs string            [EXPERIMENTAL] The path to the 'real' host root filesystem.--skip-headers             If true, avoid header prefixes in the log messages--skip-log-headers         If true, avoid headers when opening log files (no effect when -logtostderr=true)-v, --v Level                  number for the log level verbosity
Use "kubeadm certs renew [command] --help" for more information about a command.
[root@k8s-master01 ~]# 

2.1 重要参数说明

all: 更新所有可用证书，一般可以直接使用这个更新所有。
apiserver: 单独更新服务 Kubernetes API 的证书，相当于针对单个组件进行生成。
apiserver-etcd-client: 更新 apiserver 用来访问 etcd 的证书。
其它都类似。

2.2 备份旧证书文件目录

主节点上操作，生成新的证书之前最好备份一下数据：

cp -rp /var/lib/etcd /var/lib/etcd.bak
cp -rp /etc/kubernetes /etc/kubernetes.bak

3 配置1年期限证书

以下命令是批量生成1年期的新证书，需要在所有主节点上执行。

#kubeadm certs renew all

 注意：这种自动更新所有证书后，为期也只是1年。

此时所有证书都已重新生成，在 /etc/kubernetes/ 目录下的相关文件都会被更新。

/etc/kubernetes/admin.conf
/etc/kubernetes/controller-manager.conf
/etc/kubernetes/scheduler.conf 

警告： 如果你运行了一个 HA 集群，这个命令需要在所有主节点节点上执行。
说明： certs renew 使用现有的证书作为属性 (Common Name、Organization、SAN 等) 的权威来源， 而不是 kubeadm-config ConfigMap 。强烈建议使它们保持同步。

3.1 查看1年证书

再次查看证书时间就延续了一年：

#kubeadm certs check-expiration

最后重启服务即可。

4 配置50年期限证书

下面我们里生成50年的证书，证书有效期修改方法。

4.1 部署go环境

4.1.2 解压安装包

上传并解压安装包，执行一下命令：

cd /root/
wget  https://studygolang.com/dl/golang/go1.22.4.linux-amd64.tar.gz
tar zxf go1.22.4.linux-amd64.tar.gz -C /usr/local/

4.1.3 配置环境变量

echo "export PATH=$PATH:/usr/local/go/bin" >> .bash_profile
source .bash_profile

4.1.4 验证GO环境

# go version
[root@k8s-master01 ~]# go version
go version go1.22.4 linux/amd64
[root@k8s-master01 ~]#

5 Kubernetes源码下载

首先查看一下k8s的版本，源码要下载跟你当前系统对应的版本，所以我这边下载 v1.28.2就可以。

5.1 上传并解压文件

cd /root/
unzip -q kubernetes-v1.28.2.zip

5.2 修改源代码文件

证书的有效期是需要修改两个文件 constants.go 和 cert.go。

5.3 修改配置constants.go

修改查找 CertificateValidity 字段：

cd /root/kubernetes-v1.28.2
vim ./cmd/kubeadm/app/constants/constants.go
const (// KubernetesDir is the directory Kubernetes owns for storing various configuration filesKubernetesDir = "/etc/kubernetes"// ManifestsSubDirName defines directory name to store manifestsManifestsSubDirName = "manifests"// TempDirForKubeadm defines temporary directory for kubeadm// should be joined with KubernetesDir.TempDirForKubeadm = "tmp"// CertificateBackdate defines the offset applied to notBefore for CA certificates generated by kubeadmCertificateBackdate = time.Minute * 5// CertificateValidity defines the validity for all the signed certificates generated by kubeadmCertificateValidity = time.Hour * 24 * 365 * 50// DefaultCertificateDir defines default certificate directoryDefaultCertificateDir = "pki"
----------------
#默认是1年，CertificateValidity的后面加个50就是50年。

5.4 修改配置cert.go

修改查找第二个 duration365d 字段：

cd cd /root/kubernetes-v1.28.2
vim ./staging/src/k8s.io/client-go/util/cert/cert.go
// NewSelfSignedCACert creates a CA certificate
func NewSelfSignedCACert(cfg Config, key crypto.Signer) (*x509.Certificate, error) {now := time.Now()// returns a uniform random value in [0, max-1), then add 1 to serial to make it a uniform random value in [1, max).serial, err := cryptorand.Int(cryptorand.Reader, new(big.Int).SetInt64(math.MaxInt64-1))if err != nil {return nil, err}serial = new(big.Int).Add(serial, big.NewInt(1))notBefore := now.UTC()if !cfg.NotBefore.IsZero() {notBefore = cfg.NotBefore.UTC()}tmpl := x509.Certificate{SerialNumber: serial,Subject: pkix.Name{CommonName:   cfg.CommonName,Organization: cfg.Organization,},DNSNames:              []string{cfg.CommonName},NotBefore:             notBefore,NotAfter:              now.Add(duration365d * 50).UTC(),KeyUsage:              x509.KeyUsageKeyEncipherment | x509.KeyUsageDigitalSignature | x509.KeyUsageCertSign,BasicConstraintsValid: true,IsCA:                  true,}certDERBytes, err := x509.CreateCertificate(cryptorand.Reader, &tmpl, &tmpl, key.Public(), key)if err != nil {return nil, err}       return x509.ParseCertificate(certDERBytes)
}       
------------------
#修改 duration365d 字段，默认是10年，修改成50。

5.5 编译源代码文件

cd /root/kubernetes-v1.28.2
make WHAT=cmd/kubeadm GOFLAGS=-v

5.5.1 编译过程

[root@k8s-master01 kubernetes-v1.28.2]# make WHAT=cmd/kubeadm GOFLAGS=-v
go version go1.20.8 linux/amd64
+++ [0702 03:08:31] Building go targets for linux/amd64k8s.io/kubernetes/cmd/kubeadm (static)
internal/goarch
encoding
internal/unsafeheader
internal/abi
internal/cpu
internal/coverage/rtcov
internal/goexperiment
internal/bytealg
internal/goos
.....
#确认编译是否有错提示，为 0 表示编译没有报错。
[root@k8s-master01 kubernetes-v1.28.2]# echo $?
0
[root@k8s-master01 kubernetes-v1.28.2]# 

5.5.2 检查编译结果

编译完后查看结果，就可以看到生成的kubeadm二进文件。

[root@k8s-master01 kubernetes-v1.28.2]# ls -l /root/kubernetes-v1.28.2/_output/bin/
total 49612
-rwxr-xr-x 1 root root 50802688 Jul  2 03:22 kubeadm
[root@k8s-master01 kubernetes-v1.28.2]# 

6 替换kubeadm

下面准备把新的kubeadm替换旧的，所以需要把旧的备份一下，记住有多台master都需要备份一下。

#备份
mv /usr/bin/kubeadm /usr/bin/kubeadm_20240702bak
#替换
cp /root/kubernetes-v1.28.2/_output/bin/kubeadm /usr/bin/

7 证书更新

记住如有多台master都需要执行刷新。

#kubeadm certs renew all
[root@k8s-master01 ~]# kubeadm certs renew all
[renew] Reading configuration from the cluster...
[renew] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'
certificate embedded in the kubeconfig file for the admin to use and for kubeadm itself renewed
certificate for serving the Kubernetes API renewed
certificate the apiserver uses to access etcd renewed
certificate for the API server to connect to kubelet renewed
certificate embedded in the kubeconfig file for the controller manager to use renewed
certificate for liveness probes to healthcheck etcd renewed
certificate for etcd nodes to communicate with each other renewed
certificate for serving etcd renewed
certificate for the front proxy client renewed
certificate embedded in the kubeconfig file for the scheduler manager to use renewed
Done renewing certificates. You must restart the kube-apiserver, kube-controller-manager, kube-scheduler and etcd, so that they can use the new certificates.
[root@k8s-master01 ~]# echo $?
0
[root@k8s-master01 ~]#

8 重启服务器节点master

#kubectl -n kube-system get cm kubeadm-config -o yaml
[root@k8s-master01 ~]# kubectl -n kube-system get cm kubeadm-config -o yaml
apiVersion: v1
data:ClusterConfiguration: |apiServer:extraArgs:authorization-mode: Node,RBACtimeoutForControlPlane: 4m0sapiVersion: kubeadm.k8s.io/v1beta3certificatesDir: /etc/kubernetes/pkiclusterName: kubernetescontrollerManager: {}dns: {}etcd:local:dataDir: /var/lib/etcdimageRepository: registry.k8s.iokind: ClusterConfigurationkubernetesVersion: v1.28.2networking:dnsDomain: cluster.localpodSubnet: 10.244.0.0/16serviceSubnet: 10.96.0.0/12scheduler: {}
kind: ConfigMap
metadata:creationTimestamp: "2024-05-09T17:44:43Z"name: kubeadm-confignamespace: kube-systemresourceVersion: "234"uid: fa84e3b1-a1f6-4e26-9b33-77c8c2218d70
[root@k8s-master01 ~]# 

重启后查看新的证书，看结果除了ca以外的证书都变50年了。

9 查看证书

[root@k8s-master01 ~]# kubeadm certs check-expiration
[check-expiration] Reading configuration from the cluster...
[check-expiration] FYI: You can look at this config file with 'kubectl -n kube-system get cm kubeadm-config -o yaml'CERTIFICATE                EXPIRES                  RESIDUAL TIME   CERTIFICATE AUTHORITY   EXTERNALLY MANAGED
admin.conf                 Jun 19, 2074 19:38 UTC   49y             ca                      no      
apiserver                  Jun 19, 2074 19:38 UTC   49y             ca                      no      
apiserver-etcd-client      Jun 19, 2074 19:38 UTC   49y             etcd-ca                 no      
apiserver-kubelet-client   Jun 19, 2074 19:38 UTC   49y             ca                      no      
controller-manager.conf    Jun 19, 2074 19:38 UTC   49y             ca                      no      
etcd-healthcheck-client    Jun 19, 2074 19:38 UTC   49y             etcd-ca                 no      
etcd-peer                  Jun 19, 2074 19:38 UTC   49y             etcd-ca                 no      
etcd-server                Jun 19, 2074 19:38 UTC   49y             etcd-ca                 no      
front-proxy-client         Jun 19, 2074 19:38 UTC   49y             front-proxy-ca          no      
scheduler.conf             Jun 19, 2074 19:38 UTC   49y             ca                      no      CERTIFICATE AUTHORITY   EXPIRES                  RESIDUAL TIME   EXTERNALLY MANAGED
ca                      May 07, 2034 17:44 UTC   9y              no      
etcd-ca                 May 07, 2034 17:44 UTC   9y              no      
front-proxy-ca          May 07, 2034 17:44 UTC   9y              no      
[root@k8s-master01 ~]# 

同理其他台的master，只需要把kubeadm文件，从第一台更新过的scp过去，然后按上面步骤，重新生成新的证书文件就可以了。记住更新过需要重启一下服务或者服务器。到此，证书期限更新完成。