精华 k8s 入门安装配置并部署nginx

网友投稿 1262 2022-10-11

精华 k8s 入门安装配置并部署nginx

k8s 搭建

1，关闭 swap 内存确保禁止掉swap分区

K8s的要求，在每个宿主机上执行：

sudo swapoff -a#修改/etc/fstab，注释掉swap那行，持久化生效sudo vi /etc/fstab

安装Docker

apt update && apt install docker.io && systemctl start docker && systemctl enable docker

sudo apt-get update && sudo apt-get install -y ca-certificates curl software-properties-common apt-transport-curlcurl -s | sudo apt-key add -

sudo tee /etc/apt/sources.list.d/kubernetes.list <

安装Kubelet kubeadm kubectl

apt-get update && apt-get install -y kubelet kubeadm kubectl# 它是用来锁住这几个apt包的更新的，如果一旦手误更新了这些包，K8s集群就会因为版本不兼容挂了：apt-mark hold kubelet kubeadm kubectlsystemctl enable kubelet && systemctl start kubelet

设置主机hostname

hostnamectl set-hostname k8s-masterhostnamectl set-hostname k8s-node1hostnamectl set-hostname k8s-node2hostnamectl set-hostname k8s-node3vi /etc/hosts

service-cidr和pod-network-cidr介绍

在用kubadm安装k8s时出现一个疑问，service-cidr和pod-network-cidr这个地址如何配置

参数说明

--apiserver-advertise-address=192.168.181.131 这个参数就是master主机的IP地址，例如我的Master主机的IP是：192.168.181.131--image-repository=registry.aliyuncs.com/google_containers 这个是镜像地址，由于国外地址无法访问，故使用的阿里云仓库地址：registry.aliyuncs.com/google_containers--kubernetes-version=v1.17.4 这个参数是-的k8s软件版本号-service-cidr=10.96.0.0/12 这个参数后的IP地址直接就套用10.96.0.0/12 ,以后安装时也套用即可，不要更改--pod-network-cidr=10.244.0.0/16 k8s内部的pod节点之间网络可以使用的IP段，不能和service-cidr写一样，如果不知道怎么配，就先用这个10.244.0.0/16

service-cidr 的选取不能和PodCIDR及本机网络有重叠或者冲突。一般可以选择一个本机网络和PodCIDR都没有用到的私网地址段，比如PODCIDR使用192.168.0.1/16, 那么service cidr可以选择172.16.0.1/20. 主机网段可以选10.1.0.1/8. 三者之间网络无重叠冲突即可。

Docker 和 kubelet 驱动不一致处理：javascript:void(0)

Kubernetes 升级至 1.24 后集群无法启动报错 docker 驱动跟 kubelet 驱动不一致解决方案； # kubelet cgroup driver: \"systemd\" is different from docker cgroup driversudo docker info|grep Cgroupjournalctl -f -u kubelet

初始化k8s 集群

kubeadm reset && kubeadm init \--image-repository registry--hangzhou.aliyuncs.com/google_containers \--kubernetes-version=v1.22.2 \--pod-network-cidr=192.168.3.0/8 \--service-cidr=10.96.0.0/16 \--apiserver-advertise-address=192.168.2.129 \--v=6 # 查看节点状态kubectl get nodeNAME STATUS ROLES AGE VERSIONk8s-master NotReady control-plane,master 10m v1.22.2# 注意复制下下列两项内容：sudo mkdir -p $HOME/.kube && sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config && sudo chown $(id -u):$(id -g) $HOME/.kube/configkubeadm join 10.206.0.6:6443 --token jgzjsx.rpgeo2xsi4g88qvv --discovery-token-ca-cert-hash sha256:7c02cc3d7a3a4d881101129087a41876bd0dd3135e261b7624fc70f655db78a4

安装CNI

a pod network add-on 下面就是要安装Container Network Interface(CNI),这是必须的，不然Master Node 会处于NotReady状态，无法部署任何应用。也就是要先安装CNI才能部署pod.

kubectl apply -f join 10.206.0.6:6443 --token z8b9l2.9etm9mskhzzrlfya \ --discovery-token-ca-cert-hash sha256:2c07adb82773e53e0fc243cda29b165666c25d8cf4255eab7009d2c625bc3603

查看各节点状态

root@k8s-master:/home/timeless# kubectl get nodesNAME STATUS ROLES AGE VERSIONk8s-master Ready control-plane,master 42m v1.22.2k8s-node1 Ready 39m v1.22.2k8s-node2 Ready 39m v1.22.2k8s-node3 Ready 39m v1.22.2

查看 namesace pod 系统应用日志

kubectl -n kube-system logs calico-node-2wkzd

部署 nginx 服务

1、创建nginx-rc.yaml

apiVersion: v1kind: ReplicationControllermetadata: name: nginx-controllerspec: replicas: 2 selector: name: nginx template: metadata: labels: name: nginx spec: containers: - name: nginx image: nginx ports: - containerPort: 80

2、创建 nginx-service-nodeport.yaml

apiVersion: v1kind: Servicemetadata: name: nginx-service-nodeportspec: ports: - port: 8000 targetPort: 80 protocol: TCP type: NodePort selector: name: nginx

3、创建pod

kubectl create -f nginx-rc.yaml

4、创建service

kubectl create -f nginx-service-nodeport.yaml

5、查看pod

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get podsNAME READY STATUS RESTARTS AGEnginx-controller-7kt4z 0/1 ContainerCreating 0 18snginx-controller-wllwc 0/1 ContainerCreating 0 18s

pull image failed

root@k8s-master:/home/timeless# kubectl get podNAME READY STATUS RESTARTS AGEnginx-controller-cttqr 0/1 ImagePullBackOff 0 4m1snginx-controller-z69jv 0/1 ImagePullBackOff 0 4m1s

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get podsNAME READY STATUS RESTARTS AGEnginx-controller-7kt4z 1/1 Running 1 1hnginx-controller-wllwc 1/1 Running 1 1h

查看 pod 的共享命名空间的IP

root@k8s-master:/home/timeless# kubectl get pod nginx-deployment-748755bf57-778d2 --template={{.status.podIP}}192.109.131.23

每个node 节点都可访问每一个Pod 都拥有一个扁平化的共享空间IP

root@k8s-master:/home/timeless# curl 192.109.131.23Welcome to nginx!''''''''

查看集群节点状态：

# 查看集群状态kubectl get csGet "dial tcp 127.0.0.1:10251: connect: connection refused

自學k8s-kubeadm部署過程中遇到的dial tcp 127.0.0.1:10251: connect: connection refused錯誤

查看nginx 状态：

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get svcNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEkubernetes ClusterIP 10.96.0.1 443/TCP 135mnginx-service-nodeport NodePort 10.96.132.209 8000:31857/TCP 36m

kubectl rc 扩缩容：

root@VM-0-6-ubuntu:/home/ubuntu# kubectl scale rc nginx-controller --replicas=5replicationcontroller/nginx-controller scaledroot@VM-0-6-ubuntu:/home/ubuntu# kubectl get svc,pod,nodeNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEservice/kubernetes ClusterIP 10.96.0.1 443/TCP 20hservice/nginx-service-nodeport NodePort 10.96.132.209 8000:31857/TCP 18hNAME READY STATUS RESTARTS AGEpod/nginx-controller-9n5k6 1/1 Running 0 3m53spod/nginx-controller-dqvnf 1/1 Running 0 3m53spod/nginx-controller-qrnwc 1/1 Running 0 3m53spod/nginx-controller-w4wnh 1/1 Running 0 18hpod/nginx-controller-wlngk 1/1 Running 0 18hNAME STATUS ROLES AGE VERSIONnode/vm-0-6-ubuntu Ready control-plane,master 20h v1.22.2node/vm-0-9-ubuntu Ready 20h v1.22.2

查看集群详细信息

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get pod -o wideNAME READY STATUS RESTARTS AGE IP NODE NOMINATED NODE READINESS GATESnginx-controller-9n5k6 1/1 Running 0 23h 192.168.202.4 vm-0-9-ubuntu nginx-controller-dqvnf 1/1 Running 0 23h 192.168.202.5 vm-0-9-ubuntu nginx-controller-qrnwc 1/1 Running 0 23h 192.168.202.6 vm-0-9-ubuntu nginx-controller-w4wnh 1/1 Running 0 41h 192.168.202.3 vm-0-9-ubuntu nginx-controller-wlngk 1/1 Running 0 41h 192.168.202.2 vm-0-9-ubuntu

查看pod 的 label

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get pod --show-labelsNAME READY STATUS RESTARTS AGE LABELSnginx-controller-9n5k6 1/1 Running 0 27h name=nginxnginx-controller-dqvnf 1/1 Running 0 27h name=nginxnginx-controller-qrnwc 1/1 Running 0 27h name=nginxnginx-controller-w4wnh 1/1 Running 0 46h name=nginxnginx-controller-wlngk 1/1 Running 0 46h name=nginx

根据label 查找 Pod

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get pod -l name=nginxNAME READY STATUS RESTARTS AGEnginx-controller-9n5k6 1/1 Running 0 27hnginx-controller-dqvnf 1/1 Running 0 27hnginx-controller-qrnwc 1/1 Running 0 27hnginx-controller-w4wnh 1/1 Running 0 46hnginx-controller-wlngk 1/1 Running 0 46h

创建deployment

查看 deployment yaml语法格式

root@VM-0-6-ubuntu:/home/ubuntu# kubectl explain deployment.apiVersionKIND: DeploymentVERSION: apps/v1FIELD: apiVersion DESCRIPTION: APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: vi nginx-deployment.yaml root@VM-0-6-ubuntu:/home/ubuntu# kubectl explain deployment.apiVersionKIND: DeploymentVERSION: apps/v1FIELD: apiVersion DESCRIPTION: APIVersion defines the versioned schema of this representation of an object. Servers should convert recognized schemas to the latest internal value, and may reject unrecognized values. More info: apiVersion: v1 #指定api版本，此值必须在kubectl apiversion中 kind: Pod #指定创建资源的角色/类型 metadata: #资源的元数据/属性 name: test-pod #资源的名字，在同一个namespace中必须唯一 labels: #设定资源的标签 k8s-app: apache version: v1 kubernetes.io/cluster-service: "true" annotations: #自定义注解列表 - name: String #自定义注解名字 spec: #specification of the resource content 指定该资源的内容 restartPolicy: Always #表明该容器一直运行，默认k8s的策略，在此容器退出后，会立即创建一个相同的容器 nodeSelector: #节点选择，先给主机打标签kubectl label nodes kube-node1 zone=node1 zone: node1 containers: - name: test-pod #容器的名字 image: 10.192.21.18:5000/test/chat:latest #容器使用的镜像地址 imagePullPolicy: Never #三个选择Always、Never、IfNotPresent，每次启动时检查和更新（从registery）images的策略， # Always，每次都检查 # Never，每次都不检查（不管本地是否有） # IfNotPresent，如果本地有就不检查，如果没有就拉取 command: ['sh'] #启动容器的运行命令，将覆盖容器中的Entrypoint,对应Dockefile中的ENTRYPOINT args: ["$(str)"] #启动容器的命令参数，对应Dockerfile中CMD参数 env: #指定容器中的环境变量 - name: str #变量的名字 value: "/etc/run.sh" #变量的值 resources: #资源管理 requests: #容器运行时，最低资源需求，也就是说最少需要多少资源容器才能正常运行 cpu: 0.1 #CPU资源（核数），两种方式，浮点数或者是整数+m，0.1=100m，最少值为0.001核（1m） memory: 32Mi #内存使用量 limits: #资源限制 cpu: 0.5 memory: 1000Mi ports: - containerPort: 80 #容器开发对外的端口 name: #名称 protocol: TCP livenessProbe: #pod内容器健康检查的设置 #通过 path: / #URI地址 port: 80 #host: 127.0.0.1 #主机地址 scheme: HTTP initialDelaySeconds: 180 #表明第一次检测在容器启动后多长时间后开始 timeoutSeconds: 5 #检测的超时时间 periodSeconds: 15 #检查间隔时间 #也可以用这种方法 #exec: 执行命令的方法进行监测，如果其退出码不为0，则认为容器正常 # command: # - cat # - /tmp/health #也可以用这种方法 #tcpSocket: //通过tcpSocket检查健康 # port: number lifecycle: #生命周期管理 postStart: #容器运行之前运行的任务 exec: command: - 'sh' - 'yum upgrade -y' preStop:#容器关闭之前运行的任务 exec: command: ['service stop'] volumeMounts: #挂载持久存储卷 - name: volume #挂载设备的名字，与volumes[*].name 需要对应 mountPath: /data #挂载到容器的某个路径下 readOnly: True volumes: #定义一组挂载设备 - name: volume #定义一个挂载设备的名字 #meptyDir: {} hostPath: path: /opt #挂载设备类型为hostPath，路径为宿主机下的/opt,这里设备类型支持很多种 #nfs

创建一个yaml文件

apiVersion: apps/v1kind: Deploymentmetadata: name: nginx-deploymentspec: replicas: 3 selector: matchLabels: app: nginx template: metadata: labels: app: nginx spec: containers: - name: nginx image: nginx:1.10 ports: - containerPort: 80

执行创建deployment

root@VM-0-6-ubuntu:/home/ubuntu# kubectl create -f nginx-deployment.yamldeployment.apps "nginx-deployment" createdroot@VM-0-6-ubuntu:/home/ubuntu# kubectl get deployment

查看pod 标签：

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get pod --show-labelsNAME READY STATUS RESTARTS AGE LABELSnginx-controller-9n5k6 1/1 Running 0 28h name=nginxnginx-controller-dqvnf 1/1 Running 0 28h name=nginxnginx-controller-qrnwc 1/1 Running 0 28h name=nginxnginx-controller-w4wnh 1/1 Running 0 46h name=nginxnginx-controller-wlngk 1/1 Running 0 46h name=nginxnginx-deployment-897f8f586-htc7n 1/1 Running 0 21m app=nginx,pod-template-hash=897f8f586nginx-deployment-897f8f586-lt5zr 1/1 Running 0 21m app=nginx,pod-template-hash=897f8f586nginx-deployment-897f8f586-p2npp 1/1 Running 0 21m app=nginx,pod-template-hash=897f8f586

通过标签查找 Pod

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get pod -l app=nginxNAME READY STATUS RESTARTS AGEnginx-deployment-897f8f586-htc7n 1/1 Running 0 23mnginx-deployment-897f8f586-lt5zr 1/1 Running 0 23mnginx-deployment-897f8f586-p2npp 1/1 Running 0 24m

查看 deployment 创建过程：

Deployment 管理的是replicaset-controller，RC会创建Pod。Pod自身会-镜像并启动镜像

root@VM-0-6-ubuntu:/home/ubuntu# kubectl describe rs nginx-deploymentName: nginx-deployment-748755bf57Namespace: defaultSelector: app=nginx,pod-template-hash=748755bf57Labels: app=nginx pod-template-hash=748755bf57Annotations: deployment.kubernetes.io/desired-replicas: 3 deployment.kubernetes.io/max-replicas: 4 deployment.kubernetes.io/revision: 1Controlled By: Deployment/nginx-deploymentReplicas: 0 current / 0 desiredPods Status: 0 Running / 0 Waiting / 0 Succeeded / 0 FailedPod Template: Labels: app=nginx pod-template-hash=748755bf57 Containers: nginx: Image: nginx:1.10 Port: 80/TCP Host Port: 0/TCP Environment: Mounts: Volumes: Events: Name: nginx-deployment-897f8f586Namespace: defaultSelector: app=nginx,pod-template-hash=897f8f586Labels: app=nginx pod-template-hash=897f8f586Annotations: deployment.kubernetes.io/desired-replicas: 3 deployment.kubernetes.io/max-replicas: 4 deployment.kubernetes.io/revision: 2Controlled By: Deployment/nginx-deploymentReplicas: 3 current / 3 desiredPods Status: 3 Running / 0 Waiting / 0 Succeeded / 0 FailedPod Template: Labels: app=nginx pod-template-hash=897f8f586 Containers: nginx: Image: nginx:1.11 Port: 80/TCP Host Port: 0/TCP Environment: Mounts: Volumes: Events:

升级nginx镜像

root@VM-0-6-ubuntu:/home/ubuntu# kubectl set image deploy/nginx-deployment nginx=nginx:1.11deployment.apps "nginx-deployment" image updatedroot@VM-0-6-ubuntu:/home/ubuntu# kubectl exec -it nginx-deployment-897f8f586-htc7n bashkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.root@nginx-deployment-897f8f586-htc7n:/# nginx -vnginx version: nginx/1.11.13ctrl+ D 退出

升级镜像的过程是逐步进行的，pod不会一下子全部关闭，而是一个一个升级

查看发布过程

root@nginx-deployment-897f8f586-htc7n:/# kubectl rollout status deploy/nginx-deploymentdeployment "nginx-deployment" successfully rolled out

查看Deployment 历史版本

root@VM-0-6-ubuntu:/home/ubuntu# kubectl rollout history deploy/nginx-deploymentdeployment.apps/nginx-deployment REVISION CHANGE-CAUSE1 2 # 显示历史有两个版本查看两个版本root@VM-0-6-ubuntu:/home/ubuntu# kubectl rollout history deploy/nginx-deployment --revision=1deployment.apps/nginx-deployment with revision #1Pod Template: Labels: app=nginx pod-template-hash=748755bf57 Containers: nginx: Image: nginx:1.10 Port: 80/TCP Host Port: 0/TCP Environment: Mounts: Volumes: root@VM-0-6-ubuntu:/home/ubuntu# kubectl rollout history deploy/nginx-deployment --revision=2deployment.apps/nginx-deployment with revision #2Pod Template: Labels: app=nginx pod-template-hash=897f8f586 Containers: nginx: Image: nginx:1.11 Port: 80/TCP Host Port: 0/TCP Environment: Mounts: Volumes:

编辑deployment

修改nginx 版本为 1.12

apiVersion: apps/v1kind: Deploymentmetadata: annotations: deployment.kubernetes.io/revision: "2" creationTimestamp: "2021-10-19T07:08:12Z" generation: 2 name: nginx-deployment namespace: default resourceVersion: "226930" uid: 008b5b22-ceeb-454d-bed3-d3bd7f17476dspec: progressDeadlineSeconds: 600 replicas: 3 revisionHistoryLimit: 10 selector: matchLabels: app: nginx strategy: rollingUpdate: maxSurge: 25% maxUnavailable: 25% type: RollingUpdate template: metadata: creationTimestamp: null labels: app: nginx spec: containers: - image: nginx:1.12 imagePullPolicy: IfNotPresent name: nginx ports: - containerPort: 80 protocol: TCP resources: {}

查看升级过程

root@VM-0-6-ubuntu:/home/ubuntu# kubectl edit deploy/nginx-deploymentdeployment.apps/nginx-deployment editedroot@VM-0-6-ubuntu:/home/ubuntu# kubectl rollout status deploy/nginx-deploymentdeployment "nginx-deployment" successfully rolled outroot@VM-0-6-ubuntu:/home/ubuntu# kubectl rollout history deploy/nginx-deploymentdeployment.apps/nginx-deployment REVISION CHANGE-CAUSE1 2 3 root@VM-0-6-ubuntu:/home/ubuntu# kubectl exec -it nginx-deployment-f77774fc5-2b7f9 bashkubectl exec [POD] [COMMAND] is DEPRECATED and will be removed in a future version. Use kubectl exec [POD] -- [COMMAND] instead.root@nginx-deployment-f77774fc5-2b7f9:/# nginx -vnginx version: nginx/1.12.2

扩容/缩容（指定--replicas的数量）

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get pod -l app=nginxNAME READY STATUS RESTARTS AGEnginx-deployment-f77774fc5-2b7f9 1/1 Running 0 6m38snginx-deployment-f77774fc5-hh8kj 1/1 Running 0 6m59snginx-deployment-f77774fc5-xmzrk 1/1 Running 0 6m36sroot@VM-0-6-ubuntu:/home/ubuntu# kubectl scale deploy/nginx-deployment --replicas=5deployment.apps/nginx-deployment scaledroot@VM-0-6-ubuntu:/home/ubuntu# kubectl rollout status deploy/nginx-deploymentdeployment "nginx-deployment" successfully rolled outroot@VM-0-6-ubuntu:/home/ubuntu# kubectl get pod -l app=nginxNAME READY STATUS RESTARTS AGEnginx-deployment-f77774fc5-26nm6 1/1 Running 0 19snginx-deployment-f77774fc5-2b7f9 1/1 Running 0 7m41snginx-deployment-f77774fc5-hh8kj 1/1 Running 0 8m2snginx-deployment-f77774fc5-hklrh 1/1 Running 0 19snginx-deployment-f77774fc5-xmzrk 1/1 Running 0 7m39s

创建Service 提供对外访问接口

修改nodePort 端口范围

apiVersion: v1kind: Servicemetadata: name: nginx-service labels: app: nginxspec: ports: - port: 88 targetPort: 80 selector: app: nginx####apiVersion: 指定版本kind: 类型name: 指定服务名称labels: 标签port: Service 服务暴露的端口targetPort: 容器暴露的端口seletor: 关联的Pod的标签

创建service

root@VM-0-6-ubuntu:/home/ubuntu# kubectl create -f nginx-service.yamlservice/nginx-service createdroot@VM-0-6-ubuntu:/home/ubuntu# kubectl get svc/nginx-serviceNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEnginx-service ClusterIP 10.96.36.156 88/TCP 16s

查看service

访问nginx 服务，访问Pod是有负载均衡的

root@VM-0-6-ubuntu:/home/ubuntu# kubectl get svc/nginx-serviceNAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGEnginx-service ClusterIP 10.96.36.156 88/TCP 64sroot@VM-0-6-ubuntu:/home/ubuntu# curl 10.96.36.156:88Welcome to nginx!