第16章 基于K8S构建企业Jenkins CI/CD平台

2字数 2257阅读 569

一. 持续集成/部署/交付概述

持续集成(Continuous Integration,CI): 代码合并、构建、部署、测试都在一起,不断地执行这个过程,并对结果反馈。
持续部署(Continuous Deployment,CD): 部署到测试环境、预生产环境、生产环境。
持续交付(Continuous Delivery,CD): 将最终产品发布到生产环境,给用户使用。

image.png

image.png

image.png

角色 IP 备注 推荐配置
K8S 10.40.6.201
10.40.6.210
10.40.6.213
自行准备 CPU:2C+ 内存:4G+
Harbor 10.40.6.165
Git 10.40.6.165
Jenkins 部署在K8S平台

二. 准备工作

1. 对项目的理解

• 单体架构、微服务
• 怎么部署
• 启动是否有依赖

2. 部署到k8s平台流程

(1). 制作镜像
(2). 容器放到Pod
(3). 控制器管理Pod
(4). 暴露应用
(5). 对外发布应用
(6). 日志管理/监控

3. 不同环境区分配置文件

• configmap
• entrypoint.sh
• 统一配置中心,例如 Apollo,Disconf

4. Harbor镜像仓库

部署参考: https://www.jianshu.com/p/7ca6c59f9882

5. Git代码版本仓库

(1). 安装Git

# yum install git

(2). 创建Git用户密码

# useradd git
# passwd git

(3). 创建仓库

# su - git
$ mkdir java-demo.git
$ cd java-demo.git
$ git --bare init

(4). Git服务器SSH免交互认证

配置客户端(10.40.6.213)与Git服务器SSH免交互认证

# mkdir git && cd git
# git clone git@10.40.6.165:/home/git/java-demo.git
# touch 123
# git add .
# git commit -m 'test'
# git push origin master

(5). 密钥免交互测试

# ssh-keygen  
# ssh-copy-id git@10.40.6.165
# ssh git@10.40.6.165
# cd .. && rm -rf java-demo 
# git clone git@10.40.6.165:/home/git/java-demo.git

三. Kubernetes中部署Jenkins

参考:https://github.com/jenkinsci/kubernetes-plugin/tree/fc40c869edfd9e3904a9a56b0f80c5a25e988fa1/src/main/kubernetes

image.png

NFS服务部署:https://www.jianshu.com/p/26003390626e
创建NFS 动态供给参考:https://www.jianshu.com/p/092eb3aacefc

1. 部署有状态的jenkins Pod

# cat statefulset-jenkins.yml
apiVersion: apps/v1beta1
kind: StatefulSet
metadata:
  name: jenkins
  labels:
    name: jenkins
spec:
  serviceName: jenkins
  replicas: 1
  updateStrategy:
    type: RollingUpdate
  template:
    metadata:
      name: jenkins
      labels:
        name: jenkins
    spec:
      terminationGracePeriodSeconds: 10
      serviceAccountName: jenkins
      containers:
        - name: jenkins
          image: jenkins/jenkins:lts-alpine
          imagePullPolicy: Always
          ports:
            - containerPort: 8080
            - containerPort: 50000
          resources:
            limits:
              cpu: 1
              memory: 1Gi
            requests:
              cpu: 0.5
              memory: 500Mi
          env:
            - name: LIMITS_MEMORY
              valueFrom:
                resourceFieldRef:
                  resource: limits.memory
                  divisor: 1Mi
            - name: JAVA_OPTS
              value: -Xmx$(LIMITS_MEMORY)m -XshowSettings:vm -Dhudson.slaves.NodeProvisioner.initialDelay=0 -Dhudson.slaves.NodeProvisioner.MARGIN=50 -Dhudson.slaves.NodeProvisioner.MARGIN0=0.85
          volumeMounts:
            - name: jenkins-home
              mountPath: /var/jenkins_home
          livenessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12
          readinessProbe:
            httpGet:
              path: /login
              port: 8080
            initialDelaySeconds: 60
            timeoutSeconds: 5
            failureThreshold: 12
      securityContext:
        fsGroup: 1000
  volumeClaimTemplates:
  - metadata:
      name: jenkins-home
    spec:
      storageClassName: "managed-nfs-storage"
      accessModes: [ "ReadWriteOnce" ]
      resources:
        requests:
          storage: 1Gi

2. 创建jenkins service

# cat service-jenkins.yml
apiVersion: v1
kind: Service
metadata:
  name: jenkins
spec:
  selector:
    name: jenkins
  type: NodePort
  ports:
    -
      name: http
      port: 80
      targetPort: 8080
      protocol: TCP
      nodePort: 30006
    -
      name: agent
      port: 50000
      protocol: TCP

3. 创建jenkins ingress

# cat ingress-jenkins.yml
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: jenkins
  annotations:
    nginx.ingress.kubernetes.io/ssl-redirect: "true"
    kubernetes.io/tls-acme: "true"
    # 如果上传插件超出默认会报"413 Request Entity Too Large", 增加 client_max_body_size
    nginx.ingress.kubernetes.io/proxy-body-size: 50m
    nginx.ingress.kubernetes.io/proxy-request-buffering: "off"
    # nginx-ingress controller版本小于 0.9.0.beta-18 的配置
    ingress.kubernetes.io/ssl-redirect: "true"
    ingress.kubernetes.io/proxy-body-size: 50m
    ingress.kubernetes.io/proxy-request-buffering: "off"
spec:
  rules:
  - host: jenkins.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: jenkins
          servicePort: 80

4. jenkins认证授权

# cat service-account.yml
---
# 创建名为jenkins的ServiceAccount
apiVersion: v1
kind: ServiceAccount
metadata:
  name: jenkins

---
# 创建名为jenkins的Role,授予允许管理API组的资源Pod
kind: Role
apiVersion: rbac.authorization.k8s.io/v1beta1
metadata:
  name: jenkins
rules:
- apiGroups: [""]
  resources: ["pods"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/exec"]
  verbs: ["create","delete","get","list","patch","update","watch"]
- apiGroups: [""]
  resources: ["pods/log"]
  verbs: ["get","list","watch"]
- apiGroups: [""]
  resources: ["secrets"]
  verbs: ["get"]

---
# 将名为jenkins的Role绑定到名为jenkins的ServiceAccount
apiVersion: rbac.authorization.k8s.io/v1beta1
kind: RoleBinding
metadata:
  name: jenkins
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: Role
  name: jenkins
subjects:
- kind: ServiceAccount
  name: jenkins

5. 获取jenkins密码并登录

# kubectl create -f .
登录容器获取jenkins登录密码:
# kubectl exec -it jenkins-0 bash
bash-4.4$ cat /var/jenkins_home/secrets/initialAdminPassword
0f8c730f4b7a426098283cf94aa57231

jenkins地址:http://10.40.6.213:30006
jenkins 安装插件:pipeline、git、kubernetes

四. Jenkins在K8S中动态创建代理

1. 传统架构与K8S 架构的区别

(1). 传统Master/Slave架构,Master收到Job后,将请求转发到Slave节点处理。Slave节点数固定,Slave节点未能自动申缩容。


Jenkins Master/Slave架构

(2). K8S中Jenkins Master/Slave架构,Master收到Job后,会自动创建Slave节点处理此Job,根据客户端的Job自动申缩容。


K8S中Jenkins Master/Slave架构

2. jenkins插件连接K8S配置

需要先安装pipeline插件, 建议不在UI上配置pod创建模版,免得以后每来一个项目都要创建,管理不方便,建议使用pipeline统一 配置。
Kubernetes插件介绍:https://github.com/jenkinsci/kubernetes-plugin

image.png

五. 构建Jenkins Slave镜像

参考:https://github.com/jenkinsci/docker-jnlp-slave

image.png

1. 构建Jenkins Slave镜像环境准备

构建Jenkins Slave镜像环境准备:
代码拉取:git,安装git命令
单元测试:忽略,这不是我们擅长的,如果公司有可以写进来
代码编译:maven,安装maven包
构建镜像:Dockerfile文件、docker命令(通过挂载宿主机docker)
推送镜像:docker命令(通过挂载宿主机docker)
镜像启动后支持slave: 下载官方slave.jar包(获取:http://10.40.6.213:30006/jnlpJars/slave.jar
启动 slave.ja包:jenkins-slave启动脚步(通过参考文档URL)
maven配置文件:settings.xml (这里配置阿里云的仓库源)

获取相关文件:
Dockerfile
jenkins-slave 启动脚步
settings.xml
slave.jar

创建目录并进入:
mkdir jenkins-slave && cd jenkins-slave

2. Dockerfile配置文件

# cat Dockerfile
FROM centos:7
LABEL maintainer liuzhousheng

RUN yum install -y java-1.8.0-openjdk maven curl git libtool-ltdl-devel && \
    yum clean all && \
    rm -rf /var/cache/yum/* && \
    mkdir -p /usr/share/jenkins

COPY slave.jar /usr/share/jenkins/slave.jar
COPY jenkins-slave /usr/bin/jenkins-slave
COPY settings.xml /etc/maven/settings.xml
RUN chmod +x /usr/bin/jenkins-slave

ENTRYPOINT ["jenkins-slave"]

3. jenkins-slave启动脚步

# cat jenkins-slave
#!/usr/bin/env sh

if [ $# -eq 1 ]; then

    # if `docker run` only has one arguments, we assume user is running alternate command like `bash` to inspect the image
    exec "$@"

else

    # if -tunnel is not provided try env vars
    case "$@" in
        *"-tunnel "*) ;;
        *)
        if [ ! -z "$JENKINS_TUNNEL" ]; then
            TUNNEL="-tunnel $JENKINS_TUNNEL"
        fi ;;
    esac

    # if -workDir is not provided try env vars
    if [ ! -z "$JENKINS_AGENT_WORKDIR" ]; then
        case "$@" in
            *"-workDir"*) echo "Warning: Work directory is defined twice in command-line arguments and the environment variable" ;;
            *)
            WORKDIR="-workDir $JENKINS_AGENT_WORKDIR" ;;
        esac
    fi

    if [ -n "$JENKINS_URL" ]; then
        URL="-url $JENKINS_URL"
    fi

    if [ -n "$JENKINS_NAME" ]; then
        JENKINS_AGENT_NAME="$JENKINS_NAME"
    fi

    if [ -z "$JNLP_PROTOCOL_OPTS" ]; then
        echo "Warning: JnlpProtocol3 is disabled by default, use JNLP_PROTOCOL_OPTS to alter the behavior"
        JNLP_PROTOCOL_OPTS="-Dorg.jenkinsci.remoting.engine.JnlpProtocol3.disabled=true"
    fi

    # If both required options are defined, do not pass the parameters
    OPT_JENKINS_SECRET=""
    if [ -n "$JENKINS_SECRET" ]; then
        case "$@" in
            *"${JENKINS_SECRET}"*) echo "Warning: SECRET is defined twice in command-line arguments and the environment variable" ;;
            *)
            OPT_JENKINS_SECRET="${JENKINS_SECRET}" ;;
        esac
    fi

    OPT_JENKINS_AGENT_NAME=""
    if [ -n "$JENKINS_AGENT_NAME" ]; then
        case "$@" in
            *"${JENKINS_AGENT_NAME}"*) echo "Warning: AGENT_NAME is defined twice in command-line arguments and the environment variable" ;;
            *)
            OPT_JENKINS_AGENT_NAME="${JENKINS_AGENT_NAME}" ;;
        esac
    fi

    #TODO: Handle the case when the command-line and Environment variable contain different values.
    #It is fine it blows up for now since it should lead to an error anyway.

    exec java $JAVA_OPTS $JNLP_PROTOCOL_OPTS -cp /usr/share/jenkins/slave.jar hudson.remoting.jnlp.Main -headless $TUNNEL $URL $WORKDIR $OPT_JENKINS_SECRET $OPT_JENKINS_AGENT_NAME "$@"
fi

4. 获取slave.jar包

wget http://10.40.6.213:30006/jnlpJars/slave.jar

5. maven源配置文件settings.xml

maven源配置文件settings.xml,这里配置阿里云的源。

# cat settings.xml
<?xml version="1.0" encoding="UTF-8"?>

<settings xmlns="http://maven.apache.org/SETTINGS/1.0.0"
          xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance"
          xsi:schemaLocation="http://maven.apache.org/SETTINGS/1.0.0 http://maven.apache.org/xsd/settings-1.0.0.xsd">
  <pluginGroups>
  </pluginGroups>

  <proxies>
  </proxies>

  <servers>
  </servers>

  <mirrors>
    <mirror>
      <id>central</id>
      <mirrorOf>central</mirrorOf>
      <name>aliyun maven</name>
      <url>https://maven.aliyun.com/repository/public</url>
    </mirror>
  </mirrors>

  <profiles>
  </profiles>

</settings>

6. 构建镜像, 并推送至私有镜像仓库

# docker build -t 10.40.6.165/library/jenkins-slave-jdk:1.8 .
     ....
Successfully built 3aaa12e391fc
Successfully tagged 10.40.6.165/library/jenkins-slave-jdk:1.8

# docker login 10.40.6.165   ###先登录镜像仓库再推送
Username: admin
Password:

# docker push 10.40.6.165/library/jenkins-slave-jdk:1.8

注意: 私有镜像仓库用的是http访问,kubernetes是不信任的,得每个node节点配置信任。
参考:https://www.jianshu.com/p/7ca6c59f9882

六. Jenkins Pipeline构建流水线发布

1. Jenkins Pipeline 核心概念

• Jenkins Pipeline是一套插件,支持在Jenkins中实现持续集成、交付管道;
• Pipeline通过特定语法从简单到复杂的传输管道进行建模;
① 声明式:遵循与Groovy相同语法。pipeline { }
② 脚本式:支持Groovy大部分功能,也是非常表达和灵活的工具。node { }
• Jenkins Pipeline的定义被写入一个文本文件,称为Jenkinsfile。

Jenkins Pipeline 核心概念:
Node: 节点,一个 Node 就是一个 Jenkins 节点,Master 或者 Agent,是执行 Step 的具体运行环境,比如我们之前动态运行的 Jenkins Slave 就是一个 Node 节点
Stage: 阶段,一个 Pipeline 可以划分为若干个 Stage,每个 Stage 代表一组操作,比如:Build、Test、Deploy等,Stage 是一个逻辑分组的概念,可以跨多个 Node
Step: 步骤,Step 是最基本的操作单元,可以是打印一句话,也可以是构建一个 Docker 镜像,由各类 Jenkins 插件提供,比如命令:sh ‘make’,就相当于我们平时 shell 终端中执行 make 命令一样。
参考:https://jenkins.io/doc/book/pipeline/syntax/

image.png

image.png

2. 拉取代码配置

(1). Pipeline语法使用

image.png

(2). 生成Pipeline语句

image.png

(3). 拉取Git代码秘钥配置

image.png

(4). Pileline语句

image.png

然后构建拉取代码测试

七. 编写Pipeline脚本完成CI阶段

参考文档:https://github.com/jenkinsci/kubernetes-plugin
pipeline script注意:
podTemplate 中label和node括号里的字符串要一致
podTemplate 中cloud为之前配置jenkins Cloud 名称
containerTemplate中name最好保持为"jnlp"
pipeline script变量docker_registry_auth、git_auth、8s_auth通过保存在jenkins凭据中相应的凭据ID。

Pipeline Script如下:

// 公共
def registry = "10.40.6.165"
// 项目
def project = "project"
def app_name = "java-demo"
def image_name = "${registry}/${project}/${app_name}:${BUILD_NUMBER}"
def git_address = "git@10.40.6.165:/home/git/java-demo.git"
// 认证
def secret_name = "registry-pull-secret"
def docker_registry_auth = "eba0f763-747e-47e3-a7ed-20eaf5cbab31"  //Harbor login auth
def git_auth = "fec843cc-3cc4-4c97-969f-df0b0f1bcc50" //git login auth
def k8s_auth = "80e66a86-d189-4555-b1ef-054285031b7a"  // k8s auth,CI阶段不用到, CD 部署时才用到

podTemplate(label: 'jenkins-slave', cloud: 'kubernetes', containers: [
    containerTemplate(
        name: 'jnlp',
        image: "${registry}/library/jenkins-slave-jdk:1.8"
    ),
  ],
  volumes: [
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
    hostPathVolume(mountPath: '/usr/bin/docker', hostPath: '/usr/bin/docker')
  ],
)
{
  node("jenkins-slave"){
      // 第一步
      stage('拉取代码'){
         checkout([$class: 'GitSCM', branches: [[name: '${Branch}']], userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_address}"]]])
      }
      // 第二步
      stage('代码编译'){
          sh "mvn clean package -Dmaven.test.skip=true"
      }
      // 第三步
      stage('构建镜像'){
          //Harbor镜像仓库登录验证,
          withCredentials([usernamePassword(credentialsId: "${docker_registry_auth}", passwordVariable: 'password', usernameVariable: 'username')]) {
            sh """
              echo '
                FROM 10.40.6.165/project/java-demo:v1
                RUN rm -rf /usr/local/tomcat/webapps/*
                ADD target/*.war /usr/local/tomcat/webapps/ROOT.war
              ' > Dockerfile
              ls
              ls target
              docker build -t ${image_name} .
              docker login -u ${username} -p '${password}' ${registry}
              docker push ${image_name}
            """
            }
      }
  }
}

pipeline script代码可选择保存在git代码首目录的代码中
测试刚构建的镜像:
docker run -d -p 666:8080 10.40.6.165/project/java-demo:17
访问地址:http://10.40.6.213:666

八. Jenkins在K8s中持续部署

Kubernetes Continuous Deploy插件: 用于将资源配置部署到Kubernetes
插件介绍: https://plugins.jenkins.io/kubernetes-cd
支持以下资源类型:
• Deployment
• Replica Set
• Daemon Set
• Pod
• Job
• Service
• Ingress
• Secret

Jenkins将镜像部署到kubernetes需要配置:
①. 要连接kubernetes 需要用到kubeconfig配置文件,要在pipeline 中读取此配置文件,得配置到jenkins到凭据中,使用jenkins Kubernetes Continuous Deploy插件读取此凭据的ID(kubeconfigID)即可;
②. 指定部署哪个资源文件,deplayment yaml文件;
③. 资源是否使用到secret验证信息,得指定。

1. 生成kubeconfig

(1). 主体角色绑定

liuzhousheng 用户主体绑定deployment-secret角色:
mkdir deployment-client && cd deployment-client

创建角色权限
# cat rbac-role.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
  name: deployment-secret
rules:
- apiGroups: [""]
  resources: ["pods","services"]
  verbs: ["get", "list", "watch", "create", "delete","update","patch"]
- apiGroups: ["extensions", "apps"]
  resources: ["deployments","ingresses"]
  verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]

user liuzhousheng主体绑定deployment-secret角色
# cat rbac-rolebinding.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
  name: deployment-rolebinding
subjects:
- kind: User
  name: liuzhousheng # Name is case sensitive
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role #this must be Role or ClusterRole
  name: deployment-secret # this must match the name of the Role or ClusterRole you wish to bind to
  apiGroup: rbac.authorization.k8s.io

# kubectl create -f rbac-role.yaml
# kubectl create -f rbac-rolebinding.yaml

(2). 生成kubeconfig配置文件

根据集群根证书颁发客户端证书,然后生成连接集群配置文件liuzhousheng-kubeconfig配置文件。
mkdir liuzhousheng && cd liuzhousheng

# cat rbac-user.sh
#!/bin/bash
cat > liuzhousheng-csr.json <<EOF
{
  "CN": "liuzhousheng",
  "hosts": [],
  "key": {
    "algo": "rsa",
    "size": 2048
  },
  "names": [
    {
      "C": "CN",
      "L": "BeiJing",
      "ST": "BeiJing"
    }
  ]
}
EOF
###签发一个客户端证书,注意要指定根证书
cfssl gencert -ca=ca.pem -ca-key=ca-key.pem -config=ca-config.json -profile=kubernetes liuzhousheng-csr.json | cfssljson -bare liuzhousheng

#-----------------------------------
##生成配置文件,使用配置文件连接集群

#配置集群,这里的ca.pem为集群ca证书
kubectl config set-cluster kubernetes \
  --certificate-authority=ca.pem \
  --embed-certs=true \
  --server=https://10.40.6.175:6443 \
  --kubeconfig=liuzhousheng-kubeconfig

#客户端证书配置
kubectl config set-credentials liuzhousheng \
  --client-key=liuzhousheng-key.pem \
  --client-certificate=liuzhousheng.pem \
  --embed-certs=true \
  --kubeconfig=liuzhousheng-kubeconfig

#配置上下文
kubectl config set-context default \
  --cluster=kubernetes \
  --user=liuzhousheng \
  --kubeconfig=liuzhousheng-kubeconfig

kubectl config use-context default --kubeconfig=liuzhousheng-kubeconfig
# cp  /usr/local/src/k8s/kube-apiserver/{ca.pem,ca-key.pem,ca-config.json} ./
# bash rbac-user.sh
# cat liuzhousheng-kubeconfig

2. 配置kuberconfig凭据

将liuzhousheng-kubeconfig配置文件保存到jenkins凭据中


image.png

image.png

3. 编写deploy.yaml文件

将deploy.yaml文件提交到代码仓库中:

# cat deploy.yaml
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: java-demo
  template:
    metadata:
      labels:
        app: java-demo
    spec:
      imagePullSecrets:
      - name: $SECRET_NAME
      containers:
      - name: tomcat
        image: $IMAGE_NAME
        ports:
        - containerPort: 8080
          name: web
        livenessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 5
          failureThreshold: 12
        readinessProbe:
          httpGet:
            path: /
            port: 8080
          initialDelaySeconds: 60
          timeoutSeconds: 5
          failureThreshold: 12


---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  type: NodePort
  selector:
    app: java-demo
  ports:
    - protocol: TCP
      port: 80
      targetPort: 8080

---
apiVersion: extensions/v1beta1
kind: Ingress
metadata:
  name: web
spec:
  rules:
  - host: java.example.com
    http:
      paths:
      - path: /
        backend:
          serviceName: web
          servicePort: 80

# git add .
# git commit -m 'deploy.yaml'
# git push origin master

4. 编写pipeline 脚步

pipeline脚本:

// 公共
def registry = "10.40.6.165"
// 项目
def project = "project"
def app_name = "java-demo"
def image_name = "${registry}/${project}/${app_name}:${BUILD_NUMBER}"
def git_address = "git@10.40.6.165:/home/git/java-demo.git"
// 认证
def secret_name = "registry-pull-secret"
def docker_registry_auth = "eba0f763-747e-47e3-a7ed-20eaf5cbab31"  //Harbor login auth
def git_auth = "fec843cc-3cc4-4c97-969f-df0b0f1bcc50" //git login auth
def k8s_auth = "5a9f013d-b01e-43b6-b7d7-aef2b54c64c7"   //k8s login auth

podTemplate(label: 'jenkins-slave', cloud: 'kubernetes', containers: [
    containerTemplate(
        name: 'jnlp',
        image: "${registry}/library/jenkins-slave-jdk:1.8"
    ),
  ],
  volumes: [
    hostPathVolume(mountPath: '/var/run/docker.sock', hostPath: '/var/run/docker.sock'),
    hostPathVolume(mountPath: '/usr/bin/docker', hostPath: '/usr/bin/docker')
  ],
)
{
  node("jenkins-slave"){
      // 第一步
      stage('拉取代码'){
         checkout([$class: 'GitSCM', branches: [[name: '${Branch}']], userRemoteConfigs: [[credentialsId: "${git_auth}", url: "${git_address}"]]])
      }
      // 第二步
      stage('代码编译'){
          sh "mvn clean package -Dmaven.test.skip=true"
      }
      // 第三步
      stage('构建镜像'){
          //Harbor镜像仓库登录验证,
          withCredentials([usernamePassword(credentialsId: "${docker_registry_auth}", passwordVariable: 'password', usernameVariable: 'username')]) {
            sh """
              echo '
                FROM 10.40.6.165/project/java-demo:v1
                RUN rm -rf /usr/local/tomcat/webapps/*
                ADD target/*.war /usr/local/tomcat/webapps/ROOT.war
              ' > Dockerfile
              ls
              ls target
              docker build -t ${image_name} .
              docker login -u ${username} -p '${password}' ${registry}
              docker push ${image_name}
            """
            }
      }
    // 第四步
      stage('部署到K8S平台'){
          sh """
          sed -i 's#\$IMAGE_NAME#${image_name}#' deploy.yaml
          sed -i 's#\$SECRET_NAME#${secret_name}#' deploy.yaml
          """
          kubernetesDeploy configs: 'deploy.yaml', kubeconfigId: "${k8s_auth}"
      }
  }
}

脚本注意变量:
k8s_auth:jenkins保存kubeconfig配置凭据的ID

5. 创建harbor镜像仓库secret验证

pipeline脚步中的secret_name变量为:访问harbor镜像仓库私有库secret验证,在kubernetes master 创建,如下:

# kubectl create secret --help
# kubectl create secret docker-registry --help
# kubectl create secret docker-registry registry-pull-secret --docker-username=admin --docker-password=Harbor12345 --docker-email=888888@qq.com --docker-server=10.40.6.165

6. 配置代码拉取分支变量

Branch:代码拉取分支选择


image.png

7.构建测试

jenkins构建,然后绑定hosts访问 10.40.6.213 java.example.com

九. Pipeline脚本与源代码版本管理

1. Pipeline 脚本版本管理

将Pipeline 脚本内容保存到同源代码git代码仓库中的Jenkinsfile文件中,同代码版本控制

# ll
drwxr-xr-x 2 root root    34 6月  29 18:12 db
-rw-r--r-- 1 root root  1155 7月   1 22:09 deploy.yaml
-rw-r--r-- 1 root root  2255 7月   1 23:16 Jenkinsfile
-rw-r--r-- 1 root root 11357 6月  29 18:12 LICENSE
-rw-r--r-- 1 root root  1930 6月  29 18:12 pom.xml
-rw-r--r-- 1 root root    88 6月  29 18:12 README.md
drwxr-xr-x 3 root root    18 6月  29 18:12 src

# git add .
# git commit -m 'add Jenkinsfile'
# git push origin master

2. 修改jenkins Pipeline定义

修改jenkins UI配置


image.png
回滚:
# kubectl get deployment
# kubectl rollout history deployment web

回滚到上一个版本
# kubectl rollout history deployment web

回滚到指定版本
# kubectl rollout undo deployment web --to-revision=5

扩缩容:
# kubectl scale deployment web --replicas=5

十. K8s滚动发布实现原理

1. 滚动发布原理

每次只升级一个或多个服务,升级完成后加入
生产环境,不断执行这个过程,直到集群中的
全部旧版升级新版本。
特点:
• 用户无感知,平滑过渡 缺点:
• 部署周期长
• 发布策略较复杂
• 不易回滚


image.png

2. K8s滚动发布架构

1个Deployment
2个ReplicaSet


image.png

十一. 在K8s中实现灰度发布方案

1. 灰度发布原理

只升级部分服务,即让一部分用户继续用老版本,一 部分用户开始用新版本,如果用户对新版本没有什么 意见,那么逐步扩大范围,把所有用户都迁移到新版 本上面来。
特点:
• 保证整体系统稳定性
• 用户无感知,平滑过渡 缺点:
• 自动化要求高


image.png

2. K8s灰度发布架构

结合LB,2个Deployment


image.png

十二. 小结

❖ 使用Jenkins三个插件
• Kubernetes
• Pipeline
• Kubernetes Continuous Deploy

❖ CI/CD环境特点
• Slave弹性伸缩
• 基于镜像隔离构建环境 • 流水线发布,易维护

❖ Jenkins参数化构建可帮助你完成更复杂环境CI/CD