基于docker的微服务容器化与编排

准备

在本人的微服务系列中,已经演示了各个spring cloud微服务组件的使用,以及相关的示例微服务应用。在每次启动微服务和对微服务进行扩容、缩容都不方便,本文使用docker-compose将以下的微服务容器化,并进行自动化部署。

相关代码请参考:springcloud-demo
各服务治理组件的介绍,请参考我的微服务系列文章。

1.微服务治理组件列表

名称 描述 内部端口 暴露端口 是否部署集群
discovery 基于Eureka Server的服务注册中心 8761/8762 8761/8762
configserver 基于cloud config的配置中心 8089 8089
gateway 基于zuul的API网关 8086 8086
track 基于seluth+zipkin的服务跟踪 9411 9411
hystrix-dashboard 基于hystrix-dashboard的服务监控 8087 8087
hystrix-turbine-mq 基于turbine+rabbitmq的服务监控数据收集 8089 8089

2.微服务示例列表

名称 描述 内部端口 暴露端口 是否部署集群
hello 服务提供者,依赖mysql数据库 8000 自动
world 服务提供者 8010 自动
helloworld 服务消费者,使用restTemplate+ribbon调用 8020 自动
helloworld-feign 服务消费者,使用feign方式调用 8030 自动

3.环境与工具

  • 环境 linux (ubuntu 16)
  • 工具 docker+gitlab+rabbitmq+docker registry+intellij idea+maven

操作步骤

本文编写两个docker-compose.yml文件,将服务治理和服务示例分开,主要是因为服务示例依赖了服务治理,部分示例必须要等待服务治理组件加载完毕才能正常启动,比如hello项目使用了配置中心config server的配置,config server未加载完毕,hello项目启动异常。

虽然depends_onlinks等具有启动顺序的问题,但解决不了启动是否 ready的问题,关于这个问题,可以参考我的文章:docker-compose 启动顺序

1. 编写 Dockerfile

在每个项目的根目录中,编写Dockerfile,文件内容为

FROM java:8-jre-alpine
RUN sed -i 's/dl-cdn.alpinelinux.org/mirrors.ustc.edu.cn/' /etc/apk/repositories
VOLUME /tmp
ADD target/*.jar app.jar
RUN sh -c 'touch /app.jar'
RUN echo $(date) > /image_built_at
ENTRYPOINT ["java","-Djava.security.egd=file:/dev/./urandom","-jar","/app.jar","--spring.profiles.active=${SPRING_PROFILES_ACTIVE}"]

其中SPRING_PROFILES_ACTIVE为后面docker-compose.yml传入的环境变量,指定项目以使用哪段配置启动。

2. 在原有的配置文件application.yml增加spring.profiles:docker的配置

主要是使用docker后,服务依赖的其他服务的连接发生改变,通过配置环境变量来动态接收docker-compose的配置参数。

  • track增加的spring.profiles:docker配置
---
spring:
  profiles: docker
server:
  port: ${{SERVER_PORT}:9411}
eureka:
  client:
    serviceUrl:
      defaultZone: ${EUREKA_SERVER_LIST}
  • hello 增加的spring.profiles:docker配置
---
spring:
  profiles: docker
  datasource:
    url: jdbc:mysql://mysql:3306/${{MYSQL_DATABASE}:hello}
    username: ${{MYSQL_USERNAME}:user}
    password: ${{MYSQL_PASSWORD}:passworld}
    testWhileIdle: true
    validationQuery: SELECT 1
  jpa:
    show-sql: false
    hibernate:
      ddl-auto: update
      naming-strategy: org.hibernate.cfg.ImprovedNamingStrategy
      properties:
        hibernate:
          dialect: org.hibernate.dialect.MySQL5Dialect
  zipkin:
    base-url: http://track:9411

eureka:
  client:
    serviceUrl:
      defaultZone: ${EUREKA_SERVER_LIST}

${{MYSQL_USERNAME}:user} 是指配置文件在启动的时候如果没有读取到环境变量MYSQL_USERNAME的值,则使用默认值user。这种配置在开发的时候十分有用,比如切换开发、测试、生产环境配置。

3. 编写批量编译的shell脚本images-build.sh

该脚本将进入到各个项目中,使用maven对项目进行打包,然后使用docker,根据Dockerfile对项目进行构建,生成分别带有:lastest 和当前日期标签的两个镜像。

#!/bin/bash

# 编译所有的代码,生成docker 镜像
function progress(){
    local GREEN CLEAN
    GREEN='\033[0;32m'
    CLEAN='\033[0m'
    printf "\n${GREEN}$@  ${CLEAN}\n" >&2
}

set -e

# 镜像仓库前缀
REGPREFIX=127.0.0.1:5000/billjiang
TOTAL=10

#discovery server
progress "Building discovery-server(1/${TOTAL}) jar file ..."
cd common/discovery
mvn clean package -DskipTests
progress "Building discovery(eureka server)(1/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/discovery -q .) ${REGPREFIX}/discovery:$(date -u "+%Y%m%d-%H%M%S")
cd -

#config server
progress "Building config-server(2/${TOTAL}) jar file ..."
cd common/configserver
mvn clean package -DskipTests
progress "Building config server(2/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/configserver -q .) ${REGPREFIX}/configserver:$(date -u "+%Y%m%d-%H%M%S")
cd -

#gateway
progress "Building gateway-zuul(3/${TOTAL}) jar file ..."
cd common/gateway
mvn clean package -DskipTests
progress "Building gateway-zuul(3/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/gateway -q .) ${REGPREFIX}/gateway:$(date -u "+%Y%m%d-%H%M%S")
cd -

#track sleuth+zipkin
progress "Building track(sleuth+zipkin)(4/${TOTAL}) jar file ..."
cd common/track
mvn clean package -DskipTests
progress "Building track by sleuth+zipkin(4/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/track -q .) ${REGPREFIX}/track:$(date -u "+%Y%m%d-%H%M%S")
cd -

#hystrix-dashboard
progress "Building hystrix-dashboard(5/${TOTAL}) jar file ..."
cd common/hystrix-dashboard
mvn clean package -DskipTests
progress "Building hystrix-dashboard(5/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/hystrix-dashboard -q .) ${REGPREFIX}/hystrix-dashboard:$(date -u "+%Y%m%d-%H%M%S")
cd -

#hystrix-turbine-mq
progress "Buildinhg hystrix-turbine-mq(6/${TOTAL}) jar file ..."
cd common/hystrix-turbine-mq
mvn clean package -DskipTests
progress "Building hystrix-turbine-mq(6/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/hystrix-turbine-mq -q .) ${REGPREFIX}/hystrix-turbine-mq:$(date -u "+%Y%m%d-%H%M%S")
cd -

#hello
progress "Buildinhg hello service(7/${TOTAL}) jar file ..."
cd service/hello
mvn clean package -DskipTests
progress "Building hello service(7/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/hello -q .) ${REGPREFIX}/hello:$(date -u "+%Y%m%d-%H%M%S")
cd -


#world
progress "Buildinhg world service(8/${TOTAL}) jar file ..."
cd service/world
mvn clean package -DskipTests
progress "Building world service(8/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/world -q .) ${REGPREFIX}/world:$(date -u "+%Y%m%d-%H%M%S")
cd -

#helloworld
progress "Buildinhg helloworld service(9/${TOTAL}) jar file ..."
cd service/helloworld
mvn clean package -DskipTests
progress "Building helloworld service(9/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/helloworld -q .) ${REGPREFIX}/helloworld:$(date -u "+%Y%m%d-%H%M%S")
cd -

#helloworld-feign
progress "Buildinhg helloworld-feign service(10/${TOTAL}) jar file ..."
cd service/helloworld-feign
mvn clean package -DskipTests
progress "Building helloworld-feign service(10/${TOTAL}) docker image ..."
docker tag $(docker build -t ${REGPREFIX}/helloworld-feign -q .) ${REGPREFIX}/helloworld-feign:$(date -u "+%Y%m%d-%H%M%S")
cd -

4. 进入到项目目录springcloud-demo,执行bash images-build.sh进行批量编译打包

批量编译打包springcloud-demo.png

5. 创建服务治理组件的docker-compose.yml,编写如下内容

version: "2"
services:

  #启动注册中心eureka-server集群 集中管理/服务注册/服务发现
  discovery1:
    container_name: discovery1
    image: 127.0.0.1:5000/billjiang/discovery
    ports:
      - "8761:8761"
    environment:
      - ADDITIONAL_EUREKA_SERVER_LIST=http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT=8761

  discovery2:
    container_name: discovery2
    image: 127.0.0.1:5000/billjiang/discovery
    ports:
      - "8762:8762"
    environment:
      - ADDITIONAL_EUREKA_SERVER_LIST=http://discovery1:8761/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT=8762
    links:
      - discovery1

  #启动配置中心config-server集群 配置更新/配置集中管理/多环境配置
  config1:
    container_name: configserver1
    image: 127.0.0.1:5000/billjiang/configserver
    ports:
      - "8090:8090"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT=8090
    links:
      - discovery1
      - discovery2

  config2:
    container_name: configserver2
    image: 127.0.0.1:5000/billjiang/configserver
    ports:
      - "8091:8091"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT=8091
    links:
      - discovery1
      - discovery2

  #启动网关gateway,可配置成集群,使用nginx进行负载 智能路由/安全控制/负载均衡
  gateway:
    container_name: gateway
    image: 127.0.0.1:5000/billjiang/gateway
    ports:
      - "8086:8086"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT=8086
    links:
      - discovery1
      - discovery2


  # 启动seluth+zipkin zipkin服务跟踪/依赖可视化
  track:
    container_name: track
    image: 127.0.0.1:5000/billjiang/track
    ports:
      - "9411:9411"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT=9411
    links:
      - discovery1
      - discovery2


   # hystrix
  # 启动hystrix-dashboard
  hystrix-dashboard:
    container_name: hystrix-dashboard
    image: 127.0.0.1:5000/billjiang/hystrix-dashboard
    ports:
      - "8087:8087"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT=8087
    links:
      - discovery1
      - discovery2

  # 启动hystrix-turbine-mq  基于消息中间件的实时性能数据收集
  hystrix-turbine-mq:
    container_name: hystrix-turbine-mq
    image: 127.0.0.1:5000/billjiang/hystrix-turbine-mq
    ports:
      - "8089:8089"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT=8089
    links:
      - discovery1
      - discovery2

  mysql:
    container_name: mysql
    image: mysql:5.7
    environment:
      - MYSQL_ROOT_PASSWORD=passw0rd
      - MYSQL_DATABASE=foodb
      - MYSQL_USER=user1
      - MYSQL_PASSWORD=passw0rd


6. 在service目录下,创建服务示例的docker-compose.yml,编写如下内容

version: "2"
services:



  # 启动 hello
  hello:
    image: 127.0.0.1:5000/billjiang/hello
    ports:
      - "8000"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - MYSQL_DATABASE=foodb
      - MYSQL_USERNAME=user1
      - MYSQL_PASSWORD=passw0rd
    external_links:
      - mysql
      - config1
      - config2
      - gateway
      - track

  # 启动 world
  world:
    image: 127.0.0.1:5000/billjiang/world
    ports:
      - "8010"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT:8010
    external_links:
      - config1
      - config2
      - gateway
      - track

  # 启动 helloworld
  helloworld:
    image: 127.0.0.1:5000/billjiang/helloworld
    ports:
      - "8020"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT:8020
    external_links:
      - config1
      - config2
      - gateway
      - track
    links:
      - hello
      - world
  # 启动 hello
  helloworld-feign:
    image: 127.0.0.1:5000/billjiang/helloworld-feign
    ports:
      - "8030"
    environment:
      - EUREKA_SERVER_LIST=http://discovery1:8761/eureka/,http://discovery2:8762/eureka/
      - SPRING_PROFILES_ACTIVE=docker
      - SERVER_PORT=8030
    external_links:
      - config1
      - config2
      - gateway
      - track
    links:
      - hello
      - world

# 其他服务 ..........................
networks:
  default:
    external:
      name: springclouddemo_default

因为docker-compose启动的时候默认会创建一个springclouddemo_default的网络,两个网络要通信,所以服务示例指定使用springclouddemo_default,这样组件之间客户互相通信了。

6. 启动服务治理的docker-compose.yml

springcloud-demo根目录下,使用docker-compose up启动服务编排,如下界面所示:

服务治理服务编排启动1.png

服务治理服务编排启动2.png

启动后,打开注册中心地址,查看已经启动的服务:


已经启动的服务列表.png

7. 连接外部docker容器

配置中心使用到了gitlab, 服务跟踪track使用了rabbitmq,这两个容器都不在docker-compose之中,要使得能够互相连通,要把这两个容器也加入到springclouddemo_default网络。

docker network connect springclouddemo_default gitlab
docker network connect springclouddemo_default rabbitmq

使用docker network inspect springclouddemo_default可以查看到已经加入网络的容器。

8. 启动服务示例编排docker-compose.yml

进入到service目录,使用docker-compose up启动服务示例编排,如下所示:

服务示例编排1.png

服务示例编排2

9. 在Eureka Server查看新增的服务

新增注册服务.png

10. 可以使用如下命令对服务进行扩容

scale扩容.png

刷新Eureka Server,可以看到服务已经扩展到三个节点。

扩容后.png

推荐阅读更多精彩内容