前言：Kubernetes 是Google开源的容器集群管理系统，它构建于docker技术之上，基于Docker构建一个容器的调度服务，提供资源调度、均衡容灾、服务注册、动态扩缩容等功能套件，本质上可看作是基于容器技术的mini-PaaS平台。本文介绍如何基于Centos7.2构建Kubernetes多节点平台，了解 Kubernetes 的基本概念和多节点部署架构将使安装和管理更容易。建议在继续前进之前，查阅一些Kubernetes 介绍文档,这里就不对k8s的相关概念做介绍了。
** 截止至2015年9月1日，CentOS 已经把 kubernetes 加入官方源，所以现在安装k8s的各组件只需yum一下 **
一、环境说明
1. 各组件版本
- Kubernetes-1.2.0
- docker-1.10.3
- flannel-0.5.3
- etcd-2.3.7
- go-1.6.3
2. kubernetes 集群架构图：
k8s集群架构图
3. kubernetes 各角色：
- 1.1.1.100 master/etc/
- 1.1.1.101 minion(node)
- 1.1.1.102 minion(node)
以下均用IP进行操作，但是建议部署的时候，建议用hosts或者DNS取代IP。
4. 各组件用途
kube maste：
- kube-apiserver
k8s的管理接口
- kube-scheduer
k8s调度器，容器的启动、迁移、扩容缩减时候，选择哪个node，就看它了。
- kube-controller-manager
k8s对node的控制行为，比如怎么去调用node启动一个容器。
kube minion(node)：
- kubelet
负责node的管理，基本所有操作都靠它。
- kube-proxy
每个node里的container都在一个私有网络中，kube-proxy的作用就是做一个反向代理，让访问者访问这个node的时候，可以转发到内部对应的container。
Flannel：
实现多台容器跨主机网络通信
（这里和minion放在一起）
etcd：
作为kubernetes的数据库，存储了k8s自身的信息、以及各种业务容器信息等。
存储flannel网络配置信息，供各节点协调。
（这里和master放在一起）
Tips: 以上组件介绍来自 方云麟的博客。
二、安装和配置
1. 配置 master kubernetes
- 安装
yum install -y docker kubernetes-master etcd
- egrep -v '^#' /etc/etcd/etcd.conf
ETCD_NAME=1.1.1.100
ETCD_DATA_DIR="/var/lib/etcd/default.etcd"
ETCD_LISTEN_PEER_URLS="http://1.1.1.100:2380"
ETCD_LISTEN_CLIENT_URLS="http://1.1.1.100:2379","http://127.0.0.1:2379"
ETCD_INITIAL_ADVERTISE_PEER_URLS="http://1.1.1.100:2380"
ETCD_INITIAL_CLUSTER="1.1.1.100=http://1.1.1.100:2380"
ETCD_INITIAL_CLUSTER_STATE="new"
ETCD_INITIAL_CLUSTER_TOKEN="etcd-cluster"
ETCD_ADVERTISE_CLIENT_URLS="http://1.1.1.100:2379"
- cat /etc/kubernetes/config
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://1.1.1.100:8080"
- cat /etc/kubernetes/apiserver
###
# kubernetes system config
#
# The following values are used to configure the kube-apiserver
#
# The address on the local server to listen to.
KUBE_API_ADDRESS="--insecure-bind-address=0.0.0.0"
# The port on the local server to listen on.
# KUBE_API_PORT="--port=8080"
# Port minions listen on
# KUBELET_PORT="--kubelet-port=10250"
# Comma separated list of nodes in the etcd cluster
KUBE_ETCD_SERVERS="--etcd-servers=http://1.1.1.100:2379"
# Address range to use for services
KUBE_SERVICE_ADDRESSES="--service-cluster-ip-range=10.254.0.0/16"
# default admission control policies
KUBE_ADMISSION_CONTROL="--admission-control=NamespaceLifecycle,NamespaceExists,LimitRanger,SecurityContextDeny,ServiceAccount,ResourceQuota"
# Add your own!
KUBE_API_ARGS=""
- cat /etc/kubernetes/controller-manager
###
# The following values are used to configure the kubernetes controller-manager
# defaults from config and apiserver should be adequate
# Add your own!
KUBE_CONTROLLER_MANAGER_ARGS="--node-monitor-grace-period=10s --pod-eviction-timeout=10s"
- 启动 master,并查看状态
for service in etcd kube-apiserver kube-controller-manager kube-scheduler; do
systemctl enable $service
systemctl restart $service
systemctl status $service
done
2. 配置 minion kubernetes
- 安装
yum install -y docker kubernetes-node flannel
- cat /etc/kubernetes/config
# logging to stderr means we get it in the systemd journal
KUBE_LOGTOSTDERR="--logtostderr=true"
# journal message level, 0 is debug
KUBE_LOG_LEVEL="--v=0"
# Should this cluster be allowed to run privileged docker containers
KUBE_ALLOW_PRIV="--allow-privileged=false"
# How the controller-manager, scheduler, and proxy find the apiserver
KUBE_MASTER="--master=http://1.1.1.100:8080"
- cat /etc/kubernetes/kubelet
# The address for the info server to serve on (set to 0.0.0.0 or "" for all interfaces)
KUBELET_ADDRESS="--address=127.0.0.1"
# The port for the info server to serve on
# KUBELET_PORT="--port=10250"
# You may leave this blank to use the actual hostname，
#下面ip是个minion的ip
KUBELET_HOSTNAME="--hostname-override=1.1.1.101"
# location of the api-server
KUBELET_API_SERVER="--api-servers=http://1.1.1.100:8080"
# pod infrastructure container
#KUBELET_POD_INFRA_CONTAINER="--pod-infra-container-image=registry.access.redhat.com/rhel7/pod-infrastructure:latest"
# Add your own!
KUBELET_ARGS="--pod-infra-container-image=kubernetes/pause"
3. 配置flannel
- 【master】初始化flannel的etcd配置
etcdctl -C 1.1.1.100:2379 set /coreos.com/network/config '{ "Network": "10.1.0.0/16" }'
- 【minion】cat /etc/sysconfig/flanneld
# etcd url location. Point this to the server where etcd runs
FLANNEL_ETCD="http://1.1.1.100:2379"
# etcd config key. This is the configuration key that flannel queries
# For address range assignment
FLANNEL_ETCD_KEY="/coreos.com/network"
- 启动
for SERVICES in kube-proxy kubelet docker flanneld; do
systemctl restart $SERVICES
systemctl enable $SERVICES
systemctl status $SERVICES
done
- 查看状态
minion
> ip a | grep flannel | grep inet
inet 10.1.37.0/16 scope global flannel0
> ip a | grep flannel | grep inet
inet 10.1.42.0/16 scope global flannel0
master
> etcdctl ls /coreos.com/network/subnets
/coreos.com/network/subnets/10.1.42.0-24
/coreos.com/network/subnets/10.1.37.0-24
三、验证
1. 在 master 上查看 node
kubectl get nodes
NAME STATUS AGE
1.1.1.101 Ready 4m
1.1.1.102 Ready 6m
Tips：
#删除node
kubectl delete node 2.2.2.101
2. 使用pods
- 创建 pods
# 为了创建一个pod，我们需要在kubernetes master上面定义一个yaml 或者 json配置文件。然后使用kubectl命令创建pod
mkdir -p /data/pods
cd /data/pods/
cat nginx.yaml
apiVersion: v1
kind: Pod
metadata:
name: nginx
labels:
app: nginx
spec:
containers:
- name: nginx
# 这个地方我写的是我的私有仓库镜像，所以minion端的镜像配置也要改
image: 1.1.1.100:500/nginx
ports:
- containerPort: 80
- 创建pod
kubectl create -f nginx.yaml
- 查看pod
> kubectl get pod nginx
NAME READY STATUS RESTARTS AGE
nginx 0/1 ContainerCreating 0 14s
> kubectl get pod -o wide
NAME READY STATUS RESTARTS AGE NODE
nginx 0/1 ContainerCreating 0 9m 1.1.1.102
- 查看详情
> kubectl describe pod nginx
Name: nginx
Namespace: default
Node: 1.1.1.101/1.1.1.101
Start Time: Thu, 22 Sep 2016 17:46:18 +0800
Labels: app=nginx
Status: Pending
IP:
Controllers: <none>
Containers:
nginx:
Container ID:
Image: 1.1.1.100:500/nginx
Image ID:
Port: 80/TCP
QoS Tier:
cpu: BestEffort
memory: BestEffort
State: Waiting
Reason: ContainerCreating
Ready: False
Restart Count: 0
Environment Variables:
Conditions:
Type Status
Ready False
No volumes.
Events:
FirstSeen LastSeen Count From SubobjectPath Type Reason Message
--------- -------- ----- ---- ------------- -------- ------ -------
7s 7s 1 {default-scheduler } Normal Scheduled Successfully assigned nginx to 1.1.1.102
# 查看详情可用于排错；
- 自此，kubernetes初步搭建完成，后续会介绍一些它的具体使用。。。
本文参考1：http://severalnines.com/blog/installing-kubernetes-cluster-minions-centos7-manage-pods-services
本文参考2：http://www.sunmite.com/linux/installing-kubernetes-cluster-on-centos7-to-manage-pods-and-services/#comment-564
