openshift 3.11离线安装

环境描述

  • 介绍
    两个节点,一个master节点,另一个当做compute和infra节点,使用的操作系统为rhel 7.4,没有安装EFK、service broker、service catalog、metric,promethues在3.11正式GA,默认就会安装。因为本人有红帽的订阅账号,所以可以从红帽的源进行yum安装,需要提醒的是,从3.11开始,红帽官方的镜像仓库从registry.access.redhat.com变为registry.redhat.io,且拉取镜像也需要红帽的订阅账号了。
  • 主机信息,两个节点,infra和node复用同一台,openshift-cluster-test2.com为集群的入口
172.16.37.11 master.test2.com 
172.16.37.12 node.test2.com
172.16.37.12 infra.test2.com
172.16.37.11 openshift-cluster.test2.com 
  • 操作系统rhel 7.4,ocp 3.11;
yum 源搭建(172.16.37.12)
  • 本环境没有外部的DNS server作为域名解析,所以使用/etc/hosts进行主机的解析,在两台主机/etc/hosts添加如下解析,环境清单及/etc/hosts文件,操作系统为7.4
172.16.37.11 master.test2.com 
172.16.37.12 node.test2.com
172.16.37.12 infra.test2.com
172.16.37.11 openshift-cluster.test2.com
  • 配置selinux为enforcing:
[root@node ~]# cat /etc/selinux/config    
# This file controls the state of SELinux on the system.
# SELINUX= can take one of these three values:
#     enforcing - SELinux security policy is enforced.
#     permissive - SELinux prints warnings instead of enforcing.
#     disabled - No SELinux policy is loaded.
SELINUX=enforcing
# SELINUXTYPE= can take one of three two values:
#     targeted - Targeted processes are protected,
#     minimum - Modification of targeted policy. Only selected processes are protected.
#     mls - Multi Level Security protection.
SELINUXTYPE=targeted
  • 上传openshift RPM包至节点,并搭建为yum源
[root@node ~]# tar -zxvf ocp311rpms.tar.gz -C /opt/repos/
[root@node ~]# ll /opt/repos/
总用量 0
drwxr-xr-x. 4 root root 55 Oct 19 23:50 rhel-7-server-ansible-2.6-rpms
drwxr-xr-x. 4 root root 55 Oct 19 23:50 rhel-7-server-extras-rpms
drwxr-xr-x. 4 root root 55 Oct 20 11:04 rhel-7-server-ose-3.11-rpms
drwxr-xr-x. 4 root root 55 Oct 19 23:48 rhel-7-server-rpms
  • 创建本机yum源:
[root@node ~]# cat /etc/yum.repos.d/ocp-local.repo
[rhel-7-server-rpms]
name=rhel-7-server-rpms
baseurl=file:///opt/repos/rhel-7-server-rpms
enabled=1
gpgcheck=0
[rhel-7-server-extras-rpms]
name=rhel-7-server-extras-rpms
baseurl=file:///opt/repos/rhel-7-server-extras-rpms
enabled=1
gpgcheck=0
[rhel-7-server-ose-3.11-rpms]
name=rhel-7-server-ose-3.11-rpms
baseurl=file:///opt/repos/rhel-7-server-ose-3.11-rpms
enabled=1
gpgcheck=0
[rhel-7-server-ansible-2.6-rpms]
name=rhel-7-server-ansible-2.6-rpms
baseurl=file:///opt/repos/rhel-7-server-ansible-2.6-rpms
enabled=1
gpgcheck=0
  • 查看yum源是否正常
[root@node ~]# yum repolist
repolist: 5625
  • 检查yum源中openshift rpm内容是否正确
[root@node ~]# yum list|grep atomic-openshift | wc -l
20
  • 将本机源设置为yum服务器,安装httpd
yum -y install httpd
  • 配置httpd:/etc/httpd/conf.d/yum.conf
Alias /repos "/opt/repos"
<Directory "/opt/repos">
  Options +Indexes +FollowSymLinks
  Require all granted
</Directory>
<Location /repos>
SetHandler None
 </Location>
  • 修改httpd的服务为81端口,防止与router的80端口冲突
[root@node ~]# vi /etc/httpd/conf/httpd.conf
Listen 81
  • 启动httpd
systemctl enable httpd;systemctl restart httpd
  • 检查httpd是否配置正确,正确的话能读取到rpm的内容
[root@node ~]# curl 172.16.37.12:81/repos/
  • 配置yum文件ocp.repo,将原先的ocp-local.repo修改为ocp-local.repo.bak1
[root@node ~]# mv ocp-local.repo ocp-local.repo.bak1
[root@node ~]# cat /etc/yum.repos.d/ocp.repo
[rhel-7-server-rpms]
name=rhel-7-server-rpms
baseurl=http://172.16.37.12:81/repos/rhel-7-server-rpms
enabled=1
gpgcheck=0
[rhel-7-server-extras-rpms]
name=rhel-7-server-extras-rpms
baseurl=http://172.16.37.12:81/repos/rhel-7-server-extras-rpms
enabled=1
gpgcheck=0
[rhel-7-server-ose-3.11-rpms]
name=rhel-7-server-ose-3.11-rpms
baseurl=http://172.16.37.12:81/repos/rhel-7-server-ose-3.11-rpms
enabled=1
gpgcheck=0
[rhel-7-server-ansible-2.6-rpms]
name=rhel-7-server-ansible-2.6-rpms
baseurl=http://172.16.37.12:81/repos/rhel-7-server-ansible-2.6-rpms
enabled=1
gpgcheck=0
  • 检查yum源服务器是否配置完成
[root@node ~]# yum clean all;yum repolist
  • 创建免密登录并且分发密钥至其他节点
ssh-keygen;
for i in  master.test2.com  node.test2.com infra.test2.com
do
echo $i
ssh-copy-id $i;
done;
  • 脚本批量修改主机名
for i in  master.test2.com  node.test2.com 
do
echo $i
ssh $i hostnamectl "set-hostname $i";
done;
  • 安装iptabes服务
yum install iptables-services -y
  • 配置iptables
cp /etc/sysconfig/iptables /etc/sysconfig/iptables.bak.$(date "+%Y%m%d%H%M%S");
sed -i '/.*--dport 22 -j ACCEPT.*/a\-A INPUT -p tcp -m state --state NEW -m tcp --dport 53 -j ACCEPT' /etc/sysconfig/iptables;
sed -i '/.*--dport 22 -j ACCEPT.*/a\-A INPUT -p udp -m state --state NEW -m udp --dport 53 -j ACCEPT' /etc/sysconfig/iptables;
sed -i '/.*--dport 22 -j ACCEPT.*/a\-A INPUT -p tcp -m state --state NEW -m tcp --dport 5000 -j ACCEPT' /etc/sysconfig/iptables;
sed -i '/.*--dport 22 -j ACCEPT.*/a\-A INPUT -p tcp -m state --state NEW -m tcp --dport 81 -j ACCEPT' /etc/sysconfig/iptables;
systemctl restart iptables;systemctl enable iptables
  • 安装openshift-ansible
yum install openshift-ansible -y
  • 配置ansible hosts文件:/etc/ansible/hosts,注意修改cluster域名地址和主机域名
[root@node ~]# cat /etc/ansible/hosts
# Create an OSEv3 group that contains the masters, nodes, and etcd groups
[OSEv3:children]
masters
nodes
etcd
    
# Set variables common for all OSEv3 hosts
[OSEv3:vars]
# SSH user, this user should allow ssh based auth without requiring a password
ansible_ssh_user=root
openshift_deployment_type=openshift-enterprise
openshift_image_tag=v3.11.16
# If ansible_ssh_user is not root, ansible_become must be set to true
#ansible_become=true
    
    
# default selectors for router and registry services
# openshift_router_selector='node-role.kubernetes.io/infra=true'
# openshift_registry_selector='node-role.kubernetes.io/infra=true'
    
# uncomment the following to enable htpasswd authentication; defaults to DenyAllPasswordIdentityProvider
openshift_master_identity_providers=[{'name': 'htpasswd_auth', 'login': 'true', 'challenge': 'true', 'kind': 'HTPasswdPasswordIdentityProvider'}]
openshift_master_default_subdomain=test2.com
openshift_disable_check=memory_availability,disk_availability,docker_image_availability
    
os_sdn_network_plugin_name=redhat/openshift-ovs-multitenant
    
openshift_master_cluster_method=native
openshift_master_cluster_hostname=openshift-cluster.test2.com
openshift_master_cluster_public_hostname=openshift-cluster.test2.com
# false
ansible_service_broker_install=false
openshift_enable_service_catalog=false
template_service_broker_install=false
openshift_logging_install_logging=false
    
# registry passwd
oreg_url=172.16.37.12:5000/openshift3/ose-${component}:${version}
openshift_examples_modify_imagestreams=true
    
# docker config
openshift_docker_additional_registries=172.16.37.12:5000,172.30.0.0/16
openshift_docker_insecure_registries=172.16.37.12:5000,172.30.0.0/16
#openshift_docker_blocked_registries
openshift_docker_options="--log-driver json-file --log-opt max-size=1M --log-opt max-file=3"
    
# openshift_cluster_monitoring_operator_install=false
# openshift_metrics_install_metrics=true
# openshift_enable_unsupported_configurations=True
#openshift_logging_es_nodeselector='node-role.kubernetes.io/infra: "true"'
#openshift_logging_kibana_nodeselector='node-role.kubernetes.io/infra: "true"'
# host group for masters
    
[masters]
master.test2.com
    
# host group for etcd
[etcd]
master.test2.com
    
# host group for nodes, includes region info
[nodes]
master.test2.com openshift_node_group_name='node-config-master'
node.test2.com openshift_node_group_name='node-config-compute'
infra.test2.com openshift_node_group_name='node-config-infra'
  • 分发/etc/hosts文件至其他节点
ansible all -m copy -a "src=/etc/hosts dest=/etc/hosts "
  • 分发yum配置到各节点
ansible all -m copy -a "src=/etc/yum.repos.d/ocp.repo dest=/etc/yum.repos.d/ocp.repo"
  • 查看yum源是否正常
for i in  master.test2.com  node.test2.com
do
echo $i
ssh $i 'yum repolist'
done;

所有节点基础包安装

  • 基础包
yum install wget git net-tools bind-utils yum-utils iptables-services bridge-utils bash-completion kexec-tools sos psacct vim lrzsz python-setuptools unzip tree docker-1.13.1 -y
  • 更新
yum -y update

配置docker(172.16.37.12)

  • 配置/etc/containers/registries.conf为如下内容
# This is a system-wide configuration file used to
# keep track of registries for various container backends.
# It adheres to TOML format and does not support recursive
# lists of registries.
    
 # The default location for this configuration file is /etc/containers/registries.conf.
    
# The only valid categories are: 'registries.search', 'registries.insecure',
# and 'registries.block'.
    
[registries.search]
registries = ['172.16.37.12:5000']
    
# If you need to access insecure registries, add the registry's fully-qualified name.
# An insecure registry is one that does not have a valid SSL certificate or only does HTTP.
[registries.insecure]
registries = ['172.16.37.12:5000']
    
# If you need to block pull access from a registry, uncomment the section below
# and add the registries fully-qualified name.
#
# Docker only
[registries.block]
registries = ['registry.access.redhat.com']
  • 分发/etc/containers/registries.conf文件至其他节点
ansible all -m copy -a "src=/etc/containers/registries.conf dest=/etc/containers/registries.conf "
  • 启动docker服务
ansible all -a 'systemctl start docker';ansible all -a 'systemctl enable docker'
  • 配置iptables,放行流量
ansible all -m copy -a "src=/etc/sysconfig/iptables dest=/etc/sysconfig/iptables"
ansible all -a "systemctl restart iptables"

镜像仓库搭建(172.16.37.12)

  • 安装docker-distribution
yum install -y docker-distribution;systemctl start docker-distribution;systemctl enable docker-distribution
  • 上传镜像包并导入,此过程会占用大量磁盘空间和内存资源
docker load -i docker load -i 3.11-images.tar.bz2
  • 修改镜像tag
docker images|grep registry.redhat.io|awk '{print $1"/"$2}'|awk -F "/" '{print "docker tag "$1"/"$2"/"$3":"$4" 172.16.37.12:5000/"$2"/"$3":"$4}'|sh
  • 上传镜像至镜像仓库
docker images|grep 172.16.37.12:5000|awk '{print "docker push " $1":"$2}'|sh

OpenShift部署节点组件(172.16.37.12)

  • 部署前检查操作
ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/prerequisites.yml
  • 开始部署
ansible-playbook /usr/share/ansible/openshift-ansible/playbooks/deploy_cluster.yml
  • 卸载(如有需要则执行)
ansible-playbook  /usr/share/ansible/openshift-ansible/playbooks/adhoc/uninstall.yml

部署结束后

  • 创建用户(master1)
htpasswd -cb /etc/origin/master/htpasswd admin 123456
oc adm policy add-cluster-role-to-user cluster-admin admin
  • 登录页面
配置好浏览器所在机子的hosts文件里面域名对应master的IP,然后在浏览器输入地址:
https://openshift-cluster.test2.com:8443,输入用户名密码
最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 151,688评论 1 330
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 64,559评论 1 273
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 101,749评论 0 226
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 42,581评论 0 191
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 50,741评论 3 271
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 39,684评论 1 192
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 31,122评论 2 292
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 29,847评论 0 182
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 33,441评论 0 228
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 29,939评论 2 232
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 31,333评论 1 242
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 27,783评论 2 236
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 32,275评论 3 220
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 25,830评论 0 8
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 26,444评论 0 180
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 34,553评论 2 249
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 34,618评论 2 249