安装Ovirt

1 安装操作系统CentOS7.7

通过ipmitool 进入peiboot :

ipmitool -H 10.3.3.20 -I lanplus -P admin sol activate

选择exit to shell 执行下面的命令

ip addr add 172.16.167.161/24 dev enP3p9s0f0

mkdir -p /n/centos7.7/ppc/ppc64

cd /n/centos7.7/ppc/ppc64

wgethttp://172.16.167.170:8088/centos7.7/ppc/ppc64/initrd.img

wgethttp://172.16.167.170:8088/centos7.7/ppc/ppc64/vmlinuz

kexec -l /n/centos7.7/ppc/ppc64/vmlinuz --initrd=/n/centos7.7/ppc/ppc64/initrd.img --append="ip=172.16.167.182::172.16.167.1:255.255.255.0::enP3p9s0f1:none inst.stage2=nfs:172.16.167.170:/export.nfs/centos7.7:ro,nolock inst.vnc inst.vncpassword=123456" --exec

执行命令后显示如下，开始重启安装

使用VNC客户端连接，进行安装

2 安装完后配置操作系统

2.1 配置主机名

hostnamectl set-hostname ovirt-engine

2.2 关闭防火墙

[root@ovirt-engine ~]# systemctl stop firewalld

[root@ovirt-engine ~]# systemctl disable firewalld

Removed symlink /etc/systemd/system/multi-user.target.wants/firewalld.service.

Removed symlink /etc/systemd/system/dbus-org.fedoraproject.FirewallD1.service.

2.3 关闭selinux

编辑文件：/etc/selinux/config

# This file controls the state of SELinux on the system.

# SELINUX= can take one of these three values:

# enforcing - SELinux security policy is enforced.

# permissive - SELinux prints warnings instead of enforcing.

# disabled - No SELinux policy is loaded.

SELINUX=disabled

# SELINUXTYPE= can take one of three values:

# targeted - Targeted processes are protected,

# minimum - Modification of targeted policy. Only selected processes are protected.

# mls - Multi Level Security protection.

SELINUXTYPE=targeted

3 ovirt-engine安装

3.1 更新操作系统

yum update -y

更新后重启

3.2 安装ovirt-engine源

yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

3.3 安装ovirt-engine

yum install ovirt-engine

出现如下错误

Error: Package: ovirt-engine-4.3.9.4-1.el7.noarch (ovirt-4.3)

Requires: rh-postgresql10-postgresql-server

Error: Package: ovirt-engine-dwh-setup-4.3.8-1.el7.noarch (ovirt-4.3)

Requires: rh-postgresql10-postgresql-server

Error: Package: ovirt-engine-dwh-setup-4.3.8-1.el7.noarch (ovirt-4.3)

Requires: rh-postgresql10-postgresql-contrib

Error: Package: ovirt-engine-dwh-4.3.8-1.el7.noarch (ovirt-4.3)

Requires: rh-postgresql10-postgresql-contrib

Error: Package: ovirt-engine-tools-backup-4.3.9.4-1.el7.noarch (ovirt-4.3)

Requires: rh-postgresql10-postgresql

Error: Package: ovirt-engine-setup-plugin-ovirt-engine-4.3.9.4-1.el7.noarch (ovirt-4.3)

Requires: rh-postgresql10-postgresql-contrib

Error: Package: ovirt-engine-4.3.9.4-1.el7.noarch (ovirt-4.3)

Requires: rh-postgresql10-postgresql-contrib

Error: Package: ovirt-engine-dwh-4.3.8-1.el7.noarch (ovirt-4.3)

Requires: rh-postgresql10-postgresql-server

Error: Package: ovirt-engine-wildfly-overlay-17.0.1-1.el7.noarch (ovirt-4.3)

Requires: ovirt-engine-wildfly = 17.0.1-1.el7

Error: Package: ovirt-engine-setup-plugin-ovirt-engine-4.3.9.4-1.el7.noarch (ovirt-4.3)

Requires: rh-postgresql10-postgresql-server

先安装

yum install centos-release-scl

手工下载

rh-postgresql10-runtime-3.1-1.bs1.el7.ppc64le.rpm 包

然后执行安装

yum install rh-postgresql10-postgresql*

手工下载ovirt-engine-wildfly

然后编译rpm包

安装

ovirt-engine-wildfly-17.0.1-1.el7.ppc64le.rpm

ovirt-engine-wildfly-overlay-17.0.1-1.el7.noarch.rpm

yum install ovirt-engine-wildfly-17.0.1-1.el7.ppc64le.rpm ovirt-engine-wildfly-overlay-17.0.1-1.el7.noarch.rpm

3.4 配置ovirt-engine

[root@ovirt-engine ~]# engine-setup

[ INFO ] Stage: Initializing

[ INFO ] Stage: Environment setup

Configuration files: ['/etc/ovirt-engine-setup.conf.d/10-packaging-jboss.conf', '/etc/ovirt-engine-setup.conf.d/10-packaging.conf']

Log file: /var/log/ovirt-engine/setup/ovirt-engine-setup-20200512132148-vxnxxp.log

Version: otopi-1.8.4 (otopi-1.8.4-1.el7)

[ INFO ] Stage: Environment packages setup

[ INFO ] Stage: Programs detection

[ INFO ] Stage: Environment setup (late)

[ INFO ] Stage: Environment customization

--== PRODUCT OPTIONS ==--

Set up Cinderlib integration

(Currently in tech preview)

(Yes, No) [No]:

Configure Engine on this host (Yes, No) [Yes]:

Configure ovirt-provider-ovn (Yes, No) [Yes]:

Configure WebSocket Proxy on this host (Yes, No) [Yes]:

* Please note * : Data Warehouse is required for the engine.

If you choose to not configure it on this host, you have to configure

it on a remote host, and then configure the engine on this host so

that it can access the database of the remote Data Warehouse host.

Configure Data Warehouse on this host (Yes, No) [Yes]:

Configure Image I/O Proxy on this host (Yes, No) [Yes]:

Configure VM Console Proxy on this host (Yes, No) [Yes]:

--== PACKAGES ==--

[ INFO ] Checking for product updates...

[ INFO ] No product updates found

--== NETWORK CONFIGURATION ==--

Host fully qualified DNS name of this server [ovirt-engine]:

[WARNING] Host name ovirt-engine has no domain suffix

[WARNING] Failed to resolve ovirt-engine using DNS, it can be resolved only locally

Setup can automatically configure the firewall on this system.

Note: automatic configuration of the firewall may overwrite current settings.

NOTICE: iptables is deprecated and will be removed in future releases

Do you want Setup to configure the firewall? (Yes, No) [Yes]: No

[WARNING] Host name ovirt-engine has no domain suffix

[WARNING] Host name ovirt-engine has no domain suffix

[WARNING] Host name ovirt-engine has no domain suffix

--== DATABASE CONFIGURATION ==--

Where is the DWH database located? (Local, Remote) [Local]:

Setup can configure the local postgresql server automatically for the DWH to run. This may conflict with existing applications.

Would you like Setup to automatically configure postgresql and create DWH database, or prefer to perform that manually? (Automatic, Manual) [Automatic]:

Where is the Engine database located? (Local, Remote) [Local]:

Setup can configure the local postgresql server automatically for the engine to run. This may conflict with existing applications.

Would you like Setup to automatically configure postgresql and create Engine database, or prefer to perform that manually? (Automatic, Manual) [Automatic]:

--== OVIRT ENGINE CONFIGURATION ==--

Engine admin password:

Confirm engine admin password:

[WARNING] Password is weak: The password is shorter than 8 characters

Use weak password? (Yes, No) [No]: Yes

Application mode (Virt, Gluster, Both) [Both]:

Use default credentials (admin@internal) for ovirt-provider-ovn (Yes, No) [Yes]:

--== STORAGE CONFIGURATION ==--

Default SAN wipe after delete (Yes, No) [No]:

--== PKI CONFIGURATION ==--

Organization name for certificate [Test]: ovirt-engine

--== APACHE CONFIGURATION ==--

Setup can configure the default page of the web server to present the application home page. This may conflict with existing applications.

Do you wish to set the application as the default page of the web server? (Yes, No) [Yes]:

Setup can configure apache to use SSL using a certificate issued from the internal CA.

Do you wish Setup to configure that, or prefer to perform that manually? (Automatic, Manual) [Automatic]:

--== SYSTEM CONFIGURATION ==--

--== MISC CONFIGURATION ==--

Please choose Data Warehouse sampling scale:

(1) Basic

(2) Full

(1, 2)[1]:

--== END OF CONFIGURATION ==--

[ INFO ] Stage: Setup validation

--== CONFIGURATION PREVIEW ==--

Application mode : both

Default SAN wipe after delete : False

Update Firewall : False

Host FQDN : ovirt-engine

Set up Cinderlib integration : False

Configure local Engine database : True

Set application as default page : True

Configure Apache SSL : True

Engine database secured connection : False

Engine database user name : engine

Engine database name : engine

Engine database host : localhost

Engine database port : 5432

Engine database host name validation : False

Engine installation : True

PKI organization : ovirt-engine

Set up ovirt-provider-ovn : True

Configure WebSocket Proxy : True

DWH installation : True

DWH database host : localhost

DWH database port : 5432

Configure local DWH database : True

Configure Image I/O Proxy : True

Configure VMConsole Proxy : True

Please confirm installation settings (OK, Cancel) [OK]:

[ INFO ] Stage: Transaction setup

[ INFO ] Stopping engine service

[ INFO ] Stopping ovirt-fence-kdump-listener service

[ INFO ] Stopping dwh service

[ INFO ] Stopping Image I/O Proxy service

[ INFO ] Stopping vmconsole-proxy service

[ INFO ] Stopping websocket-proxy service

[ INFO ] Stage: Misc configuration (early)

[ INFO ] Stage: Package installation

[ INFO ] Stage: Misc configuration

[ INFO ] Upgrading CA

[ INFO ] Initializing PostgreSQL

[ INFO ] Creating PostgreSQL 'engine' database

[ INFO ] Configuring PostgreSQL

[ INFO ] Creating PostgreSQL 'ovirt_engine_history' database

[ INFO ] Configuring PostgreSQL

[ INFO ] Creating CA

[ INFO ] Creating/refreshing DWH database schema

[ INFO ] Configuring Image I/O Proxy

[ INFO ] Setting up ovirt-vmconsole proxy helper PKI artifacts

[ INFO ] Setting up ovirt-vmconsole SSH PKI artifacts

[ INFO ] Configuring WebSocket Proxy

[ INFO ] Creating/refreshing Engine database schema

[ INFO ] Creating/refreshing Engine 'internal' domain database schema

[ INFO ] Creating default mac pool range

[ INFO ] Adding default OVN provider to database

[ INFO ] Adding OVN provider secret to database

[ INFO ] Setting a password for internal user admin

[ INFO ] Generating post install configuration file '/etc/ovirt-engine-setup.conf.d/20-setup-ovirt-post.conf'

[ INFO ] Stage: Transaction commit

[ INFO ] Stage: Closing up

[ INFO ] Starting engine service

[ INFO ] Starting dwh service

[ INFO ] Restarting ovirt-vmconsole proxy service

--== SUMMARY ==--

[ INFO ] Restarting httpd

In order to configure firewalld, copy the files from

/etc/ovirt-engine/firewalld to /etc/firewalld/services

and execute the following commands:

firewall-cmd --permanent --add-service ovirt-postgres

firewall-cmd --permanent --add-service ovirt-https

firewall-cmd --permanent --add-service ovn-central-firewall-service

firewall-cmd --permanent --add-service ovirt-fence-kdump-listener

firewall-cmd --permanent --add-service ovirt-imageio-proxy

firewall-cmd --permanent --add-service ovirt-websocket-proxy

firewall-cmd --permanent --add-service ovirt-http

firewall-cmd --permanent --add-service ovirt-vmconsole-proxy

firewall-cmd --permanent --add-service ovirt-provider-ovn

firewall-cmd --reload

The following network ports should be opened:

tcp:2222

tcp:35357

tcp:443

tcp:5432

tcp:54323

tcp:6100

tcp:6641

tcp:6642

tcp:80

tcp:9696

udp:7410

An example of the required configuration for iptables can be found at:

/etc/ovirt-engine/iptables.example

Please use the user 'admin@internal' and password specified in order to login

Web access is enabled at:

http://ovirt-engine:80/ovirt-engine

127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomain4

https://ovirt-engine:443/ovirt-engine

[main]

Internal CA F8:C2:EB:94:61:FA:2E:8C:CD:14:AC:68:7C:55:11:88:F4:2B:EE:AF

SSH fingerprint: SHA256:MsSi0pSL6lfy2kEn0X1W0MTDVr5RPcHFoa3AQ4R+geM

--== END OF SUMMARY ==--

[ INFO ] Stage: Clean up

Log file is located at /var/log/ovirt-engine/setup/ovirt-engine-setup-20200512132148-vxnxxp.log

[ INFO ] Generating answer file '/var/lib/ovirt-engine/setup/answers/20200512132913-setup.conf'

[ INFO ] Stage: Pre-termination

[ INFO ] Stage: Termination

[ INFO ] Execution of setup completed successfully

添加计算节点

1 操作系统安装

2 更新操作系统

yum update -y

更新后重启

3 安装ovirt-engine源

yum install https://resources.ovirt.org/pub/yum-repo/ovirt-release43.rpm

yum install centos-release-scl

4 在ovirt engine web ui中添加host

在主机的子菜单下选择新建主机

填写主机配置信息

成功添加后，会自动安装node节点，一切都正常结束之后，会显示主机状态为up。

glusterfs 安装

1 每台机器

运行

yum install glusterfs-server glusterfs-geo-replication vdsm-gluster

2 启动

service glusterd start

3 检查状态

[root@ovirt-node01 ~]# service glusterd status

Redirecting to /bin/systemctl status glusterd.service

● glusterd.service - GlusterFS, a clustered file-system server

Loaded: loaded (/usr/lib/systemd/system/glusterd.service; disabled; vendor preset: disabled)

Active: active (running) since Wed 2020-05-13 23:35:19 CST; 37s ago

Docs: man:glusterd(8)

Process: 36041 ExecStart=/usr/sbin/glusterd -p /var/run/glusterd.pid --log-level $LOG_LEVEL $GLUSTERD_OPTIONS (code=exited, status=0/SUCCESS)

Main PID: 36042 (glusterd)

Tasks: 9

CGroup: /system.slice/glusterd.service

└─36042 /usr/sbin/glusterd -p /var/run/glusterd.pid --log-level INFO

May 13 23:35:15 ovirt-node01 systemd[1]: Starting GlusterFS, a clustered file-system server...

May 13 23:35:19 ovirt-node01 systemd[1]: Started GlusterFS, a clustered file-system server.

3 在其中一台机器上执行

[root@ovirt-engine ~]# gluster peer probe ovirt-node01

peer probe: success.

[root@ovirt-engine ~]# gluster peer probe ovirt-node02

peer probe: success.

4 查看集群状态

[root@ovirt-engine ~]# gluster peer status

Number of Peers: 2

Hostname: ovirt-node01

Uuid: e557b235-2656-4ff0-8896-46d2912ad910

State: Peer in Cluster (Connected)

Hostname: ovirt-node02

Uuid: b2073777-8bc6-4534-a78b-5b51a8146dd8

State: Peer in Cluster (Connected)

[root@ovirt-node02 ~]# gluster peer status

Number of Peers: 2

Hostname: ovirt-engine

Uuid: 4a48a513-696b-4b11-8c20-08683b839734

State: Peer in Cluster (Connected)

Hostname: ovirt-node01

Uuid: e557b235-2656-4ff0-8896-46d2912ad910

State: Peer in Cluster (Connected)

5 创建卷

gluster volume create vmios_volumes replica 3 ovirt-engine:/vmdata/ios ovirt-node01:/vmdata/ios ovirt-node02:/vmdata/ios

6 激活卷

gluster volume start vmios_volumes

7 开启NFS支持

yum install centos-release-gluster6

yum install nfs-ganesha nfs-ganesha-gluster

service nfs-ganesha start

参考https://docs.gluster.org/en/latest/Administrator%20Guide/NFS-Ganesha%20GlusterFS%20Integration/

在github下载相应脚本

分别执行下面步骤

1 修改配置文件/etc/ganesha/ganesha.conf

EXPORT{

Export_Id = 1 ; # Export ID unique to each export

Path = "volume_path"; # Path of the volume to be exported. Eg: "/test_volume"

FSAL {

name = GLUSTER;

hostname = "10.xx.xx.xx"; # IP of one of the nodes in the trusted pool

volume = "volume_name"; # Volume name. Eg: "test_volume"

}

Access_type = RW; # Access permissions

Squash = No_root_squash; # To enable/disable root squashing

Disable_ACL = TRUE; # To enable/disable ACL

Pseudo = "pseudo_path"; # NFSv4 pseudo path for this export. Eg: "/test_volume_pseudo"

Protocols = "3","4" ; # NFS protocols supported

Transports = "UDP","TCP" ; # Transport protocols supported

SecType = "sys"; # Security flavors supported

}

2 github下载脚本https://github.com/gluster/glusterfs/tree/release-3.10/extras/ganesha/scripts

然后执行

[root@ovirt-node01 ganesha]# ./create-export-ganesha.sh /etc/ganesha on vmios_volumes

[root@ovirt-node01 ganesha]# ./dbus-send.sh /etc/ganesha on vmios_volumes

3 Turn on features.cache-invalidation for that volume - gluster volume set \<volume name> features.cache-invalidation on

4检查是否挂载上

#showmount -e localhost

在ovirt管理界面添加glusterfs和gluster nfs类型如果遇到下面错误

Error while executing action Add Storage Connection: Permission settings on the specified path do not allow access to the storage.

Verify permission settings on the specified storage path.

解决办法

gluster volume set <volname> storage.owner-uid 36

gluster volume set <volname> storage.owner-gid 36

如果遇到

Description of problem

Trying to create new gluster storage domain blocked with CDA (below) although the glusterfs-cli pkg installed on host.

from GUI-

=========

Error while executing action: Cannot add Storage Connection. Host camel-vdsc.qa.lab.tlv.redhat.com cannot connect to Glusterfs. Verify that glusterfs-cli package is installed on the host.

解决办法

在管理界面先维护主机然后再activate

5 上传镜像

engine-iso-uploader --iso-domain=[ISODomain] upload \

/usr/share/virtio-win/virtio-win-0.1-74.iso \

/usr/share/virtio-win/virtio-win-drivers-20120712-1.vfd

[root@ovirt-engine ~]# engine-iso-uploader --iso-domain=vmiso-storage upload CentOS-7-ppc64le-Minimal-1908.iso

Please provide the REST API password for the admin@internal oVirt Engine user (CTRL+D to abort):

Uploading, please wait...

INFO: Start uploading CentOS-7-ppc64le-Minimal-1908.iso

Uploading: [########################################] 100%

INFO: CentOS-7-ppc64le-Minimal-1908.iso uploaded successfully

在engine web界面添加storage

1 添加datastorage

2 添加iso storage ,支持使用NFS，不支持使用glusterfs

创建虚拟机过程中出现如下错误

The name org.fedoraproject.FirewallD1 was not provided by any .service files

处理方法：重启libvirtd服务即可。

查看网卡UUID

nmcli con | sed -n '1,2p'