【zabbix HA】zabbixHA 高可用的实现

zabbix高可用设计目标

1、keepalived服务优先级选择切换机制:对于zabbix服务器来说,只要zabbix存活和mysql存活,就能够正常记录数据,不会丢失数据,php和httpd只是web页面的访问而已,所以我在这里定义mysql和zabbix为主要服务,php和httpd为次要服务,为了实现主要服务存在,次要服务挂了;次要服务器存在,主要服务器挂了,keepalived会优先选择主要服务存在的一方作为master,让keepalived切换更为更合理。

2、数据库主主同步:不管切到哪一边都需要保持数据一致性,不可出现丢数据或者数据重复,保证数据库的高可用。

3、文件双向同步:

        (1)web文件同步:任何做web文件得配置和修改,保证两边一致,确保切换不会发生任何变化。

        (2)zabbix服务文件同步:对zabbix_servr的配置文件做的任何修改,和脚本的修改进行同步,也是为了保证两边的服务一致性,没有落后情况,达到无需人工干预自动切换服务可正常时候。

        (3)到此完成了zabbix得无缝切换,无故障时间!

一、实验环境

操作系统: CentOS7.2 Minial

zabbixA: 192.168.1.102

zabbixB:  192.168.1.103 

vip:          192.168.1.120

二、软件安装

在zabbixA和zabbixB

1.安装zabbix组件

# yum  -y install epel-release

# rpm -hiv   http://repo.zabbix.com/zabbix/3.5/rhel/7/x86_64/zabbix-release-3.5-1.el7.noarch.rpm

# yum -y install zabbix-server-mysql zabbix-web.noarch zabbix-web-mysql.noarch  zabbix-java-gateway  zabbix-get

# yum  -y install  zabbix-agent  zabbix-sender 

2. 安装数据库组件

# rpm -ivh     https://dev.mysql.com/get/mysql57-community-release-el7-11.noarch.rpm

# yum repolist enabled | grep "mysql.*-community.*"

# yum -y install mysql-community-server  mysql-community-client

3.安装php httpd组件

# yum -y install  php php-gd php-mysql php-bcmath php-mbstring php-xml  httpd

4.安装文件同步组件

#  yum  -y install rsync  inotify-tools

5. 双机互信,实现免密登录

# ssh-keygen -t rsa -P '' -q  -f ~/.ssh/id_rsa

# ssh-copy-id  root@$ip

6. 设置selinux模式和防火墙

# sysctl -w net.ipv4.ip_nonlocal_bind=1

# echo "net.ipv4.ip_nonlocal_bind=1" >> /etc/sysctl.conf

# setenforce 0

# sed -i 's/^SELINUX=.*/SELINUX=permissive/g'   /etc/selinux/config

# iptables -F

三、zabbixA软件配置

1.数据库配置

# systemctl start mysqld.service

# systemctl enable mysqld.service

# cat /var/log/mysqld.log | grep password 

# mysql -u root -p"XXX"

mysql> set global validate_password_policy=0;

mysql> set global validate_password_length=1;

mysql> set password = password('mysql123');

mysql> create database zabbix character set utf8 collate utf8_bin;

mysql> grant all privileges on zabbix.* to zabbix@localhost identified by 'zabbix123';

2.zabbix-server配置

# sed -i   '/^# SourceIP/s/=.*$/=192.168.1.120/'                  /etc/zabbix/zabbix_server.conf

# sed -i   '/^# DBHost/s/=.*$/=192.168.1.120/'                     /etc/zabbix/zabbix_server.conf

# sed -i  '/^DBName/s/=.*$/=zabbix/'                                    /etc/zabbix/zabbix_server.conf

# sed -i  '/^# DBPassword/s/.*$/DBPassword=zabbix123/'  /etc/zabbix/zabbix_server.conf

3.php配置

# sed -i      's/post_max_size = 8M/post_max_size = 32M/g'                          /etc/php.ini

# sed -i      's/upload_max_filesize = 2M/upload_max_filesize = 50M/g'        /etc/php.ini

# sed -i       's/\;date.timezone =/date.timezone = Asia\/Shanghai/'               /etc/php.ini

# sed -i      's/max_execution_time = 30/max_execution_time = 600/g'          /etc/php.ini

# sed -i     's/max_input_time = 60/max_input_time = 600/g'                       /etc/php.ini

# sed -i     's/memory_limit = 128M/memory_limit = 256M/g'                      /etc/php.ini 

# systemctl restart zabbix-server

# systemctl enable zabbix-server

# systemctl restart httpd

# systemctl enable httpd

4.keepalived配置

# vim /etc/keepalived/keepalived.conf

############################################

! Configuration File for keepalived

global_defs {

   router_id LVS_DEVEL

}

vrrp_script check { 

    script "/etc/keepalived/check.sh" 

    interval 5     

}   

vrrp_instance VI_1 {

    state BACKUP

    interface eno16777736

    virtual_router_id 100

    priority 100

    advert_int 1

    nopreempt

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    track_script {   

      check

    }   

    virtual_ipaddress {

        192.168.1.120

    }

#notify_master "/etc/keepalived/zabbix.sh start"

#notify_backup "/etc/keepalived/zabbix.sh stop"

}

##############################################

# vim /etc/keepalived/check.sh

##############################################

#!/bin/bash

zabbix_server_status1=$(ps -C zabbix_server --no-heading|wc -l)

if [ "${zabbix_server_status1}" = "0" ]; then

  systemctl start zabbix_server.service

  sleep 3

  zabbix_server_status2=$(ps -C zabbix_server --no-heading|wc -l)

  if [ "${zabbix_server_status2}" = "0" ]; then

    systemctl restart keepalived.service

  fi

fi

mysqld_status1=$(ps -C mysqld --no-heading|wc -l)

if [ "${mysqld_status1}" = "0" ]; then

  systemctl start mysqld.service

  sleep 3

  mysqld_status2=$(ps -C mysqld --no-heading|wc -l)

  if [ "${mysqld_status2}" = "0" ]; then

    systemctl restart keepalived.service

  fi

fi

httpd_status1=$(ps -C httpd --no-heading|wc -l)

if [ "${httpd_status1}" = "0" ]; then

  systemctl start httpd.service

  sleep 3

  httpd_status2=$(ps -C httpd --no-heading|wc -l)

  if [ "${httpd_status2}" = "0" ]; then

    systemctl restart keepalived.service

  fi

fi

##########################################

# chmod +x /etc/keepalived/check.sh

5. 文件同步设置

# vim /opt/data_realtime_rsync.sh

####################################################################

#!/bin/bash

# File name:data_realtime_sync.sh

# File path: /opt/data_realtime_sync.sh

src_dir="/etc/zabbix"

dest_dir="/etc/zabbix"

src_ip="192.168.1.102"

dest_ip="192.168.1.103"

. /etc/init.d/functions

cd ${src_dir}

inotifywait -mrq -e modify,attrib,close_write,move,create,delete --format '%e %w%f' ./ |

while read file;do

    INO_EVENT=$(echo $file | awk '{print $1}')      

    INO_FILE=$(echo $file  | awk '{print $2}')      

    if [[ $INO_EVENT =~ 'CREATE' ]] || [[ $INO_EVENT =~ 'MODIFY' ]] || [[ $INO_EVENT =~ 'CLOSE_WRITE' ]] || [[ $INO_EVENT =~ 'MOVED_TO' ]];then

        for ip in ${dest_ip};do

            rsync -avzcR -e ssh  $(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}

        done

    fi

    if [[ $INO_EVENT =~ 'ATTRIB' ]];then

        if [ ! -d "$INO_FILE" ];then

            for ip in ${dest_ip};do

                rsync -avzcR  -e ssh  $(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}

            done

        fi

    fi

    if [[ $INO_EVENT =~ 'DELETE' ]] || [[ $INO_EVENT =~ 'MOVED_FROM' ]];then

        for ip in ${dest_ip};do

            rsync -avzcR --delete  -e ssh  $(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}

        done

    fi

done

#####################################################################

# chmod +x /opt/data_realtime_rsync.sh

# vim /etc/systemd/system/rsync-inotity.service

###############################################

[Unit]

Description=File instant rsync script

[Service]

ExecStart=/usr/bin/nohup /opt/data_realtime_rsync.sh  &

[Install]

WantedBy=multi-user.target

#################################################

6.数据库主主同步配置

# vim /etc/my.cnf

######################################################

[mysqld]

datadir=/var/lib/mysql

socket=/var/lib/mysql/mysql.sock

symbolic-links=0

log-error=/var/log/mysqld.log

pid-file=/var/run/mysqld/mysqld.pid

server-id=1                       

log-bin=mysql-bin                

auto-increment-increment = 2    

auto-increment-offset = 1      

binlog_format = mixed         

sync_binlog=1                

binlog-do-db=zabbix         

binlog-ignore-db=mysql     

binlog-ignore-db=sys

binlog-ignore-db=performance_schema

binlog-ignore-db=information_schema

replicate-do-db=zabbix            

replicate-ignore-db=mysql        

replicate-ignore-db=sys

replicate-ignore-db=performance_schema

replicate-ignore-db=information_schema

######################################################

# systemctl restart mysqld

登录zabbixB的MySQL

创建数据同步账号

# mysql -u root -p"mysql123"

mysql> GRANT REPLICATION SLAVE ON *.* TO 'repluser'@'%' IDENTIFIED BY 'Repl@123';

mysql> grant all privileges on zabbix.* to zabbix@192.168.1.120 identified by 'zabbix123';

mysql> FLUSH PRIVILEGES;

# exit

获取zabbixB二进制日志名和位置

#mysql -uroot -p"mysql123"    --execute='show master status;

# mysql -uroot -p$"mysql123" --execute='show master status;'

在zabbixA 上

#  mysql -uroot -p"mysql123"

mysql> CHANGE MASTER TO

    -> MASTER_HOST='192.168.1.103',

    -> MASTER_USER='repluser',

    -> MASTER_PASSWORD='Repl@123',

    -> MASTER_LOG_FILE='mysql-bin.000002',

    -> MASTER_LOG_POS=50334;

mysql> START SLAVE;

mysql> SHOW SLAVE STATUS\G

四、zabbixB软件配置

1.数据库配置

# systemctl start mysqld.service

# systemctl enable mysqld.service

# cat /var/log/mysqld.log | grep password 

# mysql -u root -p"XXX"

mysql> set global validate_password_policy=0;

mysql> set global validate_password_length=1;

mysql> set password = password('mysql123');

mysql> create database zabbix character set utf8 collate utf8_bin;

mysql> grant all privileges on zabbix.* to zabbix@localhost identified by 'zabbix123';

2.zabbix-server配置

# sed -i   '/^ SourceIP/s/=.*$/=192.168.1.120/'                     /etc/zabbix/zabbix_server.conf

# sed -i   '/^# DBHost/s/=.*$/=192.168.1.120/'                     /etc/zabbix/zabbix_server.conf

# sed -i  '/^DBName/s/=.*$/=zabbix/'                                    /etc/zabbix/zabbix_server.conf

# sed -i  '/^# DBPassword/s/.*$/DBPassword=zabbix123/'   /etc/zabbix/zabbix_server.conf

3.php配置

# sed -i      's/post_max_size = 8M/post_max_size = 32M/g'                          /etc/php.ini

# sed -i      's/upload_max_filesize = 2M/upload_max_filesize = 50M/g'        /etc/php.ini

# sed -i       's/\;date.timezone =/date.timezone = Asia\/Shanghai/'               /etc/php.ini

# sed -i      's/max_execution_time = 30/max_execution_time = 600/g'          /etc/php.ini

# sed -i     's/max_input_time = 60/max_input_time = 600/g'                       /etc/php.ini

# sed -i     's/memory_limit = 128M/memory_limit = 256M/g'                      /etc/php.ini 

# systemctl restart zabbix-server

# systemctl enable zabbix-server

# systemctl restart httpd

# systemctl enable httpd

4.keepalived配置

# vim /etc/keepalived/keepalived.conf

############################################

! Configuration File for keepalived

global_defs {

   router_id LVS_DEVEL

}

vrrp_script check { 

    script "/etc/keepalived/check.sh" 

    interval 5     

}   

vrrp_instance VI_1 {

    state BACKUP

    interface eno16777736

    virtual_router_id 90

    priority 100

    advert_int 1

    nopreempt

    authentication {

        auth_type PASS

        auth_pass 1111

    }

    track_script {   

      check

    }   

    virtual_ipaddress {

        192.168.1.120

    }

#notify_master "/etc/keepalived/zabbix.sh start"

#notify_backup "/etc/keepalived/zabbix.sh stop"

}

##############################################

# vim /etc/keepalived/check.sh

##############################################

#!/bin/bash

zabbix_server_status1=$(ps -C zabbix_server --no-heading|wc -l)

if [ "${zabbix_server_status1}" = "0" ]; then

  systemctl start zabbix_server.service

  sleep 3

  zabbix_server_status2=$(ps -C zabbix_server --no-heading|wc -l)

  if [ "${zabbix_server_status2}" = "0" ]; then

    systemctl restart keepalived.service

  fi

fi

mysqld_status1=$(ps -C mysqld --no-heading|wc -l)

if [ "${mysqld_status1}" = "0" ]; then

  systemctl start mysqld.service

  sleep 3

  mysqld_status2=$(ps -C mysqld --no-heading|wc -l)

  if [ "${mysqld_status2}" = "0" ]; then

    systemctl restart keepalived.service

  fi

fi

httpd_status1=$(ps -C httpd --no-heading|wc -l)

if [ "${httpd_status1}" = "0" ]; then

  systemctl start httpd.service

  sleep 3

  httpd_status2=$(ps -C httpd --no-heading|wc -l)

  if [ "${httpd_status2}" = "0" ]; then

    systemctl restart keepalived.service

  fi

fi

##########################################

# chmod +x /etc/keepalived/check.sh

5. 文件同步设置

# vim /opt/data_realtime_rsync.sh

####################################################################

#!/bin/bash

# File name:data_realtime_sync.sh

# File path: /opt/data_realtime_sync.sh

src_dir="/etc/zabbix"

dest_dir="/etc/zabbix"

src_ip="192.168.1.103"

dest_ip="192.168.1.102"

. /etc/init.d/functions

cd ${src_dir}

inotifywait -mrq -e modify,attrib,close_write,move,create,delete --format '%e %w%f' ./ |

while read file;do

    INO_EVENT=$(echo $file | awk '{print $1}')      

    INO_FILE=$(echo $file  | awk '{print $2}')      

    if [[ $INO_EVENT =~ 'CREATE' ]] || [[ $INO_EVENT =~ 'MODIFY' ]] || [[ $INO_EVENT =~ 'CLOSE_WRITE' ]] || [[ $INO_EVENT =~ 'MOVED_TO' ]];then

        for ip in ${dest_ip};do

            rsync -avzcR -e ssh  $(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}

        done

    fi

    if [[ $INO_EVENT =~ 'ATTRIB' ]];then

        if [ ! -d "$INO_FILE" ];then

            for ip in ${dest_ip};do

                rsync -avzcR  -e ssh  $(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}

            done

        fi

    fi

    if [[ $INO_EVENT =~ 'DELETE' ]] || [[ $INO_EVENT =~ 'MOVED_FROM' ]];then

        for ip in ${dest_ip};do

            rsync -avzcR --delete  -e ssh  $(dirname ${INO_FILE}) root@${dest_ip}:${dest_dir}

        done

    fi

done

#####################################################################

# chmod +x /opt/data_realtime_rsync.sh

# vim /etc/systemd/system/rsync-inotity.service

###############################################

[Unit]

Description=File instant rsync script

[Service]

ExecStart=/usr/bin/nohup /opt/data_realtime_rsync.sh  &

[Install]

WantedBy=multi-user.target

#################################################

 6.数据库主主同步配置

# vim /etc/my.cnf

######################################################

[mysqld]

datadir=/var/lib/mysql

socket=/var/lib/mysql/mysql.sock

symbolic-links=0

log-error=/var/log/mysqld.log

pid-file=/var/run/mysqld/mysqld.pid

server-id=2                       

log-bin=mysql-bin                

auto-increment-increment = 2    

auto-increment-offset = 2      

binlog_format = mixed         

sync_binlog=1                

binlog-do-db=zabbix         

binlog-ignore-db=mysql     

binlog-ignore-db=sys

binlog-ignore-db=performance_schema

binlog-ignore-db=information_schema

replicate-do-db=zabbix         

replicate-ignore-db=mysql     

replicate-ignore-db=sys

replicate-ignore-db=performance_schema

replicate-ignore-db=information_schema

######################################################

# systemctl restart mysqld

登录zabbixA的MySQL

创建数据同步账号

# mysql -u root -p"mysql123"

mysql> GRANT REPLICATION SLAVE ON *.* TO 'repluser'@'%' IDENTIFIED BY 'Repl@123';

mysql> grant all privileges on zabbix.* to zabbix@192.168.1.120 identified by 'zabbix123';

mysql> FLUSH PRIVILEGES;

# exit

获取zabbixA二进制日志名和位置

#mysql -uroot -p"mysql123"    --execute='show master status;

# mysql -uroot -p$"mysql123" --execute='show master status;'

在zabbixB上

# mysql -uroot -p"mysql123"

mysql> CHANGE MASTER TO

    -> MASTER_HOST='192.168.1.102',

    -> MASTER_USER='repluser',

    -> MASTER_PASSWORD='Repl@123',

    -> MASTER_LOG_FILE='mysql-bin.000002',

    -> MASTER_LOG_POS=50334;

mysql> START SLAVE;

mysql> SHOW SLAVE STATUS\G

 五、zabbixA  zabbixB服务启动

# systemctl start  zabbix-server

# systemctl enable zabbix-server

# systemctl start  rsync-inotify

# systemctl enable  rsync-inotify

# cat /etc/zabbix/zabbix_server.conf | grep 120

# cat /etc/zabbix/zabbix_server.conf | grep -Ev "^#|^$"

# vim  /etc/zabbix/web/zabbix.conf.php

六、访问测试

http://192.168.1.102/zabbix

http://192.168.1.103/zabbix

http://192.168.1.20/zabbix



七、参考

inotify+rsync+mysql主主复制+keepalived实现zabbix高可用

http://blog.51cto.com/3241766/2137303

Keepalived中Master和Backup主备切换机制浅析

http://blog.51cto.com/3241766/2097483

Zabbix高可用,实现zabbix的无缝切换,无故障时间

http://blog.51cto.com/yigemeng/1738174

实现Zabbix的高可用

https://www.longlong.asia/2016/10/29/zabbix-ha.html

keepalived + rsync +inotify 实现真正的高效数据实时同步

https://fandenggui.com/post/keepalived-rsync-inotify.html

真正的inotify+rsync实时同步 彻底告别同步慢

http://www.ttlsa.com/web/let-infotify-rsync-fast

Zabbix HA Requirment

https://kasperdeng.github.io/zabbix-ha-on-aws

Configuring the Zabbix server for high availability

https://www.packtpub.com/mapt/book/networking_and_servers/9781785289262/3/ch03lvl1sec18/configuring-the-zabbix-server-for-high-availability

推荐阅读更多精彩内容