kylo集群搭建教程
kylo文档中 Clustering Kylo 介绍的比较模糊，但是大概步骤都讲到了。
kylo的集群，目的是做HA，所以两个节点共用同一个数据库,在这个案例中，两个节点共用122上的mysql、activeMQ、elasticsearch 和 nifi。
| 121 | 122 | 服务 |
|---|---|---|
| N | Y | mysql |
| N | Y | activeMQ |
| N | Y | elasticsearch |
| Y | Y | nifi |
| Y | Y | kylo |
ModeShape Configuration
修改metadata-repository.json 文件内容
vim /opt/kylo/kylo-services/conf/metadata-repository.json
在最后一个追加如下内容
,"clustering": {
"clusterName":"kylo-modeshape-cluster",
"configuration":"modeshape-jgroups-config.xml",
"locking":"db"
}
修改后预览：
Kylo Configuration
在 /opt/kylo/kylo-services/conf/目录下执行如下语句
<!--这里一定要执行一个空语句的 -->
echo " " >> application.properties
echo " " >> application.properties
echo "kylo.cluster.nodeCount=2" >> application.properties
echo "kylo.cluster.jgroupsConfigFile=kylo-cluster-jgroups-config.xml" >> application.properties
sed -i 's|jms.activemq.broker.url=.*|jms.activemq.broker.url=tcp://10.88.88.122:61616|' application.properties
sed -i 's|config.elasticsearch.jms.url=.*|config.elasticsearch.jms.url=tcp://10.88.88.122:61616|' application.properties
修改nifi.rest.host指向同一个NIFI节点
121
vim /opt/kylo/kylo-services/conf/application.properties
nifi.rest.host=10.88.88.122
nifi.rest.port=8079
122
vim /opt/kylo/kylo-services/conf/application.properties
nifi.rest.host=10.88.88.122
nifi.rest.port=8079
修改121和122的 elasticsearch-rest.properties
search.rest.host=10.88.88.122
search.rest.port=9200
Quartz Scheduler Configuration
查看在/opt/kylo/setup/config/kylo-cluster目录下的配置文件：
在121上操作：
拷贝 quartz-cluster-example.properties 到 /opt/kylo/kylo-services/conf/ 目录下。并重命名为quartz.properties 不需要修改内容。
拷贝 kylo-cluster-jgroups-config-example.xml 到 /opt/kylo/kylo-services/conf/ 目录下。并重命名为
kylo-cluster-jgroups-config.xml
修改参数如下：
122上的kylo-cluster-jgroups-config.xml
<!--bind_port 不需要修改-->
<!--bind_addr 当前节点IP-->
<TCP bind_port="7900"
bind_addr="10.88.88.122"
....
<!--initial_hosts kylo的所在节点的IP，端口都是7900不变-->
<TCPPING timeout="3000" async_discovery="true" num_initial_members="2"
initial_hosts="10.88.88.122[7900],10.88.88.121[7900]"
121上的kylo-cluster-jgroups-config.xml
<!--bind_port 不需要修改-->
<!--bind_addr 当前节点IP-->
<TCP bind_port="7900"
bind_addr="10.88.88.121"
....
<!--initial_hosts kylo的所在节点的IP，端口都是7900不变-->
<TCPPING timeout="3000" async_discovery="true" num_initial_members="2"
initial_hosts="10.88.88.121[7900],10.88.88.122[7900]"
修改后预览如下：
拷贝modeshape-local-test-jgroups-config.xml 到 /opt/kylo/kylo-services/conf/ 目录下。并重命名为 modeshape-jgroups-config.xml
121上的modeshape-jgroups-config.xml
<TCP bind_port="7800"
bind_addr="10.88.88.121"
...
<TCPPING timeout="3000" async_discovery="true" num_initial_members="2"
initial_hosts="10.88.88.121[7800],10.88.88.122[7801]"
122上的modeshape-jgroups-config.xml
<TCP bind_port="7801"
bind_addr="10.88.88.122"
...
<TCPPING timeout="3000" async_discovery="true" num_initial_members="2"
initial_hosts="10.88.88.122[7801],10.88.88.121[7800]"
修改后预览如下：
测试：(可以跳过)
在121上执行
java -Djava.net.preferIP4Stack=true -cp /opt/kylo/kylo-services/conf:/opt/kylo/kylo-services/lib/*:/opt/kylo/kylo-services/plugin/* org.jgroups.tests.McastReceiverTest -bind_addr 10.88.88.121 -port 7900
在122上执行
java -Djava.net.preferIP4Stack=true -cp /opt/kylo/kylo-services/conf:/opt/kylo/kylo-services/lib/*:/opt/kylo/kylo-services/plugin/* org.jgroups.tests.McastSenderTest -bind_addr 10.88.88.122 -port 7900
修改run-kylo-services.sh文件
Modify the /opt/kylo/kylo-services/bin/run-kylo-services.sh
Add -Djava.net.preferIPv4Stack=true
java $KYLO_SERVICES_OPTS -Djava.net.preferIPv4Stack=true -cp /opt/kylo/kylo-services/conf ....
原因：
If you get a Network is unreachable error, below, you may need to do the following:
- Network unreachable error
SEVERE: JGRP000200: failed sending discovery request
java.io.IOException: Network is unreachable
at java.net.PlainDatagramSocketImpl.send(Native Method)
at java.net.DatagramSocket.send(DatagramSocket.java:693)
at org.jgroups.protocols.MPING.sendMcastDiscoveryRequest(MPING.java:295)
at org.jgroups.protocols.PING.sendDiscoveryRequest(PING.java:62)
at org.jgroups.protocols.PING.findMembers(PING.java:32)
at org.jgroups.protocols.Discovery.findMembers(Discovery.java:244)
重新新建kylo数据库
登录mysql
drop database kylo ;create database kylo;
然后
cd /opt/kylo/setup/sql/mysql
sh setup-mysql.sh 10.88.88.122 kylo kylo
cd /opt/kylo/setup/sql
sh generate-update-sql.sh
<!--在当前目录下生成两个文件-->
登录mysql
use kylo source /opt/kylo/setup/sql/mysqlkylo-db-update-script.sql;
下载Quartz
Download and extract the Quartz distribution to a machine. http://d2zwv9pap9ylyd.cloudfront.net/quartz-2.2.3-distribution.tar.gz You just need this to get the database scripts.
Run the Quartz database scripts for your database found in the docs/dbTables
再次登录mysql;
use kylo;
souroce ~/quartz-2.2.3/docs/dbTables/tables_mysql.sql;
启动kylo
访问：http://10.88.88.122:8400/index.html#!/admin/cluster
nifi 集群搭建
kylo 文档中介绍的很详细，安装官网教程配置即可；
在这里，分别在121、122上安装nifi
然后执行如下命令：
121上
sed -i "s|nifi.web.http.host=.*|nifi.web.http.host=10.88.88.121|" /opt/nifi/current/conf/nifi.properties
sed -i 's|nifi.cluster.is.node=.*|nifi.cluster.is.node=true|' /opt/nifi/current/conf/nifi.properties
sed -i 's|nifi.cluster.node.address=.*|nifi.cluster.node.address=10.88.88.121|' /opt/nifi/current/conf/nifi.properties
sed -i 's|nifi.cluster.node.protocol.port=.*|nifi.cluster.node.protocol.port=8078|' /opt/nifi/current/conf/nifi.properties
sed -i 's|nifi.zookeeper.connect.string=.*|nifi.zookeeper.connect.string=10.88.88.121:2181|' /opt/nifi/current/conf/nifi.properties
至此nifi集群模式已经设置好了，下面配置kylo和nifi的交互
修改nifi的activeMQ的配置
vi /opt/nifi/ext-config/config.properties
# 这里需要配置active MQ的主节点
jms.activemq.broker.url=tcp://10.88.88.121:61616
或者
sed -i "s|jms.activemq.broker.url=.*|jms.activemq.broker.url=tcp://10.88.88.121:61616|" /opt/nifi/ext-config/config.properties
修改nifi.properties配置文件
<!--设置超时时长，避免导入模板返回500错误-->
sed -i "s|nifi.cluster.node.connection.timeout=.*|nifi.cluster.node.connection.timeout=25 sec|" /opt/nifi/nifi-1.6.0/conf/nifi.properties
sed -i "s|nifi.cluster.node.read.timeout=.*|nifi.cluster.node.read.timeout=25 sec|" /opt/nifi/nifi-1.6.0/conf/nifi.properties
添加监控插
拷贝/opt/kylo/setup/plugins下的kylo-service-monitor-kylo-cluster-0.9.1.jar 到/opt/kylo/kylo-services/plugin/目录下
重启nifi和kylo后导入模板测试（data_ingest.zip）报错。原因是activeMQ的配置需要修改
然后在kylo中跑流程就可以成功了。
