Kylo单机安装详解

创建需要的用户和分组

  • useradd -r -m -s /bin/bash nifi && useradd -r -m -s /bin/bash kylo && useradd -r -m -s /bin/bash activemq
  • groupadd -f kylo && groupadd -f nifi && groupadd -f activemq

创建kylo安装目录

  • mkdir /opt/kylo

将安装包解压到安装目录

  • tar xvf /home/wcm/sources/kylo-0.10.0.tar -C /opt/kylo

安装前执行预处理脚本, 这里以root用户安装

  • /opt/kylo/setup/install/post-install.sh /opt/kylo root root

PS: 三个参数依次为: kylo安装目录, 安装程序的用户, 安装程序的用户组

执行安装向导脚本

  • /opt/kylo/setup/setup-wizard.sh

PS1: 安装kylo需要依赖的组件: mysql, elasticsearch, activemq, nifi, kylo
前两者已经安装好, 故只安装后面

PS2 :dladmin密码为kylo-ui登录密码

[root@dpnode05 opt]# /opt/kylo/setup/setup-wizard.sh
The working directory is /opt/kylo/setup
Welcome to the Kylo setup wizard. Lets get started !!!
 
Please enter Y/y or N/n to the following questions:
 
Enter the kylo home folder location, hit Enter for '/opt/kylo': 
Enter the kylo linux user, hit Enter for 'kylo': 
Enter the kylo linux group, hit Enter for 'users': kylo
 
Would you like to install the database scripts in a database instance? Please enter y/n: y  
Would you like Kylo to manage installing and upgrading the database automatically? Please enter y/n: n
 
OK. Disabling Liquibase in application.properties. Please see the Kylo documentation to see how to generate the database scripts
 
Press any key to continue: Which database (Enter the number)?
1) MySQL
2) PostgresSQL
3) SQL Server
> 1

Please enter the database hostname or IP, hit Enter for 'localhost'
> 
Please enter the database ADMIN username
> root
Please enter the database ADMIN password
> Creating MySQL database 'kylo'
Warning: Using a password on the command line interface can be insecure.
 
Please enter the password for the dladmin user
> 
Please re-enter the password for the dladmin user
>  
Please choose an option to configure Java for Kylo, ActiveMQ, and NiFi
1) I already have Java 8 or higher installed as the system Java and want to use that
2) Install Java 8 in the /opt/java folder for me and use that one
3) I have Java 8 or higher installed in another location already. I will provide the location
4) Java is already setup. No changes necessary
> 1
 
Would you like me to install a local elasticsearch instance? Please enter y/n: n
 
Would you like me to install a local activemq instance?  Please enter y/n: y
Enter the Activemq home folder location, hit Enter for '/opt/activemq': 
Enter the user Activemq should run as, hit Enter for 'activemq': 
Enter the linux group Activemq should run as, hit Enter for 'activemq': 
 
Would you like me to install a local nifi instance? Please enter y/n: y
Enter Nifi version you wish to install, hit Enter for '1.6.0': 
Enter the NiFi home folder location, hit Enter for '/opt/nifi': 
Enter the user NiFi should run as, hit Enter for 'nifi': 
Enter the linux group NiFi should run as, hit Enter for 'nifi': 
Using system Java 8 and remove the JAVA_HOME variable from kylo-ui and kylo-services
Removing custom JAVA_HOME from kylo-ui and kylo-services
installing ActiveMQ
The active home folder is /opt/activemq 
Create the /opt/activemq directory
Download activemq and install
  % Total    % Received % Xferd  Average Speed   Time    Time     Time  Current
                                 Dload  Upload   Total   Spent    Left  Speed
  0 55.8M    0  192k    0     0  76476      0  0:12:45  0:00:02  0:12:43  125k

修改配置文件

  • vim /opt/kylo/kylo-services/conf/application.properties
spring.datasource.url=jdbc:mysql://dpnode05:3306/kylo
spring.datasource.username=root
spring.datasource.password=123456
spring.datasource.driverClassName=com.mysql.jdbc.Driver

复制Mysql驱动

  • cp /home/wcm/mysql-connector-java-8.0.15.jar /opt/kylo/kylo-services/lib/

生成sql脚本

  • cd /opt/kylo/setup/sql/mysql
  • ./setup-mysql.sh dpnode05 root 123456
  • cd /opt/kylo/setup/sql
  • ./generate-update-ql.sh
-rw-r--r-- 1 root root    0 Feb 22 15:04 kylo-db-update-script.sql
-rw-r--r-- 1 root root    0 Feb 22 15:04 kylo-db-update-script.sql.bac

修改脚本, 解决时区异常问题

vim /opt/kylo/setup/sql/kylo-db-update-script.sql

将第90行中的 1970-01-01 00:00:01 加8小时, 改为 1970-01-01 08:00:01

CREATE TABLE kylo.FEED (id BINARY(16) NOT NULL, name VARCHAR(100) NOT NULL, description VARCHAR(255) NULL, FEED_TYPE VARCHAR(45) NULL, created_time timestamp DEFAULT NOW() NOT NULL, modified_time timestamp DEFAULT '1970-01-01 08:00:01' NOT NULL);

进入Mysql客户端

  • 创建数据库: create database if not exists kylo character set utf8 collate utf8_general_ci;
  • 使用数据库: use kylo;
  • 设置环境变量: SET GLOBAL log_bin_trust_function_creators = 1;
    PS: 该环境变量解决sql异常:ERROR 1418 (HY000): This function has none of DETERMINISTIC, NO SQL, or READS SQL DATA in its declaration and binary logging is enabled
    执行sql脚本: souce kylo-db-update-script.sql

安装elasticsearch, 启动并执行kylo索引脚本

/opt/kylo/bin/create-kylo-indexes-es.sh

启动依赖程序(elasticesearch, nifi, activemq)

  1. 检查是否启动

    • ps -ef | grep elasticesearch
    • ps -ef | grep activemq
    • ps -ef | grep nifi
  2. 没启动则启动, 启动方式:

    • nohup $ELASTICHSEARCH_HOME/elasticsearch &
    • service activemq start
    • service nifi start
  3. 启动kylo

  • kylo-service start

启动后个服务地址:

PS: kylo-services启动后仍有报错, 尚未解决, 暂时找到问题原因

java.security.AccessControlException: Not authorized to perform the action: Access Templates
        at com.thinkbiganalytics.metadata.modeshape.security.action.JcrAllowedActions.checkPermission(JcrAllowedActions.java:226)
        at com.thinkbiganalytics.metadata.modeshape.security.DefaultAccessController.lambda$null$0(DefaultAccessController.java:75)
        at java.util.Optional.map(Optional.java:215)
        at com.thinkbiganalytics.metadata.modeshape.security.DefaultAccessController.lambda$checkPermission$2(DefaultAccessController.java:74)
        at com.thinkbiganalytics.metadata.modeshape.JcrMetadataAccess.read(JcrMetadataAccess.java:422)
        at com.thinkbiganalytics.metadata.modeshape.JcrMetadataAccess.read(JcrMetadataAccess.java:369)
        at com.thinkbiganalytics.metadata.persistence.AggregateMetadataAccess.read(AggregateMetadataAccess.java:86)
        at com.thinkbiganalytics.metadata.modeshape.security.DefaultAccessController.checkPermission(DefaultAccessController.java:72)
        at com.thinkbiganalytics.metadata.modeshape.security.DefaultAccessController.checkPermission(DefaultAccessController.java:62)
        at com.thinkbiganalytics.repository.filesystem.RepositoryMonitor.lambda$getAllRegisteredTemplatesAsMap$3(RepositoryMonitor.java:138)
        at com.thinkbiganalytics.metadata.persistence.AggregateMetadataAccess.lambda$null$3(AggregateMetadataAccess.java:124)
        at com.thinkbiganalytics.metadata.config.OperationalMetadataTransactionTemplateMetadataAccess$2.doInTransaction(OperationalMetadataTransactionTemplateMetadataAccess.java:187)
        at org.springframework.transaction.support.TransactionTemplate.execute(TransactionTemplate.java:133)
        at com.thinkbiganalytics.metadata.config.OperationalMetadataTransactionTemplateMetadataAccess.read(OperationalMetadataTransactionTemplateMetadataAccess.java:144)
        at com.thinkbiganalytics.metadata.config.OperationalMetadataTransactionTemplateMetadataAccess.read(OperationalMetadataTransactionTemplateMetadataAccess.java:118)
        at com.thinkbiganalytics.metadata.persistence.AggregateMetadataAccess.lambda$wrap$5(AggregateMetadataAccess.java:124)
        at com.thinkbiganalytics.metadata.modeshape.JcrMetadataAccess.execute(JcrMetadataAccess.java:444)
        at com.thinkbiganalytics.metadata.modeshape.JcrMetadataAccess.read(JcrMetadataAccess.java:401)
        at com.thinkbiganalytics.metadata.modeshape.JcrMetadataAccess.read(JcrMetadataAccess.java:369)
        at com.thinkbiganalytics.metadata.persistence.AggregateMetadataAccess.read(AggregateMetadataAccess.java:86)
        at com.thinkbiganalytics.repository.filesystem.RepositoryMonitor.getAllRegisteredTemplatesAsMap(RepositoryMonitor.java:137)
        at com.thinkbiganalytics.repository.filesystem.RepositoryMonitor.watchRepositories(RepositoryMonitor.java:99)
        at com.thinkbiganalytics.repository.filesystem.FilesystemRepositoryService.monitorRepositories(FilesystemRepositoryService.java:141)
        at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
        at sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
        at sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
        at java.lang.reflect.Method.invoke(Method.java:498)
        at org.springframework.scheduling.support.ScheduledMethodRunnable.run(ScheduledMethodRunnable.java:65)
        at org.springframework.scheduling.support.DelegatingErrorHandlingRunnable.run(DelegatingErrorHandlingRunnable.java:54)
        at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
        at java.util.concurrent.FutureTask.runAndReset(FutureTask.java:308)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$301(ScheduledThreadPoolExecutor.java:180)
        at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:294)
        at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1142)
        at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
        at java.lang.Thread.run(Thread.java:745)
2019-02-22 15:47:58 WARN  localhost-startStop-2:WebappClassLoaderBase:180 - The web application [ROOT] appears to have started a thread named [modeshape-stats-2-thread-1] but has failed to stop it. This
 is very likely to create a memory leak. Stack trace of thread:
 sun.misc.Unsafe.park(Native Method)
 java.util.concurrent.locks.LockSupport.parkNanos(LockSupport.java:215)
 java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.awaitNanos(AbstractQueuedSynchronizer.java:2078)
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:1093)
 java.util.concurrent.ScheduledThreadPoolExecutor$DelayedWorkQueue.take(ScheduledThreadPoolExecutor.java:809)
 java.util.concurrent.ThreadPoolExecutor.getTask(ThreadPoolExecutor.java:1067)
 java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1127)
 java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:617)
最后编辑于
©著作权归作者所有,转载或内容合作请联系作者
  • 序言:七十年代末,一起剥皮案震惊了整个滨河市,随后出现的几起案子,更是在滨河造成了极大的恐慌,老刑警刘岩,带你破解...
    沈念sama阅读 151,829评论 1 331
  • 序言:滨河连续发生了三起死亡事件,死亡现场离奇诡异,居然都是意外死亡,警方通过查阅死者的电脑和手机,发现死者居然都...
    沈念sama阅读 64,603评论 1 273
  • 文/潘晓璐 我一进店门,熙熙楼的掌柜王于贵愁眉苦脸地迎上来,“玉大人,你说我怎么就摊上这事。” “怎么了?”我有些...
    开封第一讲书人阅读 101,846评论 0 226
  • 文/不坏的土叔 我叫张陵,是天一观的道长。 经常有香客问我,道长,这世上最难降的妖魔是什么? 我笑而不...
    开封第一讲书人阅读 42,600评论 0 191
  • 正文 为了忘掉前任,我火速办了婚礼,结果婚礼上,老公的妹妹穿的比我还像新娘。我一直安慰自己,他们只是感情好,可当我...
    茶点故事阅读 50,780评论 3 272
  • 文/花漫 我一把揭开白布。 她就那样静静地躺着,像睡着了一般。 火红的嫁衣衬着肌肤如雪。 梳的纹丝不乱的头发上,一...
    开封第一讲书人阅读 39,695评论 1 192
  • 那天,我揣着相机与录音,去河边找鬼。 笑死,一个胖子当着我的面吹牛,可吹牛的内容都是我干的。 我是一名探鬼主播,决...
    沈念sama阅读 31,136评论 2 293
  • 文/苍兰香墨 我猛地睁开眼,长吁一口气:“原来是场噩梦啊……” “哼!你这毒妇竟也来了?” 一声冷哼从身侧响起,我...
    开封第一讲书人阅读 29,862评论 0 182
  • 序言:老挝万荣一对情侣失踪,失踪者是张志新(化名)和其女友刘颖,没想到半个月后,有当地人在树林里发现了一具尸体,经...
    沈念sama阅读 33,453评论 0 229
  • 正文 独居荒郊野岭守林人离奇死亡,尸身上长有42处带血的脓包…… 初始之章·张勋 以下内容为张勋视角 年9月15日...
    茶点故事阅读 29,942评论 2 233
  • 正文 我和宋清朗相恋三年,在试婚纱的时候发现自己被绿了。 大学时的朋友给我发了我未婚夫和他白月光在一起吃饭的照片。...
    茶点故事阅读 31,347评论 1 242
  • 序言:一个原本活蹦乱跳的男人离奇死亡,死状恐怖,灵堂内的尸体忽然破棺而出,到底是诈尸还是另有隐情,我是刑警宁泽,带...
    沈念sama阅读 27,790评论 2 236
  • 正文 年R本政府宣布,位于F岛的核电站,受9级特大地震影响,放射性物质发生泄漏。R本人自食恶果不足惜,却给世界环境...
    茶点故事阅读 32,293评论 3 221
  • 文/蒙蒙 一、第九天 我趴在偏房一处隐蔽的房顶上张望。 院中可真热闹,春花似锦、人声如沸。这庄子的主人今日做“春日...
    开封第一讲书人阅读 25,839评论 0 8
  • 文/苍兰香墨 我抬头看了看天上的太阳。三九已至,却和暖如春,着一层夹袄步出监牢的瞬间,已是汗流浃背。 一阵脚步声响...
    开封第一讲书人阅读 26,448评论 0 181
  • 我被黑心中介骗来泰国打工, 没想到刚下飞机就差点儿被人妖公主榨干…… 1. 我叫王不留,地道东北人。 一个月前我还...
    沈念sama阅读 34,564评论 2 249
  • 正文 我出身青楼,却偏偏与公主长得像,于是被迫代替她去往敌国和亲。 传闻我的和亲对象是个残疾皇子,可洞房花烛夜当晚...
    茶点故事阅读 34,623评论 2 249