克隆虚拟机
我克隆过来是直接可用的，可以 ping 通外网；若克隆出来的虚拟机有问题，可以查阅相关资料进行修改。
VMware虚拟机克隆CentOS后网卡修改方法
配置ssh 免登录
- 启动三台机器，分别修改机器名为 master、slave1、slave2 ，重启系统
[root@master ~]# vim /etc/sysconfig/network
NETWORKING=yes
iHOSTNAME=master
- 修改master 上的 etc/hosts
[root@master ~]# vim /etc/hosts
1 127.0.0.1 localhost localhost.localdomain localhost4 localhost4.localdomai n4
2 ::1 localhost localhost.localdomain localhost6 localhost6.localdomai n6
3 192.168.1.121 master
4 192.168.1.115 slave1
5 192.168.1.116 slave2
6 192.168.1.127 slave3
3.将hosts文件复制到slave1 和slave2
[root@master ~]# su hadoop
[hadoop@master root]$
[hadoop@master root]$ cd
[hadoop@master ~]$ sudo scp /etc/hosts root@slave1:/etc
We trust you have received the usual lecture from the local System
Administrator. It usually boils down to these three things:
#1) Respect the privacy of others.
#2) Think before you type.
#3) With great power comes great responsibility.
[sudo] password for hadoop:
The authenticity of host 'slave1 (192.168.1.115)' can't be established.
RSA key fingerprint is 59:a1:54:8c:86:55:eb:a8:a8:85:76:ca:59:05:f4:6f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'slave1,192.168.1.115' (RSA) to the list of known hosts.
root@slave1's password:
hosts 100% 242 0.2KB/s 00:00
[hadoop@master ~]$ sudo scp /etc/hosts root@slave2:/etc
- 在master机器上使用Hadoop用户登录（确保接下来的操作都是通过hadoop用户执行）。执行 ssh -keygen -t rsa 命令产生公钥。
[hadoop@master ~]$ ssh -keygen -t rsa
Bad escape character 'ygen'.
[hadoop@master ~]$ ssh-keygen -t rsa
Generating public/private rsa key pair.
Enter file in which to save the key (/home/hadoop/.ssh/id_rsa):
Enter passphrase (empty for no passphrase):
Enter same passphrase again:
Your identification has been saved in /home/hadoop/.ssh/id_rsa.
Your public key has been saved in /home/hadoop/.ssh/id_rsa.pub.
The key fingerprint is:
ec:38:ba:0a:39:6d:97:60:b0:41:5a:e6:f3:b7:94:3e hadoop@master
The key's randomart image is:
+--[ RSA 2048]----+
| .o |
|o+ |
|+ o |
| + o o |
|. o . + S |
| + . = + |
|+ o o E . |
| + . . o |
| ..o. |
+-----------------+
- 将公钥复制到 slave1 和 slave2
[hadoop@master ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub slave1
The authenticity of host 'slave1 (192.168.1.115)' can't be established.
RSA key fingerprint is 59:a1:54:8c:86:55:eb:a8:a8:85:76:ca:59:05:f4:6f.
Are you sure you want to continue connecting (yes/no)? yes
Warning: Permanently added 'slave1,192.168.1.115' (RSA) to the list of known hosts.
hadoop@slave1's password:
Now try logging into the machine, with "ssh 'slave1'", and check in:
.ssh/authorized_keys
to make sure we haven't added extra keys that you weren't expecting.
[hadoop@master ~]$ ssh-copy-id -i ~/.ssh/id_rsa.pub slave2
- 再次登录，可以发现不需要密码就可以登录slave1 ，slave2
v[hadoop@master ~]$ ssh slave1
[hadoop@slave1 ~]$
安装JDK
- 使用 yum search jdk 在线查找jdk 列表，任意选择一个版本进行安装
[root@master ~]# yum search jdk
[root@master ~]# yum install java-1.7.0-openjdk-devel.i686 -y
- 配置Java环境变量
# 查询Jdk 路径
[root@master ~]# whereis java
java: /usr/bin/java /etc/java /usr/lib/java /usr/share/java /usr/share/man/man1/java.1.gz
[root@master ~]# ll /usr/bin/java
lrwxrwxrwx. 1 root root 22 Sep 21 19:41 /usr/bin/java -> /etc/alternatives/java
#修改配置文件
法一：直接在末尾追究如下语句
[root@master ~]# vim /etc/profile
export JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-devel.i686
export MAVEN_HOME=/home/hadoop/local/opt/apache-maven-3.3.1
export JRE_HOME=${JAVA_HOME}/jre
export CLASSPATH=.:${JAVA_HOME}/lib/dt.jar:${JAVA_HOME}/lib/tools.jar
export PATH=${JAVA_HOME}/bin:$MAVEN_HOME/bin:$PATH
法二：系统环境变量文件一般不要轻易更改，可以选择在 /etc/profile.d/java.sh 文件中更改
[root@master ~]# vim /etc/profile.d/java.sh
添加如上所示的语句，然后保存。
# 保存配置后使用source 命令立即生效，进行测试
[root@master ~]# source /etc/profile.d/java.sh
[hadoop@master ~]$ java -version
java version "1.7.0_191"
OpenJDK Runtime Environment (rhel-2.6.15.4.el6_10-i386 u191-b01)
OpenJDK Client VM (build 24.191-b01, mixed mode, sharing)
安装配置MySQL（主节点）
#安装mysql服务器
[root@master ~]# yum install mysql-server
#设置开机启动
[root@master ~]# chkconfig mysqld on
#启动mysql服务
[root@master ~]# service mysqld start
Initializing MySQL database: Installing MySQL system tables...
OK
#设置root的初始密码
[root@master ~]# mysqladmin -u root password '******'
#进入mysql命令行，创建以下数据库
[root@master ~]# mysql -uroot -p******
mysql> #hive
mysql> create database hive DEFAULT CHARSET utf8 COLLATE utf8_general_ci;
Query OK, 1 row affected (0.00 sec)
mysql> #activity monitor
mysql> create database amon DEFAULT CHARSET utf8 COLLATE utf8_general_ci;Query OK, 1 row affected (0.00 sec)
#设置root授权访问以上所有的数据库
mysql> grant all privileges on *.* to 'root'@'%' identified by '123456' with grant option;
Query OK, 0 rows affected (0.00 sec)
mysql> flush privileges;
Query OK, 0 rows affected (0.00 sec)
所有节点配置NTP服务
集群中的所有主机必须保持时间同步，如果时间相差较大会引起一些问题。具体配置如下
#所有节点安装相关组件
[root@master ~]# yum install ntp
#完成后配置开机启动
[root@master ~]# chkconfig ntpd on
#检查设置是否成功，其中2--5为 on 就代表成功
[root@master ~]# chkconfig --list ntpd
ntpd 0:off 1:off 2:on 3:on 4:on 5:on 6:off
- 修改配置文件
打开master机器 /etc/ntp.conf 文件，留下关键信息。
driftfile /var/lib/ntp/drift
restrict 127.0.0.1
restrict -6 ::1
restrict default nomodify notrap
server 127.127.1.0
fudge 127.127.1.0 stratum 8
includefile /etc/ntp/crypto/pw
keys /etc/ntp/keys
- 配置主节
在配置之前，先使用ntpdate手动同步一下时间，免得本机与对时中心时间差距太大，使得ntpd不能正常同步。这里选用65.55.56.206作为对时中心,ntpdate -u 65.55.56.206
- 配置文件完成，保存退出，启动服务，执行如下命令：service ntpd start ；
- 检查是否成功：
[root@master ~]# ntpstat
#若出现以下字符，表示成功；否则需要等待5min左右
synchronised to NTP server (85.199.214.101) at stratum 2
time correct to within 438 ms
polling server every 64 s
安装Cloudera Manager Server 和Agent
主节点解压安装
cloudera manager的目录默认位置在/opt下，解压：cloudera-manager-centos7-cm5.12.1_x86_64.tar.gz 将解压后的cm 5.12.1 和cloudera目录放到/opt目录下。
[root@master opt]# cd cloudera-manager-centos7-cm5.12.1_x86_64
[root@master cloudera-manager-centos7-cm5.12.1_x86_64]# mv cloudera /opt/
[root@master cloudera-manager-centos7-cm5.12.1_x86_64]# mv cm-5.12.1 /opt/
首先需要去MySql的官网下载JDBC驱动，http://dev.mysql.com/downloads/connector/j/，解压后，找到mysql-connector-java-8.0.12.jar，放到/opt/cm-5.12.1/share/cmf/lib/中。
注：我在官网上下载的是 .rpm格式。经过多次加压之后可以找到目标文件：mysql-connector-java-8.0.12.jar
- 在主节点初始化CM5的数据库
/opt/cm-5.12.1/share/cmf/schema/scm_prepare_database.sh mysql cm -hmaster -uroot -p123456 --scm-host master scm scm scm
[root@master schema]# ./scm_prepare_database.sh mysql cm -hmaster -uroot -p123456 --scm-host master scm scm scm --force
JAVA_HOME=/usr/lib/jvm/java-1.7.0-openjdk-devel.i686
Verifying that we can write to /opt/cm-5.12.1/etc/cloudera-scm-server
./scm_prepare_database.sh: line 379: /usr/lib/jvm/java-1.7.0-openjdk-devel.i686/bin/java: No such file or directory
--> Error 127, ignoring (because force flag is set)
Creating SCM configuration file in /opt/cm-5.12.1/etc/cloudera-scm-server
groups: cloudera-scm: No such user
Executing: /usr/lib/jvm/java-1.7.0-openjdk-devel.i686/bin/java -cp /usr/share/java/mysql-connector-java.jar:/usr/share/java/oracle-connector-java.jar:/opt/cm-5.12.1/share/cmf/schema/../lib/* com.cloudera.enterprise.dbutil.DbCommandExecutor /opt/cm-5.12.1/etc/cloudera-scm-server/db.properties com.cloudera.cmf.db.
./scm_prepare_database.sh: line 441: /usr/lib/jvm/java-1.7.0-openjdk-devel.i686/bin/java: No such file or directory
--> Error 127, ignoring (because force flag is set)
All done, your SCM database is configured correctly!
- Agent配置
修改/opt/cm-5.3.3/etc/cloudera-scm-agent/config.ini中的server_host为主节点的主机名。 - 同步Agent到其他节点
scp -r /opt/cm-5.3.3 root@slave1:/opt/
scp -r /opt/cm-5.3.3 root@slave1:/opt/
- 在所有节点创建cloudera-scm用户
useradd --system --home=/opt/cm-5.12.1/run/cloudera-scm-server/ --no-create-home --shell=/bin/false --comment "Cloudera SCM User" cloudera-scm
准备Parcels，用以安装CDH
- 将CHD5相关的Parcel包放到主节点的/opt/cloudera/parcel-repo/目录中（parcel-repo需要手动创建）。
相关的文件如下：
CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha1
CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel
manifest.json
最后将 CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha1 ，重命名为CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha，这点必须注意，否则，系统会重新下载CDH-5.12.1-1.cdh5.12.1.p0.3-el7.parcel.sha1 文件。
- 相关启动脚本
通过/opt/cm-5.12.1/etc/init.d/cloudera-scm-server start启动服务端。
通过/opt/cm-5.12.1/etc/init.d/cloudera-scm-agent start启动Agent服务。
我们启动的其实是个service脚本，需要停止服务将以上的start参数改为stop就可以了，重启是restart。
2018-09-25_105437.jpg
如上所示，启动服务端时报错，正在进一步学习。
未完 。。。。
修改
**[root@master ~]# vim /opt/cm-5.12.1/etc/init.d/cloudera-scm-server
**
chown -R cloudera-scm:cloudera-scm /opt/cm-5.7.0/run/cloudera-scm-agent
