一、Hadoop集群配置如下




主机名IPJDK版本描述
m1     192.168.142.1441.8.0_161  Hadoop集群2.7  NameNode
s1      192.168.142.145 1.8.0_161  Hadoop集群2.7  DataNode1
s2     192.168.142.1461.8.0_161  Hadoop集群2.7  DataNode2



别忘了在/etc/目录下,修改host文件,添加主机名与IP地址之间的映射








二、ZooKeeper - 3.4.10 下载地址



百度网盘下载地址:zookeeper-3.4.10.tar.gz


官网下载地址:https://mirrors.tuna.tsinghua.edu.cn/apache/zookeeper/





三、ZooKeeper 安装



下载后的tar.gz上传至m1机器上的/usr/local目录下,并解压


tar -zxvf  zookeeper-3.4.10.tar.gz


解压后,进入zookeeper目录如下








四、ZooKeeper 配置




(1)配置环境变量


vim /etc/profile    (添加如下内容)






export JAVA_HOME=/usr/local/jdk1.8.0_161
export CLASSPATH=.:$JAVA_HOME/lib/dt.jar:$JAVA_HOME/lib/tools.jar
export HBASE_HOME=/usr/local/hbase-1.3.1
export HADOOP_HOME=/usr/local/hadoop-2.7.6
export ZOOKEEPER_HOME=/usr/local/zookeeper-3.4.10
export PATH=$PATH:$HADOOP_HOME/bin:$HBASE_HOME/bin:$ZOOKEEPER_HOME/bin:$HADOOP_HOME/sbin:$JAVA_HOME/bin



添加后,使用source命令,使刚才修改的配置立即生效


source /etc/profile




(2)配置conf



cp zoo_sample.cfg zoo.cfg







打开zoo.cfg,需要配置的内容如下:



vim zoo.cfg







dataDir=/usr/local/zookeeper-3.4.10/data
dataLogDir=/usr/local/zookeeper-3.4.10/data/logs
server.1=m1:2888:3888
server.2=s1:2888:3888
server.3=s2:2888:3888




1、指定Zookeeper的数据存放路径(/usr/local/zookeeper-3.4.10/data)

2、指定Zookeeper的数据日志存放路径(/usr/local/zookeeper-3.4.10/data/logs)

3、客户端连接zookeeper服务端的默认端口为2181

4、Zookeeper集群最好是奇数个节点,根据选举机制,半数以上的票数才能当选为leader(假如有两个节点,你选我,我选你,根据半数以上票数规则,是无法确定集群中到底是你来当leader还是我来当),因此,我们基于Hadoop的集群,分别以NaneNode和两个DataNode作为Zookeeper集群的三个节点

5、server.1 、server.2、server.3表示Zookeeper集群中各个节点所在的主机及服务端口号

6、端口2888:Zookeeper服务之间的通信端口,比如leader和follower之间的通信

7、端口3888:Zookeeper选举leader用的端口




(3)创建data和日志目录


切换到zookeeper目录下,创建data目录和日志目录(-p 递归创建目录,检查目录是否存在,不存在则创建)


mkdir -p data

mkdir -p data/logs



(4)创建myid文件


      由于Zookeeper集群中的每一个Zk服务都会有一个唯一标识自己的ID(zk进程ID),这个ID应用在myid和zoo.cfg两个文件中,其中myid文件中存放的是当前Zk服务的标识ID,而zoo.cfg中以server.ID来指定当前的Zk服务进程对应的是哪一台主机,当Zk服务启动的时候,会首先读取myid中的id标识,拿着这个id再去zoo.cfg文件中进行匹配查找,查找的目的是为了弄清楚当前Zk服务在整个ZooKeeper集群中的位置

      因此,如果没有指定Zk服务的myid,Zk是启动不了的


     


分别在m1、s1、s2三台主机上的ZK安装目录下的data目录下创建myid文件,并依次写入值:1,2,3如下:















(5)分别在三台机器上启动zk服务



zkServer.sh start







启动集群后,zkServer之间会互相进行投票选举,选出来一个leader,查看zkServer状态如下:




zkServer.sh status









(6)客户端连接Zk集群



 zkCli.sh -server m1:2181,s1:2181,s2:2181



Connecting to m1:2181,s1:2181,s2:2181
2018-07-12 21:04:11,689 [myid:] - INFO  [main:Environment@100] - Client environment:zookeeper.version=3.4.10-39d3a4f269333c922ed3db283be479f9deacaa0f, built on 03/23/2017 10:13 GMT
2018-07-12 21:04:11,693 [myid:] - INFO  [main:Environment@100] - Client environment:host.name=m1
2018-07-12 21:04:11,693 [myid:] - INFO  [main:Environment@100] - Client environment:java.version=1.8.0_161
2018-07-12 21:04:11,696 [myid:] - INFO  [main:Environment@100] - Client environment:java.vendor=Oracle Corporation
2018-07-12 21:04:11,696 [myid:] - INFO  [main:Environment@100] - Client environment:java.home=/usr/local/jdk1.8.0_161/jre
2018-07-12 21:04:11,696 [myid:] - INFO  [main:Environment@100] - Client environment:java.class.path=/usr/local/zookeeper-3.4.10/bin/../build/classes:/usr/local/zookeeper-3.4.10/bin/../build/lib/*.jar:/usr/local/zookeeper-3.4.10/bin/../lib/slf4j-log4j12-1.6.1.jar:/usr/local/zookeeper-3.4.10/bin/../lib/slf4j-api-1.6.1.jar:/usr/local/zookeeper-3.4.10/bin/../lib/netty-3.10.5.Final.jar:/usr/local/zookeeper-3.4.10/bin/../lib/log4j-1.2.16.jar:/usr/local/zookeeper-3.4.10/bin/../lib/jline-0.9.94.jar:/usr/local/zookeeper-3.4.10/bin/../zookeeper-3.4.10.jar:/usr/local/zookeeper-3.4.10/bin/../src/java/lib/*.jar:/usr/local/zookeeper-3.4.10/bin/../conf:.:/usr/local/jdk1.8.0_161/lib/dt.jar:/usr/local/jdk1.8.0_161/lib/tools.jar
2018-07-12 21:04:11,696 [myid:] - INFO  [main:Environment@100] - Client environment:java.library.path=/usr/java/packages/lib/amd64:/usr/lib64:/lib64:/lib:/usr/lib
2018-07-12 21:04:11,696 [myid:] - INFO  [main:Environment@100] - Client environment:java.io.tmpdir=/tmp
2018-07-12 21:04:11,696 [myid:] - INFO  [main:Environment@100] - Client environment:java.compiler=<NA>
2018-07-12 21:04:11,696 [myid:] - INFO  [main:Environment@100] - Client environment:os.name=Linux
2018-07-12 21:04:11,696 [myid:] - INFO  [main:Environment@100] - Client environment:os.arch=amd64
2018-07-12 21:04:11,697 [myid:] - INFO  [main:Environment@100] - Client environment:os.version=2.6.32-431.el6.x86_64
2018-07-12 21:04:11,697 [myid:] - INFO  [main:Environment@100] - Client environment:user.name=root
2018-07-12 21:04:11,697 [myid:] - INFO  [main:Environment@100] - Client environment:user.home=/root
2018-07-12 21:04:11,697 [myid:] - INFO  [main:Environment@100] - Client environment:user.dir=/root
2018-07-12 21:04:11,698 [myid:] - INFO  [main:ZooKeeper@438] - Initiating client connection, connectString=m1:2181,s1:2181,s2:2181 sessionTimeout=30000 watcher=org.apache.zookeeper.ZooKeeperMain$MyWatcher@446cdf90
Welcome to ZooKeeper!
2018-07-12 21:04:11,724 [myid:] - INFO  [main-SendThread(s2:2181):ClientCnxn$SendThread@1032] - Opening socket connection to server s2/192.168.142.146:2181. Will not attempt to authenticate using SASL (unknown error)
JLine support is enabled
2018-07-12 21:04:11,839 [myid:] - INFO  [main-SendThread(s2:2181):ClientCnxn$SendThread@876] - Socket connection established to s2/192.168.142.146:2181, initiating session
[zk: m1:2181,s1:2181,s2:2181(CONNECTING) 0] 2018-07-12 21:04:11,907 [myid:] - INFO  [main-SendThread(s2:2181):ClientCnxn$SendThread@1299] - Session establishment complete on server s2/192.168.142.146:2181, sessionid = 0x36491c9d1e60000, negotiated timeout = 30000

WATCHER::

WatchedEvent state:SyncConnected type:None path:null



Logo

权威|前沿|技术|干货|国内首个API全生命周期开发者社区

更多推荐