Linux安装Kudu+Impala(RedHat or CentOS)

首先查看系统的版本，RedHat和CentOs通过cat /etc/redhat-release查看版本，这里我的系统版本是CentOS Linux release 7.3.1611 (Core)其他系统查看版本参考：https://www.cnblogs.com/wzk-0000/p/7483262.html一、安装Kudu1. 下载cloudera-kudu.repo地址：h...

我是蚁人

2929人浏览 · 2019-02-20 14:18:25

我是蚁人 · 2019-02-20 14:18:25 发布

首先查看系统的版本，RedHat和CentOs通过
cat /etc/redhat-release
查看版本，这里我的系统版本是CentOS Linux release 7.3.1611 (Core)

其他系统查看版本参考：https://www.cnblogs.com/wzk-0000/p/7483262.html

一、安装Kudu

1. 下载cloudera-kudu.repo
地址：https://kudu.apache.org/docs/installation.html
参考地址：https://cwiki.apache.org/confluence/display/IMPALA/Building+Impala

这里我的linux版本是7.3.1，所以选择CentOs7版本的库，将cloudera-kudu.repo放到/etc/yum.repos.d目录下

2.执行安装命令

sudo yum install kudu                         # Base Kudu files
sudo yum install kudu-master                  # Kudu master init.d service script and default configuration
(如果是集群安装的话，仅在master运行)

sudo yum install kudu-tserver                 # Kudu tablet server init.d service script and default configuration
(如果是集群安装，tserver仅在table服务的机器安装)
sudo yum install kudu-client0                 # Kudu C++ client shared library
sudo yum install kudu-client-devel            # Kudu C++ client SDK

3.安装Kudu RPMs（可选）

如果需要安装Kudu RPMs，先手动下载下来，然后使用命令sudo rpm -ivh <RPM to install>来安装。

4.启动服务

$ sudo service kudu-master start
$ sudo service kudu-tserver start

我在启动Kudu的时候报错了(日志路径/var/log/kudu/)：
F0124 15:35:50.682546 306685 master_main.cc:68] Check failed: _s.ok() Bad status: Service unavailable: Cannot initialize clock: Error reading clock. Clock considered unsynchronized

看样子像是时钟服务的问题，错误的解决：

sudo yum install ntp
sudo /etc/init.d/ntpd restart（ntp服务重启）或者 sudo service ntpd restart

如果重启有错：

sudo /bin/systemctl start ntpd.service

如果/etc/init.d/ntpd不是ntpd的服务目录，可以使用whereis ntpd查找一下目录

有任何启动的错误，参考：
https://kudu.apache.org/docs/troubleshooting.html
http://community.cloudera.com/t5/Beta-Releases-RecordService/Kudu-service-failed-to-start-on-Cannot-initialize-clock/td-p/47303

5.关闭服务

$ sudo service kudu-master stop
$ sudo service kudu-tserver stop

6.如果需要设置系统启动的时候启动kudu

$ sudo chkconfig kudu-master on                # RHEL / CentOS / SLES
$ sudo chkconfig kudu-tserver on               # RHEL / CentOS / SLES

$ sudo update-rc.d kudu-master defaults        # Debian / Ubuntu
$ sudo update-rc.d kudu-tserver defaults       # Debian / Ubuntu

~~~~~~~~~~~~~~~~~~~~分割线~~~~~~~~~~~~~~~~~~~~~~~~~

安装Impala

参考：http://lxw1234.com/archives/2017/06/862.htm

版本：2.7.0+cdh5.9.0

sudo wget http://archive.cloudera.com/beta/impala-kudu/redhat/7/x86_64/impala-kudu/0/RPMS/x86_64/impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo wget http://archive.cloudera.com/beta/impala-kudu/redhat/7/x86_64/impala-kudu/0/RPMS/x86_64/impala-kudu-catalog-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo wget http://archive.cloudera.com/beta/impala-kudu/redhat/7/x86_64/impala-kudu/0/RPMS/x86_64/impala-kudu-state-store-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm	
sudo wget http://archive.cloudera.com/beta/impala-kudu/redhat/7/x86_64/impala-kudu/0/RPMS/x86_64/impala-kudu-server-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo wget http://archive.cloudera.com/cdh5/redhat/7/x86_64/cdh/5/RPMS/noarch/bigtop-utils-0.7.0+cdh5.16.1+0-1.cdh5.16.1.p0.3.el7.noarch.rpm

sudo wget http://archive.cloudera.com/beta/impala-kudu/redhat/7/x86_64/impala-kudu/0/RPMS/x86_64/impala-kudu-shell-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo wget http://archive.cloudera.com/beta/impala-kudu/redhat/7/x86_64/impala-kudu/0/RPMS/x86_64/impala-kudu-udf-devel-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm

sudo wget http://archive.cloudera.com/beta/impala-kudu/redhat/7/x86_64/impala-kudu/0/RPMS/x86_64/impala-kudu-debuginfo-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm

1.执行安装

sudo yum install -y python-setuptools
sudo rpm -ivh bigtop-utils-0.7.0+cdh5.12.0+0-1.cdh5.12.0.p0.37.el6.noarch.rpm 
sudo rpm -ivh impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm --nodeps

（这里--nodeps代表了取消hadoop的一些依赖安装，否则会报错
error: Failed dependencies:
   hadoop is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   hadoop-hdfs is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   hadoop-yarn is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   hadoop-mapreduce is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   hbase is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   hive >= 0.12.0+cdh5.1.0 is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   zookeeper is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   hadoop-libhdfs is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   avro-libs is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   parquet is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   sentry >= 1.3.0+cdh5.1.0 is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
   sentry is needed by impala-kudu-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64
）

sudo rpm -ivh impala-kudu-catalog-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-state-store-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-server-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-shell-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm
sudo rpm -ivh impala-kudu-udf-devel-2.7.0+cdh5.9.0+0-1.cdh5.9.0.p0.11.el7.x86_64.rpm

2.配置

sudo vim /etc/default/bigtop-utils

export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.151-1.b12.el7_4.x86_64

这里如果找不到JAVA_HOME，可以通过whereis java或者which java，要是再找不到，去/usr/lib/jvm下看看吧

单机安装的话不需要其他的配置了

如果是集群的话，需要配置

vim /etc/default/impala 
IMPALA_CATALOG_SERVICE_HOST=10.255.206.155
IMPALA_STATE_STORE_HOST=10.255.206.155

这里两个host是指master的地址

3.hdfs和hive对接配置

cd /etc/impala/conf.dist
sudo cp /etc/hadoop/conf/core-site.xml core-site.xml
sudo cp /etc/hadoop/conf/hdfs-site.xml hdfs-site.xml
sudo cp /etc/hadoop/conf/hive-site.xml hive-site.xml
sudo cp /etc/hive/hive-env.sh hive-env.sh

修改hdfs-site.xml，添加

    <property>
          <name>dfs.datanode.hdfs-blocks-metadata.enabled</name>
          <value>true</value>
    </property>
    <property>
           <name>dfs.client.use.legacy.blockreader.local</name>
           <value>false</value>
    </property>
    <property>
       <name>dfs.datanode.data.dir.perm</name>
       <value>750</value>
    </property>
    <property>
       <name>dfs.block.local-path-access.user</name>
       <value>hadoop</value>
    </property>
    <property>
       <name>dfs.client.file-block-storage-locations.timeout</name>
       <value>10000</value>
    </property>

4.启动服务

sudo service impala-state-store start
sudo service impala-catalog start
sudo service impala-server start

测试impala的环境：
http://host:25020 (catalog)
http://host:25000 (impala server)
http://host:25010 (state-store)

执行impala-shell 进入impala命令行

如果现实not connected, 尝试使用connect localhost:8051

-------OK----享受其中吧

Linux

更多推荐

Linux虚拟文件系统之文件系统卸载（sys_umount())

Linux中卸载文件系统由umount系统调用实现，入口函数为sys_umount()。较于文件系统的安装较为简单，下面是具体的实现。1. /*sys_umont系统调用*/2. SYSCALL_DEFINE2(umount, char __user *, name, int, flags)3. {4.struct path path;

Linux

网卡速率和双工模式的配置

http://linux.chinaitlab.com/system/792187.html1、mii-tool 配置网络设备协商方式的工具； 1.1 mii-tool 介绍； mii-tool - view, manipulate media-independent interface status （mii-tool 是查看，管理介质的网络接口的状态）

Linux

Linux系统下超级终端Minicom的使用方法（例如：连接交换机，路由器等）转http://baike.baidu.com/view/2911642.htm?fr=ala0_1

Linux系统下超级终端Minicom的使用方法 　　Linux下的Minicom的功能与下的超级终端功能相似，适于在通过超级终端对设备的管理以及对嵌入操作系统的升级，现写出Minicom的使用手册： 　　1．启动minicom 　　以root权限登录系统 　　使用命令 　　minicom –s 则minicom启动，屏

Linux

所有评论(0)

查看更多评论

我是蚁人

@u010429286

已为社区贡献1条内容