Hive加载数据报相关权限问题
1.上传文件到linux文件系统[root@node1 mydata]# lltotal 129648-rw-r--r--. 1 root root 120734753 Dec6 12:53 bigtable.bigtable-rw-r--r--. 1 root root12018355 Dec6 12:51 smalltable.smalltable2.加载数据到hive创建表create ta
·
平台:H3C DataEngine
1.上传文件到linux文件系统
[root@node1 mydata]# ll
total 129648
-rw-r--r--. 1 root root 120734753 Dec 6 12:53 bigtable.bigtable
-rw-r--r--. 1 root root 12018355 Dec 6 12:51 smalltable.smalltable
2.加载数据到hive
创建表
create table bigtable(
id int,
time string,
sessionId string,
key string,
up int,
down int,
url string)
row format delimited fields terminated by '\t';
试图使用load命令加载本地文件到hive中(环境是使用beeline连接hive,因为涉及hdfs的操作,所以登录用户必须具备hdfs操作权限,因此选择使用hdfs用户登录hive)
0: jdbc:hive2://node1:10000/> load data local inpath '/usr/hdp/mydata/bigtable.bigtable' into table bigtable;
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied:
Principal [name=hdfs, type=US ER] does not have following privileges for operation
LOAD [[INSERT, DELETE, OBJECT OWNERSHIP] on Object [type=LOCAL_URI, na me=file:/usr/hdp/mydata/bigtable.bigtable]] (state=42000,code=40000)
报权限问题,给出的提示是[INSERT, DELETE, OBJECT OWNERSHIP]
,首先是[INSERT, DELETE]
权限问题,该问题是由于hdfs用户对该文件的权限不足,上面看到只有可读权限,因此提高文件的权限
[root@node1 mydata]# ll
total 129648
-rw-rw-rw-. 1 root root 120734753 Dec 6 12:53 bigtable.bigtable
-rw-rw-rw-. 1 root root 12018355 Dec 6 12:51 smalltable.smalltable
在此加载,发现[INSERT, DELETE]
权限问题已经解决,但[OBJECT OWNERSHIP]
权限问题依然存在
0: jdbc:hive2://node1:10000/> load data local inpath '/usr/hdp/mydata/bigtable.bigtable' into table bigtable;
Error: Error while compiling statement: FAILED: HiveAccessControlException Permission denied:
Principal [name=hdfs, type=USER] does not have following privileges for operation
LOAD [[OBJECT OWNERSHIP] on Object [type=LOCAL_URI, name=file:/usr/hdp /mydata/bigtable.bigtable]] (state=42000,code=40000)
这个问题就很显而易见了,使用的是hdfs用户登录,但文件的所有者属于root,因此修改文件的所有者再次导入
[root@node1 mydata]# ll
total 129648
-rw-rw-rw-. 1 hdfs root 120734753 Dec 6 12:53 bigtable.bigtable
-rw-rw-rw-. 1 hdfs root 12018355 Dec 6 12:51 smalltable.smalltable
问题解决
0: jdbc:hive2://node1:10000/> load data local inpath '/usr/hdp/mydata/bigtable.bigtable' into table bigtable;
INFO : Loading data to table hc.bigtable from file:/usr/hdp/mydata/bigtable.bigtable
INFO : Table hc.bigtable stats: [numFiles=1, totalSize=120734753]
No rows affected (6.948 seconds)
检验数据是否导入
0: jdbc:hive2://node1:10000/> select url from bigtable limit 10;
+----------------------------------------------------------------+--+
| url |
+----------------------------------------------------------------+--+
| http://www.qiyi.com/ |
| http://www.booksky.org/BookDetail.aspx?BookID=1050804&Level=1 |
| http://www.bblianmeng.com/ |
| http://lib.scnu.edu.cn/ |
| http://proxyie.cn/ |
| http://movie.douban.com/review/1128960/ |
| http://www.youku.com/ |
| http://hf.house365.com/ |
| http://www.kz321.com/ |
| http://www.iqshuma.com/ |
+----------------------------------------------------------------+--+
10 rows selected (0.55 seconds)
同理加载hdfs上的数据包权限异常同样这样处理。
更多推荐
已为社区贡献1条内容
所有评论(0)