logo
publist
写文章

简介

该用户还未填写简介

擅长的技术栈

可提供的服务

暂无可提供的服务

Ubuntu搭建 Spark、Hive、Hadoop+yarn

Ubuntu搭建 Spark、Hive、Hadoop+yarn操作系统:Ubuntu 16.04.12虚拟机:VMware® Workstation 16 Pro 16.0.0JDK:1.8.0_282Hadoop:hadoop-2.7.5 https://archive.apache.org/dist/hadoop/common/Spark:spark-2.4.7-bin-hadoop2.7 h

#hadoop#spark#hive +1
sparkstreaming连接kafka接收消息

一、写sparkstreaming代码打印接收需要调用steam.value()import org.apache.kafka.common.serialization.StringDeserializerimport org.apache.spark.SparkConfimport org.apache.spark.streaming.kafka010.ConsumerStrategies.Su

#kafka#spark
解决maven打包scala代码找不到主类问题、maven打包带依赖

折腾了一晚上加一上午,总算解决了。解决:1.21/04/13 20:55:27 INFO scheduler.EventLoggingListener: Logging events to hdfs://master:8020/directory/local-1618372526737Exception in thread "main" java.lang.NoClassDefFoundError

#maven#scala#大数据 +1
HBASE搭建过程报错解决汇总

错误1:running matser, logging to /opt/hbase-1.6.0/bin/../logs/hbase-jamjar-matser-master.outError: Could not find or load main class matser原因&解决:hbase-env.sh 中需要export你的Java路径错误2:running master, log

#hadoop#java#linux +2
Hive整合Hbase 解决map=100% reduce=0%的问题

搭建过程致谢:https://www.cnblogs.com/yfb918/p/10882323.htmlhttps://blog.csdn.net/dajiangtai007/article/details/79696853我当时在从hive插入到hbase表的时候,一直卡在map=100% reduce=0%查询日志找到原因:2021-04-17 06:18:12,884 ERROR [mai

#hadoop#大数据#java +2
到底了