Linux环境安装Hadoop单机版
一、安装版本hadoop-2.7.5下载地址:Index of /dist/hadoop/core (apache.org)二、准备工作1、安装jdk 8及以上版本2、修改主机名称(可以不修改)3、关闭防火墙三、安装1、解压到/opt目录下tar -zxvf hadoop-2.7.5.tar.gz -C /opt/2、配置hadoop环境变量vim /etc/profileexport HADOO
一、安装版本
hadoop-2.7.5
下载地址:Index of /dist/hadoop/core (apache.org)
二、准备工作
1、安装jdk 8及以上版本
2、修改主机名称(可以不修改)
3、关闭防火墙
三、安装
1、解压到/opt目录下
tar -zxvf hadoop-2.7.5.tar.gz -C /opt/
2、配置hadoop环境变量
vim /etc/profile
export HADOOP_HOME=/opt/hadoop-2.7.5
export PATH=$PATH:$HADOOP_HOME/bin
source /etc/profile
3、配置Haddop
3.1、hadoop-env.sh
vi /opt/hadoop-2.7.5/etc/hadoop/hadoop-env.sh
找到# The java implementation to use.将其下面的一行改为:
#export JAVA_HOME=${JAVA_HOME}
export JAVA_HOME=export JAVA_HOME=/usr/lib/jvm/java-1.8.0-openjdk-1.8.0.222.b03-1.el7.x86_64/j
3.2、配置core-site.xml
vim /opt/hadoop-2.7.5/etc/hadoop/core-site.xml
<configuration>
<property>
<name>hadoop.tmp.dir</name>
<value>file:///opt/hadoop-2.7.5</value>
<description>Abase for other temporary directories.</description>
</property>
<property>
<name>fs.defaultFS</name>
<value>hdfs://master:8888</value>
</property>
</configuration>
3.3、配置hdfs-site.xml
vim /opt/hadoop-2.7.5/etc/hadoop/hdfs-site.xml
<configuration>
<property>
<name>dfs.replication</name>
<value>1</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>file:///opt/hadoop-2.7.5/tmp/dfs/name</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>file:///opt/hadoop-2.7.5/tmp/dfs/data</value>
</property>
</configuration>
4、启动hadoop
4.1、第一次启动时需要先格式化
cd /opt/hadoop-2.7.5
./bin/hdfs namenode -format
如果遇到
Re-format filesystem in Storage Directory /opt/hadoop-2.7.5/tmp/dfs/name ? (Y or N)
输入:Y
4.2、启动
./sbin/start-dfs.sh
4.3、停止
./sbin/stop-dfs.sh
4.4、验证
http://localhost:50070
5、配置yarn
5.1、配置mapred-site.xml
cd /opt/hadoop-2.7.5/etc/hadoop/
p mapred-site.xml.template mapred-site.xml
vim mapred-site.xml
<configuration>
<property>
<name>mapred.job.tracker</name>
<value>master:9001</value>
</property>
<property>
<name>mapred.local.dir</name>
<value>/opt/hadoop-2.7.5/tmp/var</value>
</property>
<property>
<name>mapreduce.framework.name</name>
<value>yarn</value>
</property>
</configuration>
5.2、配置yarn-site.xml
vim yarn-site.xml
<configuration>
<!-- reducer取数据的方式是mapreduce_shuffle -->
<property>
<name>yarn.nodemanager.aux-services</name>
<value>mapreduce_shuffle</value>
</property>
</configuration>
5.3、yarn启动与停止
cd /opt/hadoop-2.7.5
./sbin/start-yarn.sh
./sbin/stop-yarn.sh
浏览器查看:http://localhost:8088
jps查看进程
更多推荐
所有评论(0)