任务一:Hadoop下载、上传、解压

1.下载

网址:https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz

wget "https://dlcdn.apache.org/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz"

2.解压

tar -zxvf hadoop-3.3.6.tar.gz -C /export/server

3.配置软连接

cd /export/server
ln -s /export/server/hadoop-3.3.6 hadoop
cd hadoop

任务二:解压后修改配置

1.修改配置文件

1.1修改workers文件
cd etc/hadoop
vim workers
node1
node2
node3
1.2配置hadoop-env.sh文件
export JAVA_HOME=/export/server/jdk
export HADOOP_HOME=/export/server/hadoop
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_LOG_DIR=$HADOOP_HOME/logs
1.3配置core-site.xml文件
<property>
    <name>fs.defaultFS</name>
    <value>hdfs://node1:8020</value>
</property>
<property>
    <name>io.file.buffer.size</name>
    <value>131072</value>
</property>
1.4配置hdfs-site.xml文件
<property>
    <name>dfs.datanode.data.dir.perm</name>
    <value>700</value>
</property>
<property>
    <name>dfs.namenode.name.dir</name>
    <value>/data/nn</value>
</property>
<property>
    <name>dfs.namenode.hosts</name>
    <value>node1,node2,node3</value>
</property>
<property>
    <name>dfs.blocksize</name>
    <value>268435456</value>
</property>
<property>
    <name>dfs.namenode.handler.count</name>
    <value>100</value>
</property>
<property>
    <name>dfs.datanode.data.dir</name>
    <value>/data/dn</value>
</property>
1.5创建dn文件夹
1.5.1node1
mkdir -p /data/nn
mkdir /data/dn
1.5.2node2/3
mkdir -p /data/dn
1.6将Hadoop文件夹复制到node2/3
1.6.1node1
cd /export/server
scp -r hadoop-3.3.6 node2:`pwd`/
scp -r hadoop-3.3.6 node3:`pwd`/
1.6.2node2/3
ln -s /export/server/hadoop-3.3.6/export/server/hadoop

2.配置环境变量

2.1在文件底部添加
vim/etc/profile
export HADOOP_HOME=/export/server/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
2.2在node2/3配置同样的环境变量

3.授权Hadoop用户(在三台均进行)

chown -R hadoop:hadoop /data
chown -R hadoop:hadoop /export

任务三:运行

1.格式化

su - hadoop
hadoop namenode -format

2.启动

start-dfs.sh
stop-dfs.sh
#如果遇到命令未找到的错误,表明环境变量未配置好,可以以绝对路径执行
/export/server/hadoop/sbin/start-dfs.sh
/export/server/hadoop/sbin/stop-dfs.sh

更多推荐