1015 Hadoop部署实践(任务一/二/三)
·
任务一:Hadoop下载、上传、解压
1.下载
网址:https://www.apache.org/dyn/closer.cgi/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz
wget "https://dlcdn.apache.org/hadoop/common/hadoop-3.3.6/hadoop-3.3.6.tar.gz"

2.解压
tar -zxvf hadoop-3.3.6.tar.gz -C /export/server
3.配置软连接
cd /export/server
ln -s /export/server/hadoop-3.3.6 hadoop
cd hadoop
任务二:解压后修改配置
1.修改配置文件
1.1修改workers文件
cd etc/hadoop
vim workers
node1
node2
node3
1.2配置hadoop-env.sh文件
export JAVA_HOME=/export/server/jdk
export HADOOP_HOME=/export/server/hadoop
export HADOOP_CONF_DIR=$HADOOP_HOME/etc/hadoop
export HADOOP_LOG_DIR=$HADOOP_HOME/logs
1.3配置core-site.xml文件
<property>
<name>fs.defaultFS</name>
<value>hdfs://node1:8020</value>
</property>
<property>
<name>io.file.buffer.size</name>
<value>131072</value>
</property>
1.4配置hdfs-site.xml文件
<property>
<name>dfs.datanode.data.dir.perm</name>
<value>700</value>
</property>
<property>
<name>dfs.namenode.name.dir</name>
<value>/data/nn</value>
</property>
<property>
<name>dfs.namenode.hosts</name>
<value>node1,node2,node3</value>
</property>
<property>
<name>dfs.blocksize</name>
<value>268435456</value>
</property>
<property>
<name>dfs.namenode.handler.count</name>
<value>100</value>
</property>
<property>
<name>dfs.datanode.data.dir</name>
<value>/data/dn</value>
</property>
1.5创建dn文件夹
1.5.1node1
mkdir -p /data/nn
mkdir /data/dn
1.5.2node2/3
mkdir -p /data/dn
1.6将Hadoop文件夹复制到node2/3
1.6.1node1
cd /export/server
scp -r hadoop-3.3.6 node2:`pwd`/
scp -r hadoop-3.3.6 node3:`pwd`/
1.6.2node2/3
ln -s /export/server/hadoop-3.3.6/export/server/hadoop
2.配置环境变量
2.1在文件底部添加
vim/etc/profile
export HADOOP_HOME=/export/server/hadoop
export PATH=$PATH:$HADOOP_HOME/bin:$HADOOP_HOME/sbin
2.2在node2/3配置同样的环境变量
3.授权Hadoop用户(在三台均进行)
chown -R hadoop:hadoop /data
chown -R hadoop:hadoop /export
任务三:运行
1.格式化
su - hadoop
hadoop namenode -format



2.启动
start-dfs.sh
stop-dfs.sh
#如果遇到命令未找到的错误,表明环境变量未配置好,可以以绝对路径执行
/export/server/hadoop/sbin/start-dfs.sh
/export/server/hadoop/sbin/stop-dfs.sh

更多推荐
所有评论(0)