Flink1.16.2集群部署
·
-
前往官网下载flink-1.16.2-bin-scala_2.12.tgz 安装包:
https://archive.apache.org/dist/flink/flink-1.16.2/

-
将安装包上传至服务器的/opt/software目录,解压文件:
tar -zxvf /opt/software/flink-1.16.2-bin-scala_2.12.tgz -C /opt/software
-
修改masters
hadoop01:8081
-
修改workers
hadoop01
hadoop02
hadoop03
-
修改 flink-conf.yaml
jobmanager.rpc.address: hadoop01
jobmanager.bind-host: 0.0.0.0
taskmanager.bind-host: 0.0.0.0
taskmanager.numberOfTaskSlots: 4
parallelism.default: 4
blob.server.port: 0
blob.server.ssl.enabled: false
blob.service.ssl.enabled: false
blob.service.cleanup.interval: 3600
# ==================== 高可用配置 ====================
high-availability: NONE
rest.bind-address: 0.0.0.0
-
分发集群
scp -r /opt/software/flink-1.16.2/ hadoop02:/opt/software
scp -r /opt/software/flink-1.16.2/ hadoop03:/opt/software
-
配置环境变量
vim /etc/profile.d/my_env.sh
#添加flink信息
export FLINK_HOME=/opt/software/flink-1.12.7
export PATH=$FLINK_HOME/bin:$PATH
-
分发环境变量
scp /etc/profile.d/my_env.sh hadoop02:/etc/profile.d/
scp /etc/profile.d/my_env.sh hadoop03:/etc/profile.d/
-
source 环境变量
hadoop01节点:source /etc/profile.d/my_env.sh
hadoop02节点:source /etc/profile.d/my_env.sh
hadoop03节点:source /etc/profile.d/my_env.sh
-
启动flink集群
./opt/software/flink-1.16.2/bin/start-cluster.sh
-
检查服务状态
在各个集群节点输入jps命令查看对应服务是否存在
hadoop01:

hadoop02:

hadoop03:

-
进入web页面
输入地址:http://hadoop01:8081(ip替换为真实ip)

执行jar中指定作业命令(示例):
flink run -c com.sa.scala.OracleToPgCDC /opt/software/flink-1.16.2/ks-stream-collect-plugins.jar

完成部署!
更多推荐
所有评论(0)