Dolphinscheduler/海豚调度器的安装
Dolphinscheduler的安装一、预安装1.1 安装节点规划1.2 软件下载二、安装2.1 创建dolphinscheduler用户2.2 安装Java2.3 安装ZookeeperStep1.Step2.Step3.Step4.Step5.Step6.2.4 安装PostgreSQL数据库2.5 创建dolphinscheduler数据库2.6安装Dolphinscheduler软件一、
·
Dolphinscheduler的安装
一、预安装
1.1 安装节点规划
ips=${ips:-"ds-it-001,ds-it-002,ds-it-003"}
sshPort=${sshPort:-"22"}
masters=${masters:-"ds-it-001,ds-it-002"}
workers=${workers:-"ds-it-001:default,ds-it-002:default,ds-it-003:default"}
alertServer=${alertServer:-"ds-it-002"}
apiServers=${apiServers:-"ds-it-002,ds-it-003"}
installPath=${installPath:-"/data/dolphinscheduler"}
deployUser=${deployUser:-"dolphinscheduler"}
1.2 软件下载
二、安装
2.1 创建dolphinscheduler用户
#使用root用户创建用户
useradd dolphinscheduler
#配置sudo免密
sed -i '$adolphinscheduler ALL=(ALL) NOPASSWD: NOPASSWD: ALL' /etc/sudoers
sed -i 's/Defaults requirett/#Defaults requirett/g' /etc/sudoers
#配置免密登录
#在各节点生成公钥和私钥
su - dolphinscheduler
ssh-keygen -t rsa -P '' -f ~/.ssh/id_rsa
touch /home/dolphinscheduler/.ssh/authorized_keys
chmod 600 ~/.ssh/authorized_keys
#将各节点生成的公钥粘贴进authorized_keys
cat > ~/.ssh/authorized_keys
2.2 安装Java
#用服务器现有的Java配置，但是必须确保Java版本是在8以上
export JAVA_HOME=/opt/third/jdk
export JRE_HOME=$JAVA_HOME/jre
export PATH=$PATH:$JAVA_HOME/bin
export CLASSPATH=./:$JAVA_HOME/lib:$JAVA_HOME/jre/lib
#确认Java版本
[root@dp-test-003 ~]# java -version
java version "1.8.0_144"
Java(TM) SE Runtime Environment (build 1.8.0_144-b01)
Java HotSpot(TM) 64-Bit Server VM (build 25.144-b01, mixed mode)
2.3 安装Zookeeper
Step1.
#在三个节点上分别下载zookeeper安装包
curl -O https://dlcdn.apache.org/zookeeper/zookeeper-3.8.0/apache-zookeeper-3.8.0-bin.tar.gz
Step2.
#解压安装包
cd /opt
sudo tar -zxf apache-zookeeper-3.8.0-bin.tar.gz
Step3.
#创建zookeeper软链接
sudo ln -s /opt/apache-zookeeper-3.8.0-bin /opt/zookeeper
Step4.
#修改zoo.cfg配置文件
cd /opt/zookeeper/conf
sudo cp zoo_sample.cfg zoo.cfg
sudo vi zoo.cfg
#修改内容如下：
dataDir=/data/zookeeper/zkdata
server.1=10.10.14.134:2888:3888;2181
server.2=10.10.14.212:2888:3888;2181
server.3=10.10.14.211:2888:3888;2181
注意如下节点跟IP务必匹配，否则启动失败，并提示如下错误：
2022-05-09 16:12:33,558 [myid:] - ERROR [QuorumPeer[myid=2](plain=0.0.0.0:2181)(secure=disabled):o.a.z.s.q.Leader@325] - Couldn't bind to /10.10.14.211:2888
java.net.BindException: Cannot assign requested address (Bind failed)
at java.net.PlainSocketImpl.socketBind(Native Method)
at java.net.AbstractPlainSocketImpl.bind(AbstractPlainSocketImpl.java:387)
at java.net.ServerSocket.bind(ServerSocket.java:375)
at java.net.ServerSocket.bind(ServerSocket.java:329)
at org.apache.zookeeper.server.quorum.Leader.createServerSocket(Leader.java:322)
at org.apache.zookeeper.server.quorum.Leader.lambda$new$0(Leader.java:301)
at java.util.stream.ReferencePipeline$3$1.accept(ReferencePipeline.java:193)
at java.util.concurrent.ConcurrentHashMap$KeySpliterator.forEachRemaining(ConcurrentHashMap.java:3527)
at java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:481)
at java.util.stream.AbstractPipeline.wrapAndCopyInto(AbstractPipeline.java:471)
at java.util.stream.ForEachOps$ForEachOp.evaluateSequential(ForEachOps.java:151)
at java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateSequential(ForEachOps.java:174)
at java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:234)
at java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:418)
at org.apache.zookeeper.server.quorum.Leader.(Leader.java:304)
at org.apache.zookeeper.server.quorum.QuorumPeer.makeLeader(QuorumPeer.java:1340)
at org.apache.zookeeper.server.quorum.QuorumPeer.run(QuorumPeer.java:1551)
Step5.
#创建myid文件
#如下步骤在zookeeper集群3个节点上分别执行，myid文件对应节点序号，例如节点1只需写入1，节点2写入2，节点3写入3
mkdir -p /data/zookeeper/zkdata
echo 1 > /data/zookeeper/zkdata/myid
echo 2 > /data/zookeeper/zkdata/myid
echo 3 > /data/zookeeper/zkdata/myid
Step6.
#启动zookeeper
cd /opt/zookeeper
sudo ./bin/zkServer.sh start
#启动成功会显示如下
[root@dp-test-001 zookeeper]# ./bin/zkServer.sh start
ZooKeeper JMX enabled by default
Using config: /opt/zookeeper/bin/../conf/zoo.cfg
Starting zookeeper ... STARTED
2.4 安装PostgreSQL数据库
访问postgresql官网选择下载对应的版本
https://www.postgresql.org/download/linux/redhat/
#登录10.10.14.134和10.10.14.211节点，安装数据库软件
# Install the repository RPM:
sudo yum install -y https://download.postgresql.org/pub/repos/yum/reporpms/EL-7-x86_64/pgdg-redhat-repo-latest.noarch.rpm
# Install PostgreSQL:
sudo yum install -y postgresql14-server
#登录主节点10.10.14.134初始化数据库
sudo /usr/pgsql-14/bin/postgresql-14-setup initdb
sudo systemctl enable postgresql-14
sudo systemctl start postgresql-14
2.5 创建dolphinscheduler数据库
su - postgres
psql
postgres=# create role dolphinscheduler with login password 'xxxxx';
CREATE ROLE
postgres=# create database dolphinscheduler owner dolphinscheduler;
CREATE DATABASE
2.6 安装Dolphinscheduler软件
注意下面的步骤只需在一个节点上执行，在执行部署脚本时，会复制软件至配置文件中指定的节点和目录。
#解压软件
cd /opt
curl -O https://dlcdn.apache.org/dolphinscheduler/3.0.0-alpha/apache-dolphinscheduler-3.0.0-alpha-bin.tar.gz
tar -zxf apache-dolphinscheduler-3.0.0-alpha-bin.tar.gz
sudo chown -R dolphinscheduler.dolphinscheduler /opt/apache-dolphinscheduler-3.0.0-alpha-bin
#修改dolphinscheduler_env.sh配置文件，JAVA_HOME，数据库、zookeeper地址
export JAVA_HOME=${JAVA_HOME:-/opt/third/jdk}
export SPRING_JACKSON_TIME_ZONE=${SPRING_JACKSON_TIME_ZONE:-UTC}
export DATABASE=${DATABASE:-postgresql}
export SPRING_PROFILES_ACTIVE=${DATABASE}
export SPRING_DATASOURCE_DRIVER_CLASS_NAME=org.postgresql.Driver
export SPRING_DATASOURCE_URL=jdbc:postgresql://dsdb-dp.gz.cvte.cn:5432/dolphinscheduler
export SPRING_DATASOURCE_USERNAME=dolphinscheduler
export SPRING_DATASOURCE_PASSWORD=xxxx
export SPRING_CACHE_TYPE=${SPRING_CACHE_TYPE:-none}
export MASTER_FETCH_COMMAND_NUM=${MASTER_FETCH_COMMAND_NUM:-10}
export REGISTRY_TYPE=${REGISTRY_TYPE:-zookeeper}
export REGISTRY_ZOOKEEPER_CONNECT_STRING=${REGISTRY_ZOOKEEPER_CONNECT_STRING:-ds-it-001:2181,ds-it-002:2181,ds-it-003:2181}
#修改install_env.sh
ips=${ips:-"ds-it-001,ds-it-002,ds-it-003"}
sshPort=${sshPort:-"22"}
masters=${masters:-"ds-it-001,ds-it-002"}
workers=${workers:-"ds-it-001:default,ds-it-002:default,ds-it-003:default"}
alertServer=${alertServer:-"ds-it-002"}
apiServers=${apiServers:-"ds-it-002,ds-it-003"}
installPath=${installPath:-"/data/dolphinscheduler"}
deployUser=${deployUser:-"dolphinscheduler"}
#初始化数据库
su - dolphinscheduler
cd /opt/apache-dolphinscheduler-3.0.0-alpha-bin
sh tools/bin/create-schema.sh
#部署dolphinscheduler集群
su - dolphinscheduler
cd /opt/apache-dolphinscheduler-3.0.0-alpha-bin
sh ./bin/install.sh
权威｜前沿｜技术｜干货｜国内首个API全生命周期开发者社区
更多推荐
1
0- 0
已为社区贡献1条内容
所有评论(0)