tez-ui docker镜像打包配置并部署K8S
tez-ui安装部署
·
一、下载tez ui安装包并修改配置文件
1.1 选择版本下载, tez-ui下载官方地址
mkdir tez-ui && cd tez-ui
wget https://repository.apache.org/content/repositories/releases/org/apache/tez/tez-ui/0.9.1/tez-ui-0.9.1.war
1.2 解压war包,修改配置文件
# 解压
unzip tez-ui-0.9.1.war
# 修改配置文件 将ENV.hosts.timeline和ENV.hosts.rm修改为对应地址
vim config/configs.env
ENV = {
hosts: {
/*
* Timeline Server Address:
* By default TEZ UI looks for timeline server at http://localhost:8188, uncomment and change
* the following value for pointing to a different address.
*/
timeline: "https://xxxx:8190",
/*
* Resource Manager Address:
* By default RM REST APIs are expected to be at http://localhost:8088, uncomment and change
* the following value to point to a different address.
*/
rm: "https://xxxx:8090",
...
timeline地址可在yarn配置文件查看yarn.timeline-service.webapp.https.address
rm地址可在yarn配置文件查看yarn.resourcemanager.webapp.https.address
二、生成docker镜像
2.1 编写对应Dockerfile
vim Dockerfile
# 基于镜像基础
FROM tomcat
# 维护者信息
MAINTAINER name dw
# 创建tez ui目录
RUN mkdir -p /usr/local/tomcat/webapps/tez-ui
# 复制文件到tez-ui目录
COPY . ./webapps/tez-ui/
# 启动
CMD ["catalina.sh", "run"]
2.2 打包镜像并上传到镜像仓库
docker build -t xxxx/tez-ui:v0.9.1 .
docker push xxxx/tez-ui:v0.9.1
2.3 运行镜像测试是否有问题
docker run --name tez-ui -p 8333:8080 -d xxxx/tez-ui:v0.9.1
此时应该可以通过http://IP:8333/tez-ui/ 访问
三、配置tez、yarn、hive
3.1 修改tez-site
tez.am.view-acls=*
tez.am.tez-ui.history-url.template=__HISTORY_URL_BASE__?viewPath=/#/tez-app/__APPLICATION_ID__
tez.history.logging.service.class=org.apache.tez.dag.history.logging.ats.ATSHistoryLoggingService
tez.tez-ui.history-url.base=http://xxxx:8333/tez-ui/
3.2 修改yarn-site
yarn.timeline-service.enabled=true
yarn.timeline-service.http-cross-origin.enabled=true
yarn.resourcemanager.system-metrics-publisher.enabled=true
3.3 修改hive配置
hive.server2.enable.doAs=true
hive.exec.failure.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook
hive.exec.post.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook,org.apache.atlas.hive.hook.HiveHook
hive.exec.pre.hooks=org.apache.hadoop.hive.ql.hooks.ATSHook
hive_timeline_logging_enabled=true
重启docker后服务应该可用了
四、部署至K8S
4.1 编写k8s文件
创建命名空间
kubectl create ns tezui
创建deployment文件
vim tezui-deployment.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
labels:
app: tezui
name: tezui
namespace: tezui
spec:
replicas: 1
selector:
matchLabels:
app: tezui
template:
metadata:
labels:
app: tezui
spec:
containers:
- image: xxxx/tez-ui:v0.9.1
name: tezui-container
ports:
- containerPort: 8080
kubectl apply -f tezui-deployment.yaml
创建service文件
vim tezui-service
apiVersion: v1
kind: Service
metadata:
name: tezui-service
namespace: tezui
spec:
type: NodePort
ports:
- port: 8080
nodePort: 30001
selector:
app: tezui
kubectl apply -f tezui-service
现在可以通过k8s节点的30001端口访问了。http:xxxx:30001/tez-ui/
有需要的话再加上ingress使用域名代理
更多推荐
已为社区贡献6条内容
所有评论(0)