Exploring what a running pod is]# docker ps | grep pause9220b8d70a54k8s.gcr.io/pause:3.1"/pause"...

submit sqlto thriftserver by beelinerun thriftserver in a podsh sbin/start-thriftserver.sh \--master k8s://https://kubernetes.default.svc.cluster.local:443 \--name spark-thriftserver \...

Kubernetes Operator for Apache Spark DesignIntroductionIn Spark 2.3, Kubernetes becomes an official scheduler backend for Spark, additionally to the standalone scheduler, Mesos, and Yarn. Compared w..

headless一旦创建了 headless ，在 k8s 里便是全局的root@spark-worker-dcpmm-statefulset-aep1-2:/# getent hostssparkthrift-headless10.60.218.26sparkthrift-headless.spark-project.svc.cluster.localroot@spark-w...

安装 spark-on-k8s-operatorhelm repo add incubator http://storage.googleapis.com/kubernetes-charts-incubatorhelm install incubator/sparkoperator --namespace spark-operatorNAME:eponymous-fleeLAST D...

启动命令shsbin/start-thriftserver.sh–master k8s://https://192.168.99.108:8443–name spark-thriftserver–conf spark.executor.instances=1–conf spark.kubernetes.container.image=zhixingheyitian/spark...

influxdb 生态圈 自带采集数据工具 telegraf，telegraf集成了很多功能，包括订阅kafka数据，但是由于telegraf 订阅 kafka 性能太低，所以自定义开发了 flume sink插件，使得 订阅数据性能获得了极大提高。InfluxSink样例代码如下，以飨读者。package org.apache.flume.sink.influx;i...

由于公司大数据架构是，flume收集所有数据，流到kafka。kafka自带相关metrics测量，而flume没有，导致数据定位发生困难。为此，特地研究了kafka metrics的相关源码模块，将其实现原理移植到flumekafka metrics 实现方式kafka 相关测量模块主要在入口处，是用scala语言实现。通过研究，发现kafka 是用 开源测量库 yammer...

检查kvm模块是否安装[root@localhost ~]# lsmod |grep kvmkvm_intel 55496 32kvm 337772 1 kvm_intel安装kubectl添加 repo配置文件，/etc/yum.repos.d/kubernetes.repo[kubernetes]name=Kubernetesbaseurl=https://packages....

环境准备To disable SELinux, run the following command:setenforce 0But this just disables it temporarily (until the next reboot). To disable it permanently, edit the/etc/selinux/config file and chang...