k8s opentelemetry 部署

清风笑烟语

336人浏览 · 2026-03-03 14:21:01

清风笑烟语 · 2026-03-03 14:21:01 发布

准备一个k8s 平台

在这里插入图片描述

进行部署：cert-manager

kubectl apply -f https://github.com/cert-manager/cert-manager/releases/download/v1.19.4/cert-manager.yaml

在这里插入图片描述

进行部署

# 可以参考官方文档
https://opentelemetry.io/zh/docs/platforms/kubernetes/operator/

operator 方式部署：

opentelemetry operator 部署

wget https://gh.llkk.cc/https://github.com/open-telemetry/opentelemetry-operator/releases/latest/download/opentelemetry-operator.yaml
# 下载完成后，进入这个文件中的17686添加一个
# --enable-go-instrumentation=true  # go 的自动注入
# --enable-nginx-instrumentation=true # nginx的自动注入
kubectl apply -f opentelemetry-operator.yaml

验证是否部署成功：

在这里插入图片描述

helm operator 部署

进行安装helm

wget https://get.helm.sh/helm-v4.1.1-linux-amd64.tar.gz
tar zxvf helm-v4.1.1-linux-amd64.tar.gz
mv linux-amd64/helm /usr/bin/helm

开始配置镜像仓库和下载：

helm repo add open-telemetry https://open-telemetry.github.io/opentelemetry-helm-charts # 添加仓库
helm repo update # 进行更新最新的
helm repo list # 进行查看是否配置成功
helm pull open-telemetry/opentelemetry-operator # 进行下载到本地
tar zxvf opentelemetry-operator-0.106.0.tgz

在这里插入图片描述

开始进行部署：

cd opentelemetry-operator 
kubectl create ns opentelemetry-collector-system
# 修改values.yaml中的87行修改成true
 85   autoInstrumentation:
 86     go:
 87       enabled: true
 # 开始进行部署：
helm install opentelemetry-collector    -n opentelemetry-collector-system .

开始部署collector服务

# opentelemetry-Collector.yaml
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: otel
spec:
  config: |
    # 接收器配置 - 定义如何接收遥测数据
    receivers:
      # OTLP (OpenTelemetry Protocol) 接收器
      otlp:
        protocols:
          # gRPC协议端点 - 用于接收gRPC格式的遥测数据
          grpc:
            endpoint: 0.0.0.0:4317
          # HTTP协议端点 - 用于接收HTTP格式的遥测数据
          http:
            endpoint: 0.0.0.0:4318

    # 导出器配置 - 定义遥测数据的发送目标
    exporters:
      # 调试导出器 - 支持链路追踪、指标、日志
      # 用于调试，输出详细的遥测数据信息到日志
      # 注意：v0.86.0之前使用 `logging` 代替 `debug`
      debug:
        verbosity: detailed
      # prometheusremotewrite:
      #   endpoint: 'http://prometheus:9090/api/v1/write'
      # # 开始普罗米接口：
      # prometheus:
      #   endpoint: "0.0.0.0:9464" # prometheus指标暴露端口
    processors:
      # 批处理器 - 将遥测数据分批发送，提高传输效率
      batch:
        timeout: 10s
        send_batch_size: 1024

    # 服务配置 - 定义启用的扩展和数据处理管道
    service:
      # 遥测配置 - 日志和性能监控配置
      telemetry:
        logs:
          # 日志级别：debug(调试信息最详细)
          level: debug
      # 数据管道配置 - 定义各类遥测数据的处理流程
      pipelines:
        # 链路追踪管道 - 处理分布式追踪数据
        traces:
          receivers: [ otlp ] # 从OTLP接收器接收数据
          processors: [ batch ] # 使用批处理器处理数据
          exporters: [ debug ] # 使用调试导出器输出追踪数据
        # 指标管道 - 处理性能指标数据
        metrics:
          receivers: [ otlp ] # 从OTLP接收器接收数据
          processors: [] # 不使用处理器，直接转发
          exporters: [ debug ] # 导出到Prometheus远程存储
        # 日志管道 - 处理应用日志数据
        logs:
          receivers: [ otlp ] # 从OTLP接收器接收数据
          processors: [ batch ] # 使用批处理器处理数据
          exporters: [ debug ] # 导出到Loki日志存储

在这里插入图片描述

开始配置自动注入

# 参考官方文档：https://opentelemetry.io/zh/docs/platforms/kubernetes/operator/automatic/
# opentelemetry-instrumentation.yaml
apiVersion: opentelemetry.io/v1alpha1
kind: Instrumentation
metadata:
  name: otel-instrumentation
spec:
  exporter:
    endpoint: http://otel-collector:4317
  propagators:
    - tracecontext
    - baggage
    - b3
  sampler:
    type: parentbased_traceidratio
    argument: "1"
  python:
    env:
      # Required if endpoint is set to 4317.
      # Python autoinstrumentation uses http/proto by default
      # so data must be sent to 4318 instead of 4317.
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: http://otel-collector:4318
  dotnet:
    env:
      # Required if endpoint is set to 4317.
      # Dotnet autoinstrumentation uses http/proto by default
      # See https://github.com/open-telemetry/opentelemetry-dotnet-instrumentation/blob/888e2cd216c77d12e56b54ee91dafbc4e7452a52/docs/config.md#otlp
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: http://otel-collector:4318
  go:
    env:
      # Required if endpoint is set to 4317.
      # Go autoinstrumentation uses http/proto by default
      # so data must be sent to 4318 instead of 4317.
      - name: OTEL_EXPORTER_OTLP_ENDPOINT
        value: http://otel-collector:4318

在这里插入图片描述

服务开启进行配置自动注入：

这边写了一个简单的服务：


package main

import (
	"github.com/gin-gonic/gin"
)

func main() {
	router := gin.Default()
	router.GET("/ping", func(c *gin.Context) {
		c.JSON(200, gin.H{"message": "pong"})
	})
	router.Run() // 默认监听 0.0.0.0:8080
}
// 可以使用我这边的镜像：里面也就是上面的服务
// swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/lushouxin/server:v1

开始给服务进行配置注释：

# server.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: server
  name: server
spec:
  replicas: 1
  selector:
    matchLabels:
      app: server
  strategy: {}
  template:
    metadata:
      labels:
        app: server
      annotations:
        instrumentation.opentelemetry.io/inject-go: "true" 
        instrumentation.opentelemetry.io/otel-go-auto-target-exe: "/app/main"  # 我服务的路径
    spec:
      containers:
      - image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/lushouxin/server:v1
        name: server
        securityContext:
          privileged: true
          runAsUser: 0
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: server
  name: server
spec:
  ports:
  - port: 8080
    protocol: TCP
    targetPort: 8080
  selector:
    app: server
  type: NodePort
status:
  loadBalancer: {}

进行验证

在这里插入图片描述

kubectl logs -f otel-collector-6bc6d65657-b6swn
在这里插入图片描述

开始进行配置存放数据的配置:这边服务暂时都通过deploy的方式安装，生产环境建议使用其他方式如helm等

开始配置普罗米修斯：

#  prometheus.yaml
apiVersion: v1
data:
  prometheus.yml: |
    # my global config
    global:
      scrape_interval: 15s # Set the scrape interval to every 15 seconds. Default is every 1 minute.
      evaluation_interval: 15s # Evaluate rules every 15 seconds. The default is every 1 minute.
    scrape_configs:
      - job_name: "prometheus"
        static_configs:
          - targets: ["localhost:9090"]
      - job_name: "otel"
        metrics_path: "/metrics"
        scrape_interval: 5s
        static_configs:
          - targets: ["otel-collector:8889"]
kind: ConfigMap
metadata:
  name: prometheus
---
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: prometheus
  name: prometheus
spec:
  replicas: 1
  selector:
    matchLabels:
      app: prometheus
  template:
    metadata:
      labels:
        app: prometheus
    spec:
      volumes:
      - name: prometheus
        configMap:
          name: prometheus
      containers:
      - image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/prom/prometheus:v3.10.0
        name: prometheus
        args:
        - --config.file=/etc/prometheus/prometheus.yml
        - --enable-feature=otlp-write-receiver
        volumeMounts:
        - name: prometheus
          mountPath: /etc/prometheus/prometheus.yml
          subPath: prometheus.yml
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: prometheus
  name: prometheus
spec:
  ports:
  - port: 9090
    protocol: TCP
    targetPort: 9090
  selector:
    app: prometheus
  type: NodePort

开始部署jaeger：

# jaeger.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: jaeger
  name: jaeger
spec:
  replicas: 1
  selector:
    matchLabels:
      app: jaeger
  strategy: {}
  template:
    metadata:
      labels:
        app: jaeger
    spec:
      containers:
      - image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/jaegertracing/all-in-one:1.71.0
        name: all-in-one
        env:
        - name: "COLLECTOR_ZIPKIN_HOST_PORT"
          value: ":9411"
        ports:
        - name: http
          containerPort: 4318
        - name: grpc
          containerPort: 4317
        - name: web
          containerPort: 16686
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: jaeger
  name: jaeger
spec:
  ports:
  - name: port-1
    port: 4318
    protocol: TCP
    targetPort: 4318
  - name: port-2
    port: 4317
    protocol: TCP
    targetPort: 4317
  - name: port-3
    port: 16686
    protocol: TCP
    targetPort: 16686
  selector:
    app: jaeger
  type: NodePort
status:
  loadBalancer: {}

开始安装loki：

apiVersion: v1
data:
  loki.yml: |
    auth_enabled: false
    server:
      http_listen_port: 3100
    ## 专用于接收处理OTEL数据
    limits_config:
      allow_structured_metadata: true
      volume_enabled: true
    common:
      path_prefix: /tmp/loki
      replication_factor: 1
      instance_addr: 0.0.0.0
      ring:
        kvstore:
          store: inmemory
    storage_config:
      tsdb_shipper:
        active_index_directory: /tmp/loki/index
        cache_location: /tmp/loki/index_cache
      filesystem:
        directory: /tmp/loki/chunks
    pattern_ingester:
      enabled: true
    compactor:
      retention_enabled: true
      retention_delete_delay: 720h 
      delete_request_store: filesystem
    schema_config:
      configs:
      - from: 2024-10-01
        store: tsdb
        object_store: filesystem
        schema: v13
        index:
          prefix: otlp_
          period: 24h 
kind: ConfigMap
metadata:
  name: loki
---
apiVersion: apps/v1
kind: Deployment
metadata:
  name: loki
  labels:
    app: loki
spec:
  replicas: 1
  selector:
    matchLabels:
      app: loki
  template:
    metadata:
      labels:
        app: loki
    spec:
      volumes:
      - name: loki
        configMap:
          name: loki
      containers:
      - name: loki
        image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/grafana/loki:3.4.1
        args:
        - "-config.file=/mnt/config/loki.yml" # 使用 ConfigMap 中的 loki.yml
        ports:
        - containerPort: 3100
          name: http
        volumeMounts:
        - name: loki
          mountPath: /mnt/config/loki.yml
          subPath: loki.yml
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: loki
  name: loki
spec:
  ports:
  - port: 3100
    protocol: TCP
    targetPort: 3100
  selector:
    app: loki
  type: NodePort
status:
  loadBalancer: {}

开始配置OpenTelemetry 配置文件：


# opentelemetry-operator.yaml 
apiVersion: opentelemetry.io/v1alpha1
kind: OpenTelemetryCollector
metadata:
  name: otel
spec:
  image: ghcr.io/open-telemetry/opentelemetry-collector-releases/opentelemetry-collector:0.145.0
  config: |
    receivers:
      otlp:
        protocols:
          grpc:
            endpoint: 0.0.0.0:4317
          http:
            endpoint: 0.0.0.0:4318
    exporters:
      debug:
        verbosity: detailed
      prometheusremotewrite:
        endpoint: 'http://prometheus:9090/api/v1/write'
      otlp_grpc/tempo:
        endpoint: http://tempo:4317
        tls:
          insecure: true
      otlp_grpc/jaeger:
        # 这个
        endpoint: jaeger:4317
        tls:
          insecure: true
      
      otlp_http/loki:
        endpoint: http://loki:3100/otlp
        tls:
          insecure: true
    processors:
      batch:
        timeout: 10s
        send_batch_size: 1024
    service:
      telemetry:
        logs:
          level: debug
      pipelines:
        traces:
          receivers: [ otlp ] 
          processors: [ batch ] 
          exporters: [ debug,otlp_grpc/tempo,otlp_grpc/jaeger ] 
        metrics:
          receivers: [ otlp ] 
          processors: [ batch ] 
          exporters: [ debug,prometheusremotewrite] 
        logs:
          receivers: [ otlp ] 
          processors: [ batch ] 
          exporters: [ debug,otlp_http/loki ]

# 开始进行加载配置：
kubectl apply -f opentelemetry-operator.yaml

开始进行验证是否把链路数据写入到jaeger 中：

在这里插入图片描述

发现链路没有问题之后，开始把jaeger替换成tempo

部署数据存储：rustfs

# 默认用户名和密码：rustfsadmin
# rustfs.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
  labels:
    app: rustfs
  name: rustfs
spec:
  replicas: 1
  selector:
    matchLabels:
      app: rustfs
  strategy: {}
  template:
    metadata:
      labels:
        app: rustfs
    spec:
      containers:
      - image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/rustfs/rustfs:1.0.0-alpha.85
        name: rustfs
        args: ["/data"] 
        resources: {}
        ports:
        - containerPort: 9000
          name: api
        - containerPort: 9001
          name: web        
---
apiVersion: v1
kind: Service
metadata:
  labels:
    app: rustfs
  name: rustfs
spec:
  ports:
  - name: api
    port: 9000
    protocol: TCP
    targetPort: 9000
  - name: web
    port: 9001
    protocol: TCP
    targetPort: 9001
  selector:
    app: rustfs
  type: NodePort
status:
  loadBalancer: {}

在这里插入图片描述

开始部署tempo

helm repo add grafana https://grafana.github.io/helm-charts
helm repo update
helm pull grafana/tempo

开始部署grafana

# Grafana server Deployment
apiVersion: apps/v1
kind: Deployment
metadata:
  name: grafana
spec:
  selector:
    matchLabels:
      name: grafana
  template:
    metadata:
      labels:
        name: grafana
    spec:
      containers:
        - image: swr.cn-north-4.myhuaweicloud.com/ddn-k8s/docker.io/grafana/grafana:12.3.3
          name: grafana
          ports:
            - containerPort: 3000
              name: ui
---
# Grafana UI Service NodePort
apiVersion: v1
kind: Service
metadata:
  labels:
    name: grafana
  name: grafana
spec:
  ports:
    - name: grafana-ui
      port: 3000
      targetPort: 3000
  selector:
    name: grafana
  type: NodePort

亚马逊云科技技术品牌专区

更多推荐

open-code-review Codex 插件教程：对话式调用评审技能指南

本文是一份面向新手的 **open-code-review Codex 插件教程**，教你在 Codex 中通过对话式方式调用 open-code-review 的 AI 代码评审技能。open-code-review 是阿里巴巴开源的 AI 代码评审工具，采用"确定性流水线 + LLM Agent"混合架构，内置多语言评审规则（NPE、线程安全、XSS、SQL 注入等），可在 Codex 里用一

亚马逊云科技技术品牌专区

OBS Studio 完整指南：免费开源录屏与直播软件从零上手

想做一场像模像样的直播，或者把屏幕内容录成教程视频，但一想到商用软件的订阅费就有些犹豫？其实你不需要花一分钱。OBS Studio 是一款免费开源的直播与录屏软件，支持 Windows、macOS 和 Linux，采集、混音、编码、推流、录制，一条流水线全包了。## 先搞清楚 OBS Studio 是什么OBS Studio 是一个"场景化"的采集合成工具：你把摄像头、窗口、浏览器、图片、

亚马逊云科技技术品牌专区

如何用MediaCrawler爬取小红书抖音数据？7大平台爬虫从安装到实战完整指南

MediaCrawler 是一个支持小红书、抖音、快手、B站、微博、百度贴吧、知乎的开源自媒体数据采集工具。它能按关键词、按帖子、按创作者主页三种方式抓取笔记/视频内容、评论甚至二级评论，并把结果存成 JSON、CSV、Excel 或数据库。适合做市场调研、竞品监控、内容选题的运营人员，以及想要现成爬虫框架而不想从零写 JS 逆向的开发者。## 从一次"手动抄数据"说起想象这个场景：你负责