1. 物理机/虚拟机系统准备(3台机器都执行)

  • 1 master + 2 node
  • ubuntu 18.04
  • 换源:vim /etc/apt/sources.list
默认注释了源码镜像以提高 apt update 速度,如有需要可自行取消注释
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-updates main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-backports main restricted universe multiverse
deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-security main restricted universe multiverse
# 预发布软件源,不建议启用
# deb https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
# deb-src https://mirrors.tuna.tsinghua.edu.cn/ubuntu/ bionic-proposed main restricted universe multiverse
  • 更新源
sudo apt update
  • 禁用交换分区(3台机器执行)
swapoff -a
  • 安装docker
sudo apt install docker.io
  • 修改主机名(每个机器都操作)
  • Master
vim /etc/hostname
改为:kubeedge-master
  • Node-01
vim /etc/hostname
改为:kubeedge-edge01
  • Node-02
vim /etc/hostname
改为:kubeedge-edge02

2. Master 节点配置

  • 依次执行以下命令
sudo apt-get update && sudo apt-get install -y apt-transport-https
apt install curl
curl https://mirrors.aliyun.com/kubernetes/apt/doc/apt-key.gpg | apt-key add -
  • 然后,执行以下命令
cat <<EOF >/etc/apt/sources.list.d/kubernetes.list
deb https://mirrors.aliyun.com/kubernetes/apt/ kubernetes-xenial main
EOF
  • 更新源
apt-get update
  • 安装Kubernetes
apt install kubelet=1.22.15-00 kubectl=1.22.15-00 kubeadm=1.22.15-00
  • 初始化Master节点(首先 sudo su,使用root用户)
kubeadm init --image-repository=registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --kubernetes-version=v1.22.15
  • 如果初始化不成功,日志如下:
    在这里插入图片描述
  • 那么执行以下命令:
vim /etc/docker/daemon.json
  • 复制以下内容到 daemon.json文件中
{
    "exec-opts": ["native.cgroupdriver=systemd"]
}
  • 然后执行以下命令:
systemctl daemon-reload
systemctl restart docker
systemctl restart kubelet
kubeadm reset
  • 然后,重新运行init命令,即:
kubeadm init --image-repository=registry.aliyuncs.com/google_containers --pod-network-cidr=10.244.0.0/16 --kubernetes-version=v1.22.15
  • 然后,运行以下命令:
mkdir -p $HOME/.kube
cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
chown $(id -u):$(id -g) $HOME/.kube/config
export KUBECONFIG=/etc/kubernetes/admin.conf
  • 最后通过以下命令查看Master是否成功:
kubectl get pods -o wide --all-namespaces

在这里插入图片描述

  • Master节点配置网络插件 flannel
  • 创建 kube-flannel.yml 文件,复制以下内容
vim ~/kube-flannel.yml
---
kind: Namespace
apiVersion: v1
metadata:
  name: kube-flannel
  labels:
    k8s-app: flannel
    pod-security.kubernetes.io/enforce: privileged
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: flannel
  name: flannel
rules:
- apiGroups:
  - ""
  resources:
  - pods
  verbs:
  - get
- apiGroups:
  - ""
  resources:
  - nodes
  verbs:
  - get
  - list
  - watch
- apiGroups:
  - ""
  resources:
  - nodes/status
  verbs:
  - patch
- apiGroups:
  - networking.k8s.io
  resources:
  - clustercidrs
  verbs:
  - list
  - watch
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  labels:
    k8s-app: flannel
  name: flannel
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: flannel
subjects:
- kind: ServiceAccount
  name: flannel
  namespace: kube-flannel
---
apiVersion: v1
kind: ServiceAccount
metadata:
  labels:
    k8s-app: flannel
  name: flannel
  namespace: kube-flannel
---
kind: ConfigMap
apiVersion: v1
metadata:
  name: kube-flannel-cfg
  namespace: kube-flannel
  labels:
    tier: node
    k8s-app: flannel
    app: flannel
data:
  cni-conf.json: |
    {
      "name": "cbr0",
      "cniVersion": "0.3.1",
      "plugins": [
        {
          "type": "flannel",
          "delegate": {
            "hairpinMode": true,
            "isDefaultGateway": true
          }
        },
        {
          "type": "portmap",
          "capabilities": {
            "portMappings": true
          }
        }
      ]
    }
  net-conf.json: |
    {
      "Network": "10.244.0.0/16",
      "Backend": {
        "Type": "vxlan"
      }
    }
---
apiVersion: apps/v1
kind: DaemonSet
metadata:
  name: kube-flannel-ds
  namespace: kube-flannel
  labels:
    tier: node
    app: flannel
    k8s-app: flannel
spec:
  selector:
    matchLabels:
      app: flannel
  template:
    metadata:
      labels:
        tier: node
        app: flannel
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values:
                - linux
      hostNetwork: true
      priorityClassName: system-node-critical
      tolerations:
      - operator: Exists
        effect: NoSchedule
      serviceAccountName: flannel
      initContainers:
      - name: install-cni-plugin
        image: docker.io/flannel/flannel-cni-plugin:v1.2.0
        command:
        - cp
        args:
        - -f
        - /flannel
        - /opt/cni/bin/flannel
        volumeMounts:
        - name: cni-plugin
          mountPath: /opt/cni/bin
      - name: install-cni
        image: docker.io/flannel/flannel:v0.22.3
        command:
        - cp
        args:
        - -f
        - /etc/kube-flannel/cni-conf.json
        - /etc/cni/net.d/10-flannel.conflist
        volumeMounts:
        - name: cni
          mountPath: /etc/cni/net.d
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
      containers:
      - name: kube-flannel
        image: docker.io/flannel/flannel:v0.22.3
        command:
        - /opt/bin/flanneld
        args:
        - --ip-masq
        - --kube-subnet-mgr
        resources:
          requests:
            cpu: "100m"
            memory: "50Mi"
        securityContext:
          privileged: false
          capabilities:
            add: ["NET_ADMIN", "NET_RAW"]
        env:
        - name: POD_NAME
          valueFrom:
            fieldRef:
              fieldPath: metadata.name
        - name: POD_NAMESPACE
          valueFrom:
            fieldRef:
              fieldPath: metadata.namespace
        - name: EVENT_QUEUE_DEPTH
          value: "5000"
        volumeMounts:
        - name: run
          mountPath: /run/flannel
        - name: flannel-cfg
          mountPath: /etc/kube-flannel/
        - name: xtables-lock
          mountPath: /run/xtables.lock
      volumes:
      - name: run
        hostPath:
          path: /run/flannel
      - name: cni-plugin
        hostPath:
          path: /opt/cni/bin
      - name: cni
        hostPath:
          path: /etc/cni/net.d
      - name: flannel-cfg
        configMap:
          name: kube-flannel-cfg
      - name: xtables-lock
        hostPath:
          path: /run/xtables.lock
          type: FileOrCreate
  • 运行以下命令启动flannel
kubectl apply -f ~/kube-flannel.yml

在这里插入图片描述

3. Node节点配置

  • 参考Master节点方案安装 kubelet, kubectl, kubeadm
apt install kubelet=1.22.15-00 kubectl=1.22.15-00 kubeadm=1.22.15-00
  • 然后运行以下命令加入node节点到master节点(该命令在master节点init成功后会出现,对应修改即可)
kubeadm join 192.168.112.64:6443 --token p4rdk9.a04jar6kywvnelpf --discovery-token-ca-cert-hash sha256:d407605d84f27512297f75c0dc13c133149086d1ce6788109144c8247675fac1
  • 运行成功后,在master节点运行以下命令可看到是否成功加入
kubectl get nodes -o wide

在这里插入图片描述

4. 核心网部署

  • 网络拓扑
    在这里插入图片描述

  • 在所有node节点上加载核心网镜像文件

tar -xzvf r2.2311.7_u22_cloud_cy.tar.gz
tar -xvf amf-r2.2311.7_u22_cloud_cy.tgz
cd amf-r2.2311.7_u22_cloud
docker load < amf_image_v2.2311.7.tar
  • Node节点上的UPF镜像需要换成 upf-v2.tar(需要支持ifconfig命令和iptables,以支持从外部自动化添加 UPF内部的NAT规则,从而实现UE访问5G网络域外数据)
iptables -t nat -A POSTROUTING -s 10.2.1.0/24 -o eth0 -j SNAT --to $ipcommand
  • 其他镜像文件加载方式类似,最终呈现以下效果:在这里插入图片描述
  • 然后在 Master节点上运行,以部署 redis 服务
docker run --name my-redis-container -d -p 10.0.0.2:6379:6379 redis
  • 其中IP(10.0.0.2)改为 Master节点IP
  • 安装NFS服务,用于容器目录挂载
  • 所有节点运行
apt install nfs-kernel-server portmap
  • 然后在 Master节点运行以下命令:
mkdir /nfs/
vim /etc/exports
/nfs *(rw,sync,no_root_squash)  注:添加到/etc/exports文件中
exportfs -arv
  • 查看NFS是否启动
showmount -e 192.168.112.64
  • 正常情况如下图所示:
    在这里插入图片描述
  • 在 /nfs 目录创建以下文件
vim amfcfg.yaml
info:
  version: 1.0.0
  description: AMF initial local configuration
configuration:
  instance: 0
  amfName: AMF
  tnlAssociationList:
  - ngapIpList:
    - 127.0.1.10
    ngapSctpPort: 38412
    weightFactor: 255
  sctpOutStreams: 2
  dscpEnabled: false
  dscpValue: 48
  relativeCapacity: 255
  backupAmfEnabled: false
  backupAmfName: AMF-BAK
  managementAddr: 127.0.1.10
  sbi:
    scheme: http
    registerIPv4: 127.0.1.10
    registerIPv6: 
    bindingIPv4: 127.0.1.10
    bindingIPv6: 
    port: 8080
    tls:
      minVersion: 1.2
      maxVersion: 1.3
    timeoutValue: 2
    shortConnection: false
  serviceNameList:
  - namf-comm
  - namf-evts
  - namf-mt
  - namf-loc
  servedGuamiList:
  - plmnId:
      mcc: 460
      mnc: 00
    regionId: 1
    setId: 1
    pointer: 1
  supportTaiList:
  - plmnId:
      mcc: 460
      mnc: 00
    tac:
    - 4388
  plmnSupportList:
  - plmnId:
      mcc: 460
      mnc: 00
    snssaiList:
    - sst: 1
      sd: 000001
  equivalentPlmn:
    enable: false
    plmnList:
    - mcc: 460
      mnc: 01
  overloadProtection:
    enable: false
    totalUeNum: 10000
    cpuPercentage: 80
  n1RateLimit:
    enable: false
    recvRateLimit: 20
    minBackoffTimerVal: 900
    maxBackoffTimerVal: 1800
  n2RecvRateLimit:
    enable: false
    recvRateLimit: 20
    overloadEnable: false
    overloadAction: 1
    overloadCheckTime: 10
  n2SendRateLimit:
    enable: false
    sendRateLimit: 20
  n8RateLimit:
    enable: false
    sendRateLimit: 20
  abnormalSignalingControl:
    enable: false
    period: 1h
    threshold: 10
  authReject2NasCause: 7
  sliceNotFound2NasCause: 7
  no5gsSubscription2NasCause: 7
  ratNotAllowed2NasCause: 27
  noAvailableSlice2NasCause: 62
  noAvailableAusf2NasCause: 15
  checkSliceInTaEnabled: false
  uePolicyEnabled: false
  smsOverNasEnabled: false
  statusReportEnabled: false
  statusReportImsiPre: 4600001
  dnnCorrectionEnabled: false
  gutiReallocationInPRUEnabled: false
  ueRadioCapabilityMatchEnabled: false
  triggerInitCtxSetupForAllNASProc: true
  supportFollowOnRequestIndication: false
  supportAllAllowedNssai: false
  abnormalSignalingControlEnabled: false
  implicitUnsubscribeEnabled: true
  supportRRCInactiveReport: false
  optimizeSignalingProcedure: false
  reAuthInServiceRequestProc: false
  skipGetSubscribedNssai: false
  supportDnnList:
  - cmnet
  sendDnnOiToSmf: false
  nrfEnabled: false
  pcfDiscoveryParameter: SUPI
  nrfUri: http://127.0.1.60:8080
  ausfUri: http://127.0.1.40:8080
  udmUri: http://127.0.1.30:8080
  smfUri: http://127.0.1.20:8080
  pcfUri: http://127.0.1.50:8080
  lmfUri: http://127.0.1.80:8080
  nefUri: http://127.0.1.70:8080
  httpManageCfg:
    ipType: ipv4
    ipv4: 127.0.1.10
    ipv6:
    port: 3030
    scheme: http
  oamConfig:
    enable: true
    ipType: ipv4
    ipv4: 127.0.1.10
    ipv6:
    port: 3030
    scheme: http
    neConfig:
      neId: 001
      rmUid: 4400HX1AMF001
      neName: AMF_001
      dn: TN
      vendorName:
      province: GD
      pvFlag: PNF
  udsfEnabled: false
  redisDb:
    enable: false
    netType: tcp
    addr: 127.0.1.90:6379
  security:
    integrityOrder:
    - algorithm: NIA2
      priority: 1
    cipheringOrder:
    - algorithm: NEA0
      priority: 1
  debugLevel: trace
  networkNameList:
  - plmnId:
      mcc: 460
      mnc: 00
    enable: false
    codingScheme: 0
    full: agt5GC
    short: agt
    timeZone: +08:00
  nssaiInclusionModeList:
  - plmnId:
      mcc: 460
      mnc: 00
    mode: A
  networkFeatureSupport5GS:
    enable: true
    length: 1
    imsVoPS: 1
    emc: 0
    emf: 0
    iwkN26: 0
    mpsi: 0
    emcN3: 0
    mcsi: 0
  drxVaule: 0
  t3502Value: 720
  configurationPriority: true
  t3512Value: 3600
  mobileReachabilityTimerValue: 3840
  implicitDeregistrationTimerValue: 3840
  non3gppDeregistrationTimerValue: 3240
  n2HoldingTimerValue: 3
  t3513:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3522:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3550:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3555:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3560:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3565:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3570:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  purgeTimer:
    enable: false
    expireTime: 24h
vim ausfcfg.yaml
configuration:
  sbi:
    scheme: http
    tls:
      verifyClientCrt: true
      verifyServerCrt: true
      minVersion: "1.2"
      maxVersion: "1.3"
    registerIPv4: 127.0.1.40
    bindingIPv4: 127.0.1.40
    port: 8080
    idleTimeout: 3
  udmUri: http://127.0.1.30:8080
  supiRanges:
  - imsi-460000100080000~imsi-460000100080099
  - imsi-460010100080000~imsi-460010100080099
  plmnSupportList:
  - mcc: "460"
    mnc: "00"
  - mcc: "460"
    mnc: "01"
  groupId: ausfGroup001
  priority: 65535
  capacity: 4096
  logLevel: info
  NFId: 101cfba8-c852-4d58-8824-3aadb32f9e31
  intraUdmIPv4: 127.0.1.30
  httpManageCfg:
    ipType: ipv4
    ipv4: 127.0.1.40
    ipv6: ""
    port: 3030
    scheme: http
  oamConfig:
    enable: true
    ipType: ipv4
    ipv4: 127.0.1.40
    ipv6: ""
    port: 3030
    scheme: http
    neConfig:
      neId: "001"
      rmUid: 4400HX1AUSF001
      neName: AUSF_001
      dn: TN
      vendorName: ""
      province: GD
      pvFlag: PNF
  nrf:
    enable: false
    nrfUri: http://127.0.1.60:8080
  telnet:
    port: 4100
  snmp:
    bindingIPv4: 127.0.1.40
    port: 4957
vim nef_conf.yaml
debuglogging:
    file: /var/log/nef.log
    level: error
httpcfg:
    ip: 127.0.1.70
    port: 8080
    scheme: http
amf:
    ip: 127.0.1.10:8080
pcf:
    ip: 127.0.1.50:8080
udm:
    ip: 127.0.1.30:8080
af:
    ip: 192.168.18.174:8080
snmpSeqNo: 13750008
localSNMP:
    ip: 127.0.1.70
    port: 4957
amfSNMP:
    ip: 127.0.1.10
    port: 4957
ausfSNMP:
    ip: 127.0.1.140
    port: 4957
udmSNMP:
    ip: 127.0.1.30
    port: 4957
smfSNMP:
    ip: 127.0.1.20
    port: 4957
pcfSNMP:
    ip: 127.0.1.50
    port: 4957
upfSNMP:
    ip: 127.0.1.100
    port: 4957
nrfSNMP:
    ip: 127.0.1.60
    port: 4957
nssfSNMP:
    ip: 192.168.8.10
    port: 4957
imsSNMP:
    ip: 192.168.8.11
    port: 4957
vim pcfcfg.yaml
configuration:
  sbi:
    scheme: http
    tls:
      verifyClientCrt: true
      verifyServerCrt: true
      minVersion: 0x0303
      maxVersion: 0x0304
    registerIPv4: 127.0.1.50
    bindingIPv4: 127.0.1.50
    port: 8080
    idleTimeout: 3
  serviceList:
    - serviceName: npcf-am-policy-control
    - serviceName: npcf-ue-policy-control
    - serviceName: npcf-smpolicycontrol
      suppFeat: 3fff
    - serviceName: npcf-policyauthorization
      suppFeat: 3
  supiRanges:
    - imsi-001010100080000~imsi-001010100080099
  plmnSupportList:
  - mcc: 001
    mnc: 01
  logLevel: info
  NFId: 88e0343e-8b79-11eb-bc48-5becbb2e9678
  systemID: 0
  httpManageCfg:
    ipType: ipv4
    ipv4: 127.0.1.50
    ipv6: ''
    port: 3030
    scheme: http
  oamConfig:
    enable: true
    ipType: ipv4
    ipv4: 127.0.1.51
    ipv6: ''
    port: 3030
    scheme: http
    neConfig:
      neId: '001'
      rmUid: 4400HX1PCF001
      neName: PCF_001
      dn: TN
      vendorName: ''
      province: GD
      pvFlag: PNF
  nrf:
    enable: false
    nrfUri: http://127.0.1.60:8080
  nef:
    nefUri: http://127.0.1.70:8080
    ueReport: false
  amfUri: http://127.0.1.10:8080
  redisDb:
    netType: tcp
    addr: 192.168.112.64:6379
  gxServer:
    enabled: false
    netType: tcp
    addr: 127.0.1.50:3868
    host: pcrf.epc.mnc000.mcc460.3gppnetwork.org
    realm: epc.mnc000.mcc460.3gppnetwork.org
  rxServer:
    enabled: false
    netType: tcp
    addr: 127.0.1.50:3867
    host: pcrf.epc.mnc000.mcc460.3gppnetwork.org
    realm: epc.mnc000.mcc460.3gppnetwork.org
  ueReport:
    enabled: false
    mqUrl: amqp://rabbitmq:SpnMq0914@192.168.111.123:5672/
    priAuthResQue: pushPriAuthResQueue
    dataUsageQue: pushDataUsageQueue
    userStatusQue: pushUserStatusQueue
    vplmnCheatQue: pushVplmnCheatQueue
  snmp:
    bindingIPv4: 127.0.1.50
    IPType: ipv4
  telnet:
    port: 4100
    userNum: 8
vim smf_conf.yaml
localconfig:
  debuglogging:
    file: /var/log/smf.log
    level: debug
  gtpucfg:
    ipv4: ""
    ipv6: ""
  httpcfg:
    scheme: http
    ip: 127.0.1.20
    ipv6: ""
    port: 8080
    serveridletime: 0
    serverauthclient: 0
    clientskipverify: false
    tlsminversion: ""
    tlsmaxversion: ""
    certexpirydays: 0
  ischarging: true
  ladndnn: ;
  ladnreleasetimer: 10
  n4interfacecfg:
    iptype: ipv4
    ipv4: 127.0.1.20
    port: 8805
  pcscfip:
    ipv4:
      default: 127.0.1.20
  smfinstanceid: b00ea745-e25a-11ec-ae22-000c29a0c97a
  snmpcfg:
    ip: 127.0.1.20
    port: 4957
    systemid: 0
  spgwcfg:
    s11ipaddr: 127.0.1.20
    localdiamcfg:
      ipaddr: 127.0.1.20
      hostname: smf
      realmname: smf.com
    gycfg:
      enable: false
      primrmtip: ""
      primrmtport: 3868
      secrmtip: ""
      secrmtport: 3868
    gxcfg:
      enable: false
      ischarge: false
      primrmtip: 127.0.1.20
      primrmtport: 3868
      secrmtip: ""
      secrmtport: 3868
  telnetip: 127.0.1.20
  uedns:
    ipv4:
      default: 114.114.114.114
      secondary: 223.5.5.5
  uemtu: 1400
  upintegrityprotrate:
    dlrate: 255
    enable: false
    ulrate: 255
localprofilecfg:
  plmnlist: 460-00;460-01
  snssailist:
  - dnnlist: example-1;example-2;example-3
    snssai:
      sd: "000000"
      sst: 1
  tailist:
  - plmnid:
      mcc: "000"
      mnc: "00"
    tac: "000000"
  tairangelist:
  - plmnId:
      mcc: "000"
      mnc: "00"
    tacRangeList:
    - end: "000111"
      pattern: ""
      start: "000000"
    - end: ""
      pattern: ^111[0-9a-fA-F]{3}$
      start: ""
nfconfig:
  amf:
    ip:
    - 127.0.1.10:8080
  nrfenable: false
  nrf: 127.0.1.60:8080
  secodarynrf: ""
  nrfenabledisc: false
  pcf:
    ip:
    - 127.0.1.50:8080
  udm:
    ip:
    - 127.0.1.30:8080
  nef: 127.0.1.70:8080
serviceareas:
- plmnId:
    mcc: ""
    mnc: ""
  tac: ""
snssaiapncfg:
- apn: ""
  snssai:
    sst: 0
    sd: ""
statusreport:
  enable: false
  traffrptperiod: 10
  totaltraffrptthreshold: 0
upfconfig:
- id: upf-1
  ip: 127.0.1.100:8805
  uednnippool:
  - dnn: test
    ueippool:
      ipv4:
      - 10.10.13.0/24
      ipv6:
      - fd0d:10:1::/50
    uestaticipv4enable: false
    uestaticipv4rangebegin: 0.0.0.0
    uestaticipv4rangeend: 0.0.0.0
    uestaticipv6enable: false
    uestaticipv6rangebegin: ""
    uestaticipv6rangeend: ""
  ueippool:
    ipv4:
    - 10.2.1.0/24
    ipv6:
    - null
  uestaticipv4enable: false
  uestaticipv4rangebegin: 10.2.1.200
  uestaticipv4rangeend: 10.2.1.240
  uestaticipv6enable: false
  uestaticipv6rangebegin: ""
  uestaticipv6rangeend: ""
upfdnaicfg:
- dnai: example
  psaupfid: upf2-id
  ulclupfid: upf3-id
upfselectcfg:
  dnntaiiupfpsaupfmap:
  - dnn: example
    iupfid: upf3-id
    psaupfid: upf4-id
    tai: "46000666666"
  dnntaisnssaiupfmap:
  - dnn: example
    snssai:
      sd: "000000"
      sst: 1
    tai: "46000666666"
    upfid:
    - upf2-id
  dnntaiupfmap:
  - dnn: example
    tai: "46000666666"
    upfid:
    - upf2-id
  dnnupfmap:
  - Dnn: cmnet
    upfid:
    - upf-1
  - Dnn: ims
    upfid:
    - upf-1
  staticueipv4upfmap:
  - endipv4: 169.0.0.64
    psaupfid: upf1-id
    startipv4: 169.0.0.12
upfulclselectcfg:
  dnntaisnssaiulclmap:
  - dnn: example
    psa1upfid: upf1-id
    psa1upfsdf: permit out ip from any to assigned
    psa2upfid: upf2-id
    psa2upfsdf: permit out ip from 192.168.0.0/16 to assigned
    snssai:
      sd: "000000"
      sst: 1
    tai: "46000666666"
    ulclupfid: upf3-id
  dnntaiulclmap:
  - dnn: example
    psa1upfid: upf1-id
    psa1upfsdf: permit out ip from any to assigned
    psa2upfid: upf2-id
    psa2upfsdf: permit out ip from 192.168.0.0/16 to assigned
    tai: "46000666666"
    ulclupfid: upf3-id
  dnnulclmap:
  - dnn: example
    psa1upfid: upf1-id
    psa1upfsdf: permit out ip from any to assigned
    psa2upfid: upf2-id
    psa2upfsdf: permit out ip from 192.168.0.0/16 to assigned
    ulclupfid: upf2-id
vim udmcfg.yaml
configuration:
  sbi:
    scheme: http
    tls:
      verifyClientCrt: true
      verifyServerCrt: true
      minVersion: '1.2'
      maxVersion: '1.3'
    registerIPv4: 127.0.1.30
    bindingIPv4: 127.0.1.30
    port: 8080
    idleTimeout: 3
  serviceNameList:
  - nudm-sdm
  - nudm-uecm
  - nudm-ueau
  groupId: "0"
  priority: 1
  capacity: 4096
  fqdn: udm.5gc.com
  supiRanges: imsi-001010100080000~imsi-001010100080099
  gpsiRanges: msisdn-69072000~msisdn-69072099
  plmnList:
  - mcc: 460
    mnc: 00
  logLevel: info
  NFId: b4c9e3d4-b7f8-4906-aade-5a84940afe29
  systemID: 0
  intraN13: false
  intraAusfIPv4: 127.0.1.40
  httpManageCfg:
    ipType: ipv4
    ipv4: 127.0.1.30
    ipv6: ''
    port: 3030
    scheme: http
  oamConfig:
    enable: true
    ipType: ipv4
    ipv4: 127.0.1.31
    ipv6: ''
    port: 3030
    scheme: http
    neConfig:
      neId: '001'
      rmUid: 4400HX1UDM001
      neName: UDM_001
      dn: TN
      vendorName: ''
      province: GD
      pvFlag: PNF
  nrf:
    enable: false
    nrfUri: http://127.0.1.60:8080
  telnet:
    port: 4100
  ssh:
    enable: false
    port: 4101
  snmp:
    bindingIPv4: 127.0.1.30
    port: 4957
  redisDb:
    netType: tcp
    addr: 192.168.112.64:6379
  s6aServer:
    enable: false
    netType: "sctp"
    addr: 127.0.1.30:3868
    host: hss.ims.mnc000.mcc460.3gppnetwork.org
    realm: ims.mnc000.mcc460.3gppnetwork.org
  cxServer:
    enable: true
    netType: "tcp"
    addr: 127.0.1.30:3868
    host: hss.ims.mnc000.mcc460.3gppnetwork.org
    realm: ims.mnc000.mcc460.3gppnetwork.org
  z1:
    enable: false
    z1Uri: "http://192.168.111.50:9999"
  authorize:
    enable: false
    hours: 1024
  sipNumberWithPlus: false
  overloadProtection:
    enable: false
    cpuPercentage: 60
    registerPerSec: 100
    retryAfterSec: 30
  trace:
    enable: false
vim upf_conf.yaml
logger:
    directory: /var/log
    level: debug
    size: 500
    number: 9

cpu:
    main: 1
    #workers: 2, 3

upf:
    pfcp:
      - addr: 127.0.1.100
    gtpu:
      - addr: 127.0.1.101
    subnet:
      - addr: 10.2.0.0/16
      #- addr: 2001:db8:cafe::1/48
        created: 0
    telnet:
      - addr: 127.0.0.1
        port: 4100
    qer_enable: 0
    local_switch_disable: 0
    flow_reflect_enable: 0
    #n9: 
      #- addr: 0.0.0.0
    #n19: 
      #- addr: 0.0.0.0
vim getIp.sh (用于upf获取网卡IP,并设置NAT规则)
#!/bin/bash

ipcommand=`ifconfig eth0 | awk 'NR==2{print $2}'`
iptables -t nat -A POSTROUTING -s 10.2.1.0/24 -o eth0 -j SNAT --to $ipcommand
  • 其中,涉及redis 的网元,包括udm和pcf,需要修改对应的IP地址到Master节点IP
  • udmcfg.yaml
  redisDb:
    netType: tcp
    addr: 192.168.112.64:6379
  • pcfcfg.yaml
  redisDb:
    netType: tcp
    addr: 192.168.112.64:6379
  • 最后,创建用于kubernetes部署的yaml文件
vim ~/cmcc_5gc.yaml
---
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cmii-core
  namespace: cmii
spec:
        #hostNetwork: true
  replicas: 1
  selector:
    matchLabels:
      app: core
  template:
    metadata:
      annotations:
        deployment.kubernetes.io/revision: '1'
      labels:
        app: core
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values:
                - linux
              - key: node-role.kubernetes.io/agent
                operator: DoesNotExist
      containers:
      - name: amf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: amf:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["amf -c /home/amfcfg.yaml"]
        ports:
          - containerPort: 38412
            hostPort: 38412
            protocol: SCTP
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: smf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: smf:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["mkdir /usr/local/etc/; mkdir /usr/local/etc/smf/; cp /home/smf_manage.yaml /usr/local/etc/smf; cp /home/smf_policy.yaml /usr/local/etc/smf;  smf -c /home/smf_conf.yaml"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: upf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: upf:v2
        command: ["/bin/sh", "-c"]
        args: ["/home/getIp.sh & upf -c /home/upf_conf.yaml "]
        ports:
          - containerPort: 2152
            hostPort: 2152
            protocol: UDP
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: ausf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: ausf:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["ausf --ausfcfg /home/ausfcfg.yaml"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: nef
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: nef:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/nef_conf.yaml /usr/local/etc/nef/ ;  GOTRACEBACK=crash /usr/local/bin/nef"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: udm
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: udm:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/udmcfg.yaml /usr/local/etc/udm/ ; GOTRACEBACK=crash /usr/local/bin/udm"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: pcf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: pcf:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/pcfcfg.yaml /usr/local/etc/pcf/ ; GOTRACEBACK=crash /usr/local/bin/pcf"]
        volumeMounts:
          - mountPath: /home
            name: nf
      volumes:
      - name: nf
        nfs:
          server: 192.168.112.64
          path: /nfs
  • 部署核心网
kubectl apply -f ~/cmcc_5gc.yaml
  • 查看部署情况,Master节点中:
kubectl get pods -o wide --all-namespaces

在这里插入图片描述

  • 在对应Node节点中查看docker 容器:
docker ps

在这里插入图片描述

参考:
Ubuntu: 搭建 NFS 服务器
KubeEdge环境搭建(支持网络插件flannel)

5. K8s核心网服务监控脚本及后台运行(Master节点)

vim sysMonitor.sh 
具备功能:
1) 监测核心网Pod运行状态,当不是Running状态的时候,新建一个核心网Pod
2) 可以读取创建Pod的IP地址,并修改amf和upf配置文件(amfcfg.yaml; upf_conf.yaml)
3) 设置了master节点的NAT规则,让Master节点作为对接基站gNB的统一入口
#!/bin/bash

isConfIp=0
isIptables=0
while true
do
	cmii=$(kubectl get pods -o wide -n cmii | grep 'cmii-core')
	#echo $cmii
	length=$(echo $cmii | wc -L)
	flag=0
	isSrv=0
	ip=""
	ip1=""
	ip2=""
	ipId=0
	isRunning=0
	node=""
	if [[ $length -gt 130 ]]
	then
		echo ""
		echo -E "More than one cmii-core service"
		srv01_stat=$(echo $cmii | awk '{print $3}')
		srv01_name=$(echo $cmii | awk '{print $1}')
		ip1=$(echo $cmii | awk '{print $6}')
		srv02_stat=$(echo $cmii | awk '{print $12}')
		srv02_name=$(echo $cmii | awk '{print $10}')
		ip2=$(echo $cmii | awk '{print $15}')
		flag=1
	else
		if [[ $length -gt 0 ]]
		then
			echo ""
			echo -E "One cmii-core service available"
			srv_name=$(echo $cmii | awk '{print $1}')
			srv_stat=$(echo $cmii | awk '{print $3}')
			ip=$(echo $cmii | awk '{print $6}')
		else
			echo "No available cmii-core, trying to create a new one ..."
                        #cp /home/cmii/smf_conf.yaml /nfs/
			kubectl apply -f /home/cmii/cmcc_5gc.yaml
			continue	
		fi
	fi
	if [[ $flag -eq 0 ]]
	then
		if [[ $srv_stat =~ "ContainerCreating" ]]
		then
			echo "Creating a new cmii-core service ... (Please Wait...)"
			isSrv=1
		elif [[ $srv_stat =~ "Running" ]]
		then
			echo $srv_name "("$ip")" "is" $srv_stat
			isSrv=1
			ipId=0
			isRunning=1
			isConfIp=1
			node=$(echo $cmii | awk '{print $9}')
		elif [[ $srv_stat =~ "CrashLoopBackOff" || $srv_stat =~ "Error" || $srv_stat =~ "Terminating" || $srv_stat =~ "NotReady" ]]
		then
			isConfIp=0
			if [[ $srv_stat =~ "CrashLoopBackOff" || $srv_stat =~ "Error" || $srv_stat =~ "NotReady" ]]
			then
				isSrv=1
				ip=$(echo $cmii | awk '{print $8}')
				ipId=0
			fi
			echo $srv_name "is abnormal"
		        #kubectl apply -f /home/cmii/cmcc_5gc.yaml	
		fi
	else
		echo $srv01_name "is" $srv01_stat
		echo $srv02_name "is" $srv02_stat
		if [[ $srv01_stat =~ "ContainerCreating" ]]
		then
			echo "Creating a new cmii-core service ... (Please Wait...)"
			isSrv=1
		elif [[ $srv01_stat =~ "Running" ]]
		then
			echo $srv01_name "is" $srv01_stat
			isSrv=1
			ipId=1
			isRunning=1
			isConfIp=1
			node=$(echo $cmii | awk '{print $9}')
		elif [[ $srv01_stat =~ "CrashLoopBackOff" || $srv01_stat =~ "Error" || $srv01_stat =~ "Terminating" || $srv01_stat =~ "NotReady" ]]
		then
			isConfIp=0
			if [[ $srv01_stat =~ "CrashLoopBackOff" || $srv01_stat =~ "Error" || $srv01_stat =~ "NotReady" ]]
			then
				isSrv=1
				ip1=$(echo $cmii | awk '{print $8}')
				ipId=1
			fi
			echo $srv01_name "is abnormal"
			#if [[ $isSrv -eq 0 ]]
			#then
				#echo $srv01_name "is abnormal, Please create a new one ..."
		        	#kubectl apply -f /home/cmii/cmcc_5gc.yaml	
			#fi
		fi

		if [[ $srv02_stat =~ "ContainerCreating" ]]
		then
			echo "Creating a new cmii-core service ... (Please Wait...)"
			isSrv=1
		elif [[ $srv02_stat =~ "Running" ]]
		then
			echo $srv02_name "is" $srv02_stat
			isSrv=1
			ipId=2
			isRunning=1
			isConfIp=1
			node=$(echo $cmii | awk '{print $9}')
		elif [[ $srv02_stat =~ "CrashLoopBackOff" || $srv02_stat =~ "Error" || $srv02_stat =~ "Terminating" || $srv02_stat =~ "NotReady" ]]
		then
			isConfIp=0
			if [[ $srv02_stat =~ "CrashLoopBackOff" || $srv02_stat =~ "Error" || $srv02_stat =~ "NotReady" ]]
			then
				isSrv=1
				ip2=$(echo $cmii | awk '{print $8}')
				ipId=2
			fi
			echo $srv01_name "is abnormal"
			#if [[ $isSrv -eq 0 ]]
			#then
			#	echo $srv02_name "is abnormal, Please create a new one ..."
		        #	kubectl apply -f /home/cmii/cmcc_5gc.yaml
			#fi	
		fi

	fi
	if [[ $isSrv -eq 0 ]]
	then
		echo "No available cmii-core, trying to create a new one ..."
                #cp /home/cmii/smf_conf.yaml /nfs/
		kubectl apply -f /home/cmii/cmcc_5gc.yaml
	fi
	if [[ isConfIp -eq 0 ]]
	then
		ipF=""
		#isConfIp=1
		if [[ $ipId -eq 0 ]]
		then
			ipF=$ip
		fi
		if [[ $ipId -eq 1 ]]
		then
			ipF=$ip1
		fi
		if [[ $ipId -eq 2 ]]
		then
			ipF=$ip2
		fi
		echo "change ip to"
		echo $ipF
		cp /home/cmii/amfcfg.yaml /home/cmii/amfcfg-tmp.yaml
		#cp /home/cmii/smf_conf.yaml /home/cmii/smf_conf-tmp.yaml
		cp /home/cmii/upf_conf.yaml /home/cmii/upf_conf-tmp.yaml
		sed -i "s/@AMFIP@/$ipF/g" /home/cmii/amfcfg-tmp.yaml
		#sed -i "s/@AMFIP@/$ipF/g" /home/cmii/smf_conf-tmp.yaml
		sed -i "s/@N3IP@/$ipF/g" /home/cmii/upf_conf-tmp.yaml
		cp /home/cmii/amfcfg-tmp.yaml /nfs/amfcfg.yaml
		#cp /home/cmii/smf_conf-tmp.yaml /nfs/smf_conf.yaml
		cp /home/cmii/upf_conf-tmp.yaml /nfs/upf_conf.yaml
	fi
	if [[ $node =~ "" ]]
	then
		#if [[ $isIptables -eq 1 ]]
		#then
		#	continue
		#fi
		echo $node
		iptablesRet=$(iptables -L -t nat --line-number | grep sctp)
		if [[ $iptablesRet == "" ]]
		then
			if [[ $node == "kubeedge-edge01" ]]
			then
				#iptables -D OUTPUT 5 -t nat
				iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.65
			fi
			if [[ $node == "kubeedge-edge02" ]]
			then
				#iptables -D OUTPUT 5 -t nat
				iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.168
			fi
		else
			ipp=$(echo $iptablesRet | awk '{print $7}')
			pa="to:"
			ippp=${ip#*${pa}}
			if [[ $node == "kubeedge-edge01" ]]
			then
				if [[ "192.168.112.65" == $ippp ]]
				then
					echo "iptables is ready ..."
				else
					iptables -D OUTPUT 5 -t nat
					iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.65
				fi
			fi
			if [[ $node == "kubeedge-edge02" ]]
			then
				if [[ "192.168.112.168" == $ippp ]]
				then
					echo "iptables is ready ..."
				else
					iptables -D OUTPUT 5 -t nat
					iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.168
				fi
			fi
		fi
		isIptables=1
	else
		echo "?"
	fi
	sleep 2
done
  • 部署后台运行,5s监控一次
  • /home/cmii/amfcfg.yaml(配置模板,使用sysMonitor.sh读取IP并修改该配置文件)
info:
  version: 1.0.0
  description: AMF initial local configuration
configuration:
  instance: 0
  amfName: AMF
  tnlAssociationList:
  - ngapIpList:
    - @AMFIP@
    ngapSctpPort: 38412
    weightFactor: 255
  sctpOutStreams: 2
  dscpEnabled: false
  dscpValue: 48
  relativeCapacity: 255
  backupAmfEnabled: false
  backupAmfName: AMF-BAK
  managementAddr: 127.0.1.10
  sbi:
    scheme: http
    registerIPv4: 127.0.1.10
    registerIPv6: 
    bindingIPv4: 127.0.1.10
    bindingIPv6: 
    port: 8080
    tls:
      minVersion: 1.2
      maxVersion: 1.3
    timeoutValue: 2
    shortConnection: false
  serviceNameList:
  - namf-comm
  - namf-evts
  - namf-mt
  - namf-loc
  servedGuamiList:
  - plmnId:
      mcc: 460
      mnc: 00
    regionId: 1
    setId: 1
    pointer: 1
  supportTaiList:
  - plmnId:
      mcc: 460
      mnc: 00
    tac:
    - 4388
  plmnSupportList:
  - plmnId:
      mcc: 460
      mnc: 00
    snssaiList:
    - sst: 1
      sd: 000001
  equivalentPlmn:
    enable: false
    plmnList:
    - mcc: 460
      mnc: 01
  overloadProtection:
    enable: false
    totalUeNum: 10000
    cpuPercentage: 80
  n1RateLimit:
    enable: false
    recvRateLimit: 20
    minBackoffTimerVal: 900
    maxBackoffTimerVal: 1800
  n2RecvRateLimit:
    enable: false
    recvRateLimit: 20
    overloadEnable: false
    overloadAction: 1
    overloadCheckTime: 10
  n2SendRateLimit:
    enable: false
    sendRateLimit: 20
  n8RateLimit:
    enable: false
    sendRateLimit: 20
  abnormalSignalingControl:
    enable: false
    period: 1h
    threshold: 10
  authReject2NasCause: 7
  sliceNotFound2NasCause: 7
  no5gsSubscription2NasCause: 7
  ratNotAllowed2NasCause: 27
  noAvailableSlice2NasCause: 62
  noAvailableAusf2NasCause: 15
  checkSliceInTaEnabled: false
  uePolicyEnabled: false
  smsOverNasEnabled: false
  statusReportEnabled: false
  statusReportImsiPre: 4600001
  dnnCorrectionEnabled: false
  gutiReallocationInPRUEnabled: false
  ueRadioCapabilityMatchEnabled: false
  triggerInitCtxSetupForAllNASProc: true
  supportFollowOnRequestIndication: false
  supportAllAllowedNssai: false
  abnormalSignalingControlEnabled: false
  implicitUnsubscribeEnabled: true
  supportRRCInactiveReport: false
  optimizeSignalingProcedure: false
  reAuthInServiceRequestProc: false
  skipGetSubscribedNssai: false
  supportDnnList:
  - cmnet
  sendDnnOiToSmf: false
  nrfEnabled: false
  pcfDiscoveryParameter: SUPI
  nrfUri: http://127.0.1.60:8080
  ausfUri: http://127.0.1.40:8080
  udmUri: http://127.0.1.30:8080
  smfUri: http://127.0.1.20:8080
  pcfUri: http://127.0.1.50:8080
  lmfUri: http://127.0.1.80:8080
  nefUri: http://127.0.1.70:8080
  httpManageCfg:
    ipType: ipv4
    ipv4: @AMFIP@
    ipv6:
    port: 3030
    scheme: http
  oamConfig:
    enable: false
    ipType: ipv4
    ipv4: 127.0.1.10
    ipv6:
    port: 3030
    scheme: http
    neConfig:
      neId: 001
      rmUid: 4400HX1AMF001
      neName: AMF_001
      dn: TN
      vendorName:
      province: GD
      pvFlag: PNF
  udsfEnabled: false
  redisDb:
    enable: false
    netType: tcp
    addr: 127.0.1.90:6379
  security:
    integrityOrder:
    - algorithm: NIA2
      priority: 1
    cipheringOrder:
    - algorithm: NEA0
      priority: 1
  debugLevel: trace
  networkNameList:
  - plmnId:
      mcc: 460
      mnc: 00
    enable: false
    codingScheme: 0
    full: agt5GC
    short: agt
    timeZone: +08:00
  nssaiInclusionModeList:
  - plmnId:
      mcc: 460
      mnc: 00
    mode: A
  networkFeatureSupport5GS:
    enable: true
    length: 1
    imsVoPS: 1
    emc: 0
    emf: 0
    iwkN26: 0
    mpsi: 0
    emcN3: 0
    mcsi: 0
  drxVaule: 0
  t3502Value: 720
  configurationPriority: true
  t3512Value: 3600
  mobileReachabilityTimerValue: 3840
  implicitDeregistrationTimerValue: 3840
  non3gppDeregistrationTimerValue: 3240
  n2HoldingTimerValue: 3
  t3513:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3522:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3550:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3555:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3560:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3565:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  t3570:
    enable: true
    expireTime: 2s
    maxRetryTimes: 2
  purgeTimer:
    enable: false
    expireTime: 24h
  • /home/cmii/upf_conf.yaml(配置模板,使用sysMonitor.sh读取IP并修改该配置文件)
logger:
    directory: /var/log
    level: debug
    size: 500
    number: 9

cpu:
    main: 1
    #workers: 2, 3

upf:
    pfcp:
      - addr: 127.0.1.100
    gtpu:
      - addr: @N3IP@
    subnet:
      - addr: 10.2.1.0/24
      #- addr: 2001:db8:cafe::1/48
        created: 0
    telnet:
      - addr: 127.0.0.1
        port: 4100
    qer_enable: 0
    local_switch_disable: 0
    flow_reflect_enable: 0
    #n9: 
      #- addr: 0.0.0.0
    #n19: 
      #- addr: 0.0.0.0
nohup /home/cmii/sysMonitor.sh > /dev/null 2>&1 &

6. 使用UERANSIM进行部署验证

  • UERANSIM源码库(编译安装可见链接,有教程)
git clone -b docker_support https://github.com/orion-belt/UERANSIM.git
  • UERANSIM gnb配置文件(config/oaicn5g-gnb.yaml)
mcc: '460'          # Mobile Country Code value
mnc: '00'           # Mobile Network Code value (2 or 3 digits)

nci: '0x000000010'  # NR Cell Identity (36-bit)
idLength: 32        # NR gNB ID length in bits [22...32]
tac: 4388          # Tracking Area Code

linkIp: 127.0.0.1   # gNB's local IP address for Radio Link Simulation (Usually same with local IP)
ngapIp: 192.168.112.64   # gNB's local IP address for N2 Interface (Usually same with local IP)
gtpIp: 192.168.112.64    # gNB's local IP address for N3 Interface (Usually same with local IP)

# List of AMF address information
amfConfigs:
  - address: 192.168.112.64
    port: 38412

# List of supported S-NSSAIs by this gNB
slices:
  - sst: 222
    sd: 123

# Indicates whether or not SCTP stream number errors should be ignored.
ignoreStreamIds: true
  • UERANSIM ue配置文件(config/oaicn5g-ue.yaml)
# IMSI number of the UE. IMSI = [MCC|MNC|MSISDN] (In total 15 digits)
supi: 'imsi-460002072701114'
# Mobile Country Code value of HPLMN
mcc: '460'
# Mobile Network Code value of HPLMN (2 or 3 digits)
mnc: '00'

# Permanent subscription key
key: '1DBD7E06EDAE124327FEED313672A125'
# Operator code (OP or OPC) of the UE
op: '27E5240241BA683F35AC172A863FBC76'
# This value specifies the OP type and it can be either 'OP' or 'OPC'
opType: 'OPC'
# Authentication Management Field (AMF) value
amf: '8000'
# IMEI number of the device. It is used if no SUPI is provided
imei: '356938035643803'
# IMEISV number of the device. It is used if no SUPI and IMEI is provided
imeiSv: '0035609204079514'

# List of gNB IP addresses for Radio Link Simulation
gnbSearchList:
  - 127.0.0.1

# UAC Access Identities Configuration
uacAic:
  mps: false
  mcs: false

# UAC Access Control Class
uacAcc:
  normalClass: 0
  class11: false
  class12: false
  class13: false
  class14: false
  class15: false

# Initial PDU sessions to be established
sessions:
  - type: 'IPv4'
    apn: 'cmnet'
    slice:
      sst: 1
      sd: 1

# Configured NSSAI for this UE by HPLMN
configured-nssai:
  - sst: 1
    sd: 1

# Default Configured NSSAI for this UE
default-nssai:
  - sst: 1
    sd: 1

# Supported integrity algorithms by this UE
integrity:
  IA1: true
  IA2: true
  IA3: true

# Supported encryption algorithms by this UE
ciphering:
  EA1: true
  EA2: true
  EA3: true

# Integrity protection maximum data rate for user plane
integrityMaxRate:
  uplink: 'full'
  downlink: 'full'
  • 测试结果
    在这里插入图片描述
    在这里插入图片描述
    在这里插入图片描述
    在这里插入图片描述
    在这里插入图片描述
    在这里插入图片描述
    的

7. 使用商用基站和终端进行测试

  • 基站IP:192.168.112.251
  • UPF IP:10.244.6.68
  • UPF所在宿主机IP:192.168.112.65
  • 方案一:基站和宿主机通信通过三层交换机网管的模式:则分别需要在基站上和交换机上设置静态路由:
基站上:

在这里插入图片描述

交换机上:

在这里插入图片描述
注:有的商用基站封装的GTPU数据包报头校验值为0,该数据包可能在交换机网关处被丢弃,这种情况,需要考虑方案二的模式。

  • 方案二:基站和宿主机通信不通过网关,直接通过交换机二层路由模式,那么只需要在基站上添加上述静态路由

8. IMS在k8s云环境中部署与测试

  • 在各个node节点安装IMS镜像文件(直接执行 ims_image_load.sh):
    在这里插入图片描述
  • 修改k8s部署的yaml文件:cmcc_5gc.yaml,修改为如下:
vim cmcc_5gc.yaml
---
#apiVersion: extensions/v1beta1
apiVersion: apps/v1
kind: Deployment
metadata:
  name: cmii-core
  namespace: cmii
spec:
        #hostNetwork: true
  replicas: 1
  selector:
    matchLabels:
      app: core
  template:
    metadata:
      annotations:
        deployment.kubernetes.io/revision: '1'
      labels:
        app: core
    spec:
      affinity:
        nodeAffinity:
          requiredDuringSchedulingIgnoredDuringExecution:
            nodeSelectorTerms:
            - matchExpressions:
              - key: kubernetes.io/os
                operator: In
                values:
                - linux
              - key: node-role.kubernetes.io/agent
                operator: DoesNotExist
      containers:
      - name: rtproxy
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: rtproxy:1.2.9 
        command: ["/bin/sh", "-c"]
        args: ["rtproxy --interface=127.0.1.120 --listen-ng=12221 --listen-cli=9900 --timeout=60 --silent-timeout=3600 --port-min=30000 --port-max=50000 --pidfile=/var/run/ngcp-rtpengine-daemon.pid --no-redis-required --foreground"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: amf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: amf:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["amf -c /home/amfcfg.yaml"]
        ports:
          - containerPort: 38412
            hostPort: 38412
            protocol: SCTP
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: smf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: smf:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["smf -c /home/smf_conf.yaml"]
        #args: ["mkdir usr/local/etc/; mkdir usr/local/etc/smf/; cp /home/smf_manage.yaml /usr/local/etc/smf; cp /home/smf_policy.yaml /usr/local/etc/smf;  smf -c /home/smf_conf.yaml"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: upf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: upf:v2
        command: ["/bin/sh", "-c"]
        args: ["/home/getIp.sh & upf -c /home/upf_conf.yaml "]
        ports:
          - containerPort: 2152
            hostPort: 2152
            protocol: UDP
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: ausf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: ausf:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["ausf --ausfcfg /home/ausfcfg.yaml"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: nef
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: nef:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/nef_conf.yaml /usr/local/etc/nef/ ;  GOTRACEBACK=crash /usr/local/bin/nef"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: udm
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: udm:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/udmcfg.yaml /usr/local/etc/udm/ ; GOTRACEBACK=crash /usr/local/bin/udm"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: pcf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: pcf:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/pcfcfg.yaml /usr/local/etc/pcf/ ; GOTRACEBACK=crash /usr/local/bin/pcf"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: pcscf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: ims:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/hosts /etc/hosts; cp /home/ims/protocols /etc/protocols ; cp -r /home/ims /usr/local/etc/; ims -f /usr/local/etc/ims/pcscf/main.cfg -DD"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: icscf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: ims:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/hosts /etc/hosts; cp /home/ims/protocols /etc/protocols ; cp -r /home/ims /usr/local/etc/; ims -f /usr/local/etc/ims/icscf/main.cfg -DD"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: scscf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: ims:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/hosts /etc/hosts; cp /home/ims/protocols /etc/protocols ; cp -r /home/ims /usr/local/etc/; ims -f /usr/local/etc/ims/scscf/main.cfg -DD"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: bgcf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: ims:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/hosts /etc/hosts; cp /home/ims/protocols /etc/protocols ; cp -r /home/ims /usr/local/etc/; ims -f /usr/local/etc/ims/bgcf/main.cfg -DD"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: mmtel
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: ims:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp /home/hosts /etc/hosts; cp /home/ims/protocols /etc/protocols ; cp -r /home/ims /usr/local/etc/; ims -f /usr/local/etc/ims/mmtel/main.cfg -DD"]
        volumeMounts:
          - mountPath: /home
            name: nf
      - name: mrf
        securityContext:
          privileged: true
          capabilities:
            add: ["NET_ADMIN"]
        image: mrf:2.2311.7
        command: ["/bin/sh", "-c"]
        args: ["cp -r /home/ims /usr/local/etc/; mrf /usr/local/etc/ims/mrf"]
        volumeMounts:
          - mountPath: /home
            name: nf
      volumes:
      - name: nf
        nfs:
          server: 192.168.112.64
          path: /nfs
  • 修改sysMonitor.sh文件,以支持IMS配置修改
vim sysMonitor.sh
#!/bin/bash

isConfIp=0
isIptables=0
while true
do
	cmii=$(kubectl get pods -o wide -n cmii | grep 'cmii-core')
	#echo $cmii
	length=$(echo $cmii | wc -L)
	flag=0
	isSrv=0
	ip=""
	ip1=""
	ip2=""
	ipId=0
	isRunning=0
	node=""
	if [[ $length -gt 130 ]]
	then
		echo ""
		echo -E "More than one cmii-core service"
		srv01_stat=$(echo $cmii | awk '{print $3}')
		srv01_name=$(echo $cmii | awk '{print $1}')
		ip1=$(echo $cmii | awk '{print $6}')
		srv02_stat=$(echo $cmii | awk '{print $12}')
		srv02_name=$(echo $cmii | awk '{print $10}')
		ip2=$(echo $cmii | awk '{print $15}')
		flag=1
	else
		if [[ $length -gt 0 ]]
		then
			echo ""
			echo -E "One cmii-core service available"
			srv_name=$(echo $cmii | awk '{print $1}')
			srv_stat=$(echo $cmii | awk '{print $3}')
			ip=$(echo $cmii | awk '{print $6}')
		else
			echo "No available cmii-core, trying to create a new one ..."
                        #cp /home/cmii/smf_conf.yaml /nfs/
			kubectl apply -f /home/cmii/cmcc_5gc.yaml
			continue	
		fi
	fi
	if [[ $flag -eq 0 ]]
	then
		if [[ $srv_stat =~ "ContainerCreating" ]]
		then
			echo "Creating a new cmii-core service ... (Please Wait...)"
			isSrv=1
		elif [[ $srv_stat =~ "Running" ]]
		then
			echo $srv_name "("$ip")" "is" $srv_stat
			isSrv=1
			ipId=0
			isRunning=1
			isConfIp=1
			node=$(echo $cmii | awk '{print $9}')
		elif [[ $srv_stat =~ "CrashLoopBackOff" || $srv_stat =~ "Error" || $srv_stat =~ "Terminating" || $srv_stat =~ "NotReady" ]]
		then
			isConfIp=0
			if [[ $srv_stat =~ "CrashLoopBackOff" || $srv_stat =~ "Error" || $srv_stat =~ "NotReady" ]]
			then
				isSrv=1
				ip=$(echo $cmii | awk '{print $8}')
				ipId=0
			fi
			echo $srv_name "is abnormal"
		        #kubectl apply -f /home/cmii/cmcc_5gc.yaml	
		fi
	else
		echo $srv01_name "is" $srv01_stat
		echo $srv02_name "is" $srv02_stat
		if [[ $srv01_stat =~ "ContainerCreating" ]]
		then
			echo "Creating a new cmii-core service ... (Please Wait...)"
			isSrv=1
		elif [[ $srv01_stat =~ "Running" ]]
		then
			echo $srv01_name "is" $srv01_stat
			isSrv=1
			ipId=1
			isRunning=1
			isConfIp=1
			node=$(echo $cmii | awk '{print $9}')
		elif [[ $srv01_stat =~ "CrashLoopBackOff" || $srv01_stat =~ "Error" || $srv01_stat =~ "Terminating" || $srv01_stat =~ "NotReady" ]]
		then
			isConfIp=0
			if [[ $srv01_stat =~ "CrashLoopBackOff" || $srv01_stat =~ "Error" || $srv01_stat =~ "NotReady" ]]
			then
				isSrv=1
				ip1=$(echo $cmii | awk '{print $8}')
				ipId=1
			fi
			echo $srv01_name "is abnormal"
			#if [[ $isSrv -eq 0 ]]
			#then
				#echo $srv01_name "is abnormal, Please create a new one ..."
		        	#kubectl apply -f /home/cmii/cmcc_5gc.yaml	
			#fi
		fi

		if [[ $srv02_stat =~ "ContainerCreating" ]]
		then
			echo "Creating a new cmii-core service ... (Please Wait...)"
			isSrv=1
		elif [[ $srv02_stat =~ "Running" ]]
		then
			echo $srv02_name "is" $srv02_stat
			isSrv=1
			ipId=2
			isRunning=1
			isConfIp=1
			node=$(echo $cmii | awk '{print $9}')
		elif [[ $srv02_stat =~ "CrashLoopBackOff" || $srv02_stat =~ "Error" || $srv02_stat =~ "Terminating" || $srv02_stat =~ "NotReady" ]]
		then
			isConfIp=0
			if [[ $srv02_stat =~ "CrashLoopBackOff" || $srv02_stat =~ "Error" || $srv02_stat =~ "NotReady" ]]
			then
				isSrv=1
				ip2=$(echo $cmii | awk '{print $8}')
				ipId=2
			fi
			echo $srv01_name "is abnormal"
			#if [[ $isSrv -eq 0 ]]
			#then
			#	echo $srv02_name "is abnormal, Please create a new one ..."
		        #	kubectl apply -f /home/cmii/cmcc_5gc.yaml
			#fi	
		fi

	fi
	if [[ $isSrv -eq 0 ]]
	then
		echo "No available cmii-core, trying to create a new one ..."
                #cp /home/cmii/smf_conf.yaml /nfs/
		kubectl apply -f /home/cmii/cmcc_5gc.yaml
	fi
	if [[ isConfIp -eq 0 ]]
	then
		ipF=""
		#isConfIp=1
		if [[ $ipId -eq 0 ]]
		then
			ipF=$ip
		fi
		if [[ $ipId -eq 1 ]]
		then
			ipF=$ip1
		fi
		if [[ $ipId -eq 2 ]]
		then
			ipF=$ip2
		fi
		echo "change ip to"
		echo $ipF
		cp /home/cmii/amfcfg.yaml /home/cmii/amfcfg-tmp.yaml
		#cp /home/cmii/smf_conf.yaml /home/cmii/smf_conf-tmp.yaml
		cp /home/cmii/upf_conf.yaml /home/cmii/upf_conf-tmp.yaml
		sed -i "s/@AMFIP@/$ipF/g" /home/cmii/amfcfg-tmp.yaml
		#sed -i "s/@AMFIP@/$ipF/g" /home/cmii/smf_conf-tmp.yaml
		sed -i "s/@N3IP@/$ipF/g" /home/cmii/upf_conf-tmp.yaml
		cp /home/cmii/amfcfg-tmp.yaml /nfs/amfcfg.yaml
		#cp /home/cmii/smf_conf-tmp.yaml /nfs/smf_conf.yaml
		cp /home/cmii/upf_conf-tmp.yaml /nfs/upf_conf.yaml
		/nfs/ims/tools/modipplmn.sh $ipF 460 00
	fi
	if [[ $node =~ "" ]]
	then
		#if [[ $isIptables -eq 1 ]]
		#then
		#	continue
		#fi
		echo $node
		iptablesRet=$(iptables -L -t nat --line-number | grep sctp)
		if [[ $iptablesRet == "" ]]
		then
			if [[ $node == "kubeedge-edge01" ]]
			then
				#iptables -D OUTPUT 5 -t nat
				iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.65
			fi
			if [[ $node == "kubeedge-edge02" ]]
			then
				#iptables -D OUTPUT 5 -t nat
				iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.168
			fi
		else
			ipp=$(echo $iptablesRet | awk '{print $7}')
			pa="to:"
			ippp=${ip#*${pa}}
			if [[ $node == "kubeedge-edge01" ]]
			then
				if [[ "192.168.112.65" == $ippp ]]
				then
					echo "iptables is ready ..."
				else
					iptables -D OUTPUT 5 -t nat
					iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.65
				fi
			fi
			if [[ $node == "kubeedge-edge02" ]]
			then
				if [[ "192.168.112.168" == $ippp ]]
				then
					echo "iptables is ready ..."
				else
					iptables -D OUTPUT 5 -t nat
					iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.168
				fi
			fi
		fi
		isIptables=1
	else
		echo "?"
	fi
	sleep 2
done
  • 将IMS配置文件复制到/nfs目录下,以加载到k8s docker化的IMS功能中:
cp -r /home/cmii/5gc/ims-r2.2311.7_u22_cloud_cy/conf/ims/ /nfs
  • 修改IMS自带脚本:/nfs/ims/tools/modipplmn.sh
#!/bin/bash
# ./modipplmn.sh 192.168.1.123 001 01

NEWIP=$1
MCC=$2
MNC=$3

MNCLEN=`expr length $MNC`

#echo $MNCLEN

MNCSTR=`sed -n '/IMS_DOMAIN/p' /nfs/ims/vars.cfg | awk '{print $3}'| awk -F '.' '{print $2}'`
MCCSTR=`sed -n '/IMS_DOMAIN/p' /nfs/ims/vars.cfg | awk '{print $3}'| awk -F '.' '{print $3}'`

#echo $MNCSTR $MCCSTR

if [ $MNCLEN = 2 ];then
        DSTPLMN=mnc0$MNC.mcc$MCC
elif [ $MNCLEN = 3 ];then
        DSTPLMN=mnc$MNC.mcc$MCC
else
	echo "Usage: $0 ip_address mcc mnc"
        exit
fi

#echo $DSTPLMN


OLDIP=`sed -n '/IMS_IP /p' /nfs/ims/vars.cfg | awk -F '"' '{print $2}'`

echo $OLDIP
##mcc=460
##mnc=00
sed -i "s/mcc=${MCCSTR:3}/mcc=$MCC/g" /nfs/ims/vars.cfg
sed -i "s/mnc=${MNCSTR:4}/mnc=$MNC/g" /nfs/ims/vars.cfg

sed -i "s/$OLDIP/$NEWIP/g" /nfs/ims/vars.cfg
sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/ims/vars.cfg

#sed -i "s/$OLDIP/$NEWIP/g" /etc/bind/agt-ims.dnszone
#sed -i "s/$OLDIP/$NEWIP/g" /etc/default/ngcp-rtpengine-daemon
#sed -i "s/$OLDIP/$NEWIP/g" /usr/local/etc/rtproxy/rtproxy.conf
sed -i "s/$OLDIP/$NEWIP/g" /nfs/ims/scscf/dispatcher.list
sed -i "s/$OLDIP/$NEWIP/g" /nfs/ims/mmtel/dispatcher.list
#sed -i "s/$OLDIP/$NEWIP/g" /nfs/ims/n5/conf/n5_profile
sed -i "s/$OLDIP/$NEWIP/g" /nfs/ims/mrf/mrf_param.conf

#sed -i "s/$OLDIP/$NEWIP/g" /etc/hosts

#sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /etc/bind/agt-ims.dnszone
#sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /etc/bind/named.conf

sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/ims/scscf/domain.list

sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/ims/pcscf/pcscf.xml
sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/ims/icscf/icscf.xml
sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/ims/scscf/scscf.xml
sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/ims/mrf/mrf_param.conf

#sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/ims/n5/conf/n5_profile
#sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/ims/n5/conf/fd_server.conf

#echo $MNCSTR 
#echo $MCCSTR 
#echo $DSTPLM
cp /home/cmii/hosts-tmp /nfs/hosts
#sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/hosts
sed -i "s/@NEWIP@/$NEWIP/g" /nfs/hosts

sed -i "s/$MNCSTR.$MCCSTR/$DSTPLMN/g" /nfs/iwf/iwf_conf.yaml
sed -i "s/$OLDIP/$NEWIP/g" /nfs/iwf/iwf_conf.yaml

echo "config successfully"

#systemd-resolve --flush-caches

#service bind9 restart
#service n5 restart
  • 修改IMS容器内/etc/hosts,以支持IMS域名通信方法
vim /home/cmii/hosts-tmp
127.0.0.1       localhost
127.0.1.1       kubeedge-master

127.0.1.30 hss.ims.mnc000.mcc460.3gppnetwork.org hss
@NEWIP@ ims.mnc000.mcc460.3gppnetwork.org       ims
@NEWIP@ pcscf.ims.mnc000.mcc460.3gppnetwork.org pcscf
@NEWIP@ icscf.ims.mnc000.mcc460.3gppnetwork.org icscf
@NEWIP@ scscf.ims.mnc000.mcc460.3gppnetwork.org scscf
@NEWIP@ mmtel.ims.mnc000.mcc460.3gppnetwork.org mmtel

# The following lines are desirable for IPv6 capable hosts
::1     ip6-localhost ip6-loopback
fe00::0 ip6-localnet
ff00::0 ip6-mcastprefix
ff02::1 ip6-allnodes
ff02::2 ip6-allrouters
  • 测试结果:手机上有 HD标识
    在这里插入图片描述
  • 测试结果,抓包:SIP协议
    在这里插入图片描述

9. IMS自动化配置

  • 网络拓扑
    在这里插入图片描述
  • 手机卡信息在核心网中配置的方式(添加一次就行,应该写到redis中了)
1. docker exec -it [upf_container_id] /bin/sh
2. apt-get install telnet
3. telnet 127.0.1.30 4100
4. 输入用户名admin,密码admin
5. add authdat:imsi=460003000100051,ki=00112233445566778899aabbccddeeff,amf=8000,algo=0,opc=11111111111111111111111111111111
6. add udmuser:imsi=460003000100051,msisdn=12307550051,sm_data=1-000001&cmnet&ims,eps_flag=1,kdc_flag=0
7. 查看已有SIM卡信息:dsp udmuser:imsi=46000xxxxx
  • 需要修改 smf_conf.yaml、as.yaml、pcscf.xml、pcscf.cfg、hosts文件,分别如下:
vim smf_conf.yaml

在这里插入图片描述

vim as.yaml

在这里插入图片描述

vim pcscf.xml

在这里插入图片描述

vim pcscf.cfg

在这里插入图片描述

vim hosts

在这里插入图片描述

  • 修改sysMonitor.sh脚本将上述文件中@NEWIP@或者@PCSCF@等改成确定IP
vim sysMonitor.sh
#!/bin/bash

isConfIp=0
isIptables=0
isRestarted=0
while true
do
	cmii=$(kubectl get pods -o wide -n cmii | grep 'cmii-core')
	#echo $cmii
	length=$(echo $cmii | wc -L)
	flag=0
	isSrv=0
	ip=""
	ip1=""
	ip2=""
	ipId=0
	isRunning=0
	node=""
	if [[ $length -gt 130 ]]
	then
		echo ""
		echo -E "More than one cmii-core service"
		srv01_stat=$(echo $cmii | awk '{print $3}')
		srv01_name=$(echo $cmii | awk '{print $1}')
		ip1=$(echo $cmii | awk '{print $6}')
		srv02_stat=$(echo $cmii | awk '{print $12}')
		srv02_name=$(echo $cmii | awk '{print $10}')
		ip2=$(echo $cmii | awk '{print $15}')
		flag=1
	else
		if [[ $length -gt 0 ]]
		then
			echo ""
			echo -E "One cmii-core service available"
			srv_name=$(echo $cmii | awk '{print $1}')
			srv_stat=$(echo $cmii | awk '{print $3}')
			ip=$(echo $cmii | awk '{print $6}')
		else
			echo "No available cmii-core, trying to create a new one ..."
                        #cp /home/cmii/smf_conf.yaml /nfs/
			kubectl apply -f /home/cmii/cmcc_5gc.yaml
			continue	
		fi
	fi
	if [[ $flag -eq 0 ]]
	then
		if [[ $srv_stat =~ "ContainerCreating" ]]
		then
			echo "Creating a new cmii-core service ... (Please Wait...)"
			isSrv=1
		elif [[ $srv_stat =~ "Running" ]]
		then
			echo $srv_name "("$ip")" "is" $srv_stat
			isSrv=1
			ipId=0
			isRunning=1
			isConfIp=1
			node=$(echo $cmii | awk '{print $9}')
		elif [[ $srv_stat =~ "CrashLoopBackOff" || $srv_stat =~ "Error" || $srv_stat =~ "Terminating" || $srv_stat =~ "NotReady" ]]
		then
			isConfIp=0
			isRunning=0
			if [[ $srv_stat =~ "CrashLoopBackOff" || $srv_stat =~ "Error" || $srv_stat =~ "NotReady" ]]
			then
				isSrv=1
				ip=$(echo $cmii | awk '{print $8}')
				ipId=0
			fi
			echo $srv_name "is abnormal"
		        #kubectl apply -f /home/cmii/cmcc_5gc.yaml	
		fi
	else
		echo $srv01_name "is" $srv01_stat
		echo $srv02_name "is" $srv02_stat
		if [[ $srv01_stat =~ "ContainerCreating" ]]
		then
			echo "Creating a new cmii-core service ... (Please Wait...)"
			isSrv=1
		elif [[ $srv01_stat =~ "Running" ]]
		then
			echo $srv01_name "is" $srv01_stat
			isSrv=1
			ipId=1
			isRunning=1
			isConfIp=1
			node=$(echo $cmii | awk '{print $9}')
		elif [[ $srv01_stat =~ "CrashLoopBackOff" || $srv01_stat =~ "Error" || $srv01_stat =~ "Terminating" || $srv01_stat =~ "NotReady" ]]
		then
			isConfIp=0
			isRunning=0
			if [[ $srv01_stat =~ "CrashLoopBackOff" || $srv01_stat =~ "Error" || $srv01_stat =~ "NotReady" ]]
			then
				isSrv=1
				ip1=$(echo $cmii | awk '{print $8}')
				ipId=1
			fi
			echo $srv01_name "is abnormal"
			#if [[ $isSrv -eq 0 ]]
			#then
				#echo $srv01_name "is abnormal, Please create a new one ..."
		        	#kubectl apply -f /home/cmii/cmcc_5gc.yaml	
			#fi
		fi

		if [[ $srv02_stat =~ "ContainerCreating" ]]
		then
			echo "Creating a new cmii-core service ... (Please Wait...)"
			isSrv=1
		elif [[ $srv02_stat =~ "Running" ]]
		then
			echo $srv02_name "is" $srv02_stat
			isSrv=1
			ipId=2
			isRunning=1
			isConfIp=1
			node=$(echo $cmii | awk '{print $9}')
		elif [[ $srv02_stat =~ "CrashLoopBackOff" || $srv02_stat =~ "Error" || $srv02_stat =~ "Terminating" || $srv02_stat =~ "NotReady" ]]
		then
			isConfIp=0
			isRunning=0
			if [[ $srv02_stat =~ "CrashLoopBackOff" || $srv02_stat =~ "Error" || $srv02_stat =~ "NotReady" ]]
			then
				isSrv=1
				ip2=$(echo $cmii | awk '{print $8}')
				ipId=2
			fi
			echo $srv01_name "is abnormal"
			#if [[ $isSrv -eq 0 ]]
			#then
			#	echo $srv02_name "is abnormal, Please create a new one ..."
		        #	kubectl apply -f /home/cmii/cmcc_5gc.yaml
			#fi	
		fi

	fi
	if [[ $isSrv -eq 0 ]]
	then
		echo "No available cmii-core, trying to create a new one ..."
                #cp /home/cmii/smf_conf.yaml /nfs/
		kubectl apply -f /home/cmii/cmcc_5gc.yaml
	fi
	if [[ isConfIp -eq 0 ]]
	then
		ipF=""
		#isConfIp=1
		if [[ $ipId -eq 0 ]]
		then
			ipF=$ip
		fi
		if [[ $ipId -eq 1 ]]
		then
			ipF=$ip1
		fi
		if [[ $ipId -eq 2 ]]
		then
			ipF=$ip2
		fi
		echo "change ip to"
		echo $ipF
		ipS="127.0.1.121"
		cp /home/cmii/amfcfg.yaml /home/cmii/amfcfg-tmp.yaml
		cp /home/cmii/smf_conf.yaml /home/cmii/smf_conf-tmp.yaml
		cp /home/cmii/upf_conf.yaml /home/cmii/upf_conf-tmp.yaml
		cp /home/cmii/pcscf.xml /home/cmii/pcscf-tmp.xml
		cp /home/cmii/pcscf.cfg /home/cmii/pcscf-tmp.cfg
		cp /home/cmii/as.yaml /home/cmii/as-tmp.yaml
		cp /home/cmii/hosts /home/cmii/hosts-tmp
		sed -i "s/@AMFIP@/$ipF/g" /home/cmii/amfcfg-tmp.yaml
		sed -i "s/@NEWIP@/$ipF/g" /home/cmii/smf_conf-tmp.yaml
		sed -i "s/@NEWIP@/$ipF/g" /home/cmii/pcscf-tmp.xml
		sed -i "s/@NEWIP@/$ipF/g" /home/cmii/pcscf-tmp.cfg
		sed -i "s/@NEWIP@/$ipS/g" /home/cmii/as-tmp.yaml
		sed -i "s/@N3IP@/$ipF/g" /home/cmii/upf_conf-tmp.yaml
		sed -i "s/@PCSCF@/$ipF/g" /home/cmii/hosts-tmp
		cp /home/cmii/amfcfg-tmp.yaml /nfs/amfcfg.yaml
		cp /home/cmii/smf_conf-tmp.yaml /nfs/smf_conf.yaml
		cp /home/cmii/upf_conf-tmp.yaml /nfs/upf_conf.yaml
		cp /home/cmii/pcscf-tmp.xml /nfs/ims/pcscf/pcscf.xml
		cp /home/cmii/pcscf-tmp.cfg /nfs/ims/pcscf/pcscf.cfg
		cp /home/cmii/as-tmp.yaml /nfs/udm/as.yaml
		#/nfs/ims/tools/modipplmn.sh 127.0.1.333 460 00
		/nfs/ims/tools/modipplmn.sh $ipS 460 00
	fi
	if [[ $node =~ "" ]]
	then
		#if [[ $isIptables -eq 1 ]]
		#then
		#	continue
		#fi
		echo $node
		iptablesRet=$(iptables -L -t nat --line-number | grep sctp)
		if [[ $iptablesRet == "" ]]
		then
			if [[ $node == "kubeedge-edge01" ]]
			then
				#iptables -D OUTPUT 5 -t nat
				iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.65
			fi
			if [[ $node == "kubeedge-edge02" ]]
			then
				#iptables -D OUTPUT 5 -t nat
				iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.168
			fi
		else
			ipp=$(echo $iptablesRet | awk '{print $7}')
			pa="to:"
			ippp=${ip#*${pa}}
			if [[ $node == "kubeedge-edge01" ]]
			then
				if [[ "192.168.112.65" == $ippp ]]
				then
					echo "iptables is ready ..."
				else
					iptables -D OUTPUT 5 -t nat
					iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.65
				fi
			fi
			if [[ $node == "kubeedge-edge02" ]]
			then
				if [[ "192.168.112.168" == $ippp ]]
				then
					echo "iptables is ready ..."
				else
					iptables -D OUTPUT 5 -t nat
					iptables -t nat -A OUTPUT -d 192.168.112.64 -p sctp -j DNAT --to-destination 192.168.112.168
				fi
			fi
		fi
		isIptables=1
	else
		echo "?"
	fi
	if [[ $isRunning -eq 1 ]]
	then
		if [[ $isRestarted -eq 0 ]]
		then
			echo "Restart containers ..."
			if [[ $node == "kubeedge-edge01" ]]
			then
				sshpass -p 'cmii' ssh -t cmii@192.168.112.65 'sudo /home/cmii/restartC.sh'
				isRestarted=1
			fi
			if [[ $node == "kubeedge-edge02" ]]
			then
				sshpass -p 'cmii' ssh -t cmii@192.168.112.168 'sudo /home/cmii/restartC.sh'
				isRestarted=1
			fi
		fi
	fi
	sleep 1
done
  • 其中运行POD后,需要将其中smf容器重启,以让修改的配置生效
  • 在对应node节点主机上部署脚本 restartC.sh
vim restartC.sh
#!/bin/bash

# 定义要匹配的容器名称
container_names=("smf")

# 执行docker ps命令并将结果保存到变量中
output=$(docker ps --format "{{.Names}}")

# 遍历容器名称列表
for name in "${container_names[@]}"; do
     # 使用grep命令从输出中匹配包含指定名称的容器,并将结果保存到对应变量中
            matching_containers=$(echo "${output}" | grep -E "${name}")
            # 如果找到匹配的容器,则逐一重启它们
                if [ -n "${matching_containers}" ]; then
                      IFS=$'\n' read -rd '' -a containers_to_restart <<<"${matching_containers}"
                      for container in "${containers_to_restart[@]}"; do
                          docker restart "${container}"
                           echo "Container ${container} has been restarted."
                      done
                 else
                           echo "No container containing '${name}' found."
                  fi
 done
  • 值得注意的是,mmtel.cfg文件中连接redis的地址得修改
    在这里插入图片描述
  • 最终效果:
    在这里插入图片描述
  • 注:有个地方,smf中配置pcscf的IP地址不能为127.x.x.x,因为该IP是UE最终进行SIP注册的目的地址,如果配置成127.x.x.x,该数据包无法从UE空口传输至网络侧,呈现的效果就是从网络侧抓包看根本就没有SIP协议数据包。

附录:终端VoNR语言通话协议数据抓包图

在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述
在这里插入图片描述

Logo

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐