k8s_day07_03

k8s_day07_03人类用户授权名称空间级别授权示例：注意因为配置简单，通常用命令式命令创建1、创建角色[root@node01 chapter9]# cat pods-reader-rbac.yamlkind: RoleapiVersion: rbac.authorization.k8s.io/v1metadata:namespace: defaultname: pods-readerrul

山贼王

866人浏览 · 2022-02-10 22:06:52

山贼王 · 2022-02-10 22:06:52 发布

k8s_day07_03

人类用户授权

名称空间级别授权示例：

注意因为配置简单，通常用命令式命令创建

1、创建角色

[root@node01 chapter9]# cat pods-reader-rbac.yaml 
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  namespace: default
  name: pods-reader
rules:
- apiGroups: [""]
  resources: ["pods", "services", "pods/log"]
  verbs: ["get", "list", "watch"]

apiGroups 为空表示是核心群组

resources 、verbs 内元素引号要加都加，不加都不加

验证结果

[root@node01 chapter9]# kubectl get roles
NAME          CREATED AT
pods-reader   2021-12-25T03:59:35Z

[root@node01 chapter9]# kubectl   describe roles/pods-reader  
Name:         pods-reader
Labels:       <none>
Annotations:  <none>
PolicyRule:
  Resources  Non-Resource URLs  Resource Names  Verbs
  ---------  -----------------  --------------  -----
  pods/log   []                 []              [get list watch]
  pods       []                 []              [get list watch]
  services   []                 []              [get list watch]
[root@node01 chapter9]#

2、并且给用户赋予角色

[root@node01 chapter9]# cat  magedu-pods-reader.yaml
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: magedu-pods-reader
  namespace: default
subjects:
- kind: User
  name: magedu
  apiGroup: rbac.authorization.k8s.io
roleRef:
  kind: Role
  name: pods-reader
  apiGroup: rbac.authorization.k8s.io
[root@node01 chapter9]#

验证结果

[root@node01 chapter9]# kubectl  get RoleBindings/magedu-pods-reader
NAME                 ROLE               AGE
magedu-pods-reader   Role/pods-reader   8m2s
[root@node01 chapter9]# 


[root@node01 chapter9]# kubectl describe  RoleBinding/magedu-pods-reader 
Name:         magedu-pods-reader
Labels:       <none>
Annotations:  <none>
Role:
  Kind:  Role
  Name:  pods-reader
Subjects:
  Kind  Name    Namespace
  ----  ----    ---------
  User  magedu  
[root@node01 chapter9]#

3、验证用户是否可以访问

[root@master01 usercerts]# kubectl get  pods/mypod   --context=magedu@kubernetes 
NAME    READY   STATUS    RESTARTS   AGE
mypod   1/1     Running   0          19h
[root@master01 usercerts]# kubectl get  nodes   --context=magedu@kubernetes 
Error from server (Forbidden): nodes is forbidden: User "magedu" cannot list resource "nodes" in API group "" at the cluster scope
[root@master01 usercerts]#

多个角色规则的写法

kubectl   get  clusterrole  admin -o yaml

rules:

- apiGroups:
  - autoscaling
  resources:
  - horizontalpodautoscalers
  verbs:
  - create
  - delete
  - deletecollection
  - patch
  - update
- apiGroups:
  - batch
  resources:
  - cronjobs
  - jobs

集群级别授权示例1：

2个默认集群角色

admin

除了集群级别资源管理权限所有权限的拥有者，（集群级别资源有 pv 、node 等等。 ns 除外，因为ns作为基础组件所有用户都有权限）
cluster-admin
all 权限

eg1 admin

避免干扰

[root@node01 chapter9]# kubectl  delete  RoleBindings --all
rolebinding.rbac.authorization.k8s.io "magedu-pods-reader" deleted

[root@node01 chapter9]# kubectl  delete  roles --all
role.rbac.authorization.k8s.io "pods-reader" deleted

授权magedu 用户拥有集群默认角色

[root@node01 chapter9]# kubectl  get clusterrole/admin 
NAME    CREATED AT
admin   2021-12-13T03:52:29Z

[root@node01 chapter9]# kubectl  create clusterrolebinding magedu-admin  --user=magedu --clusterrole=admin
clusterrolebinding.rbac.authorization.k8s.io/magedu-admin created
[root@node01 chapter9]#

验证结果

[root@master01 usercerts]# kubectl get  po/mypod  --context=magedu@kubernetes 
NAME    READY   STATUS    RESTARTS   AGE
mypod   1/1     Running   0          28h
[root@master01 usercerts]# kubectl get  pv  --context=magedu@kubernetes 
Error from server (Forbidden): persistentvolumes is forbidden: User "magedu" cannot list resource "persistentvolumes" in API group "" at the cluster scope
[root@master01 usercerts]#

[root@master01 usercerts]# kubectl get  pvc -n longhorn-system --context=magedu@kubernetes
NAME                       STATUS   VOLUME                                     CAPACITY   ACCESS MODES   STORAGECLASS   AGE
longhorn-nfs-provisioner   Bound    pvc-d7fb4eab-2c72-4ec7-a087-4dbf35440fc5   19Gi       RWO            longhorn       4d4h
[root@master01 usercerts]#

eg2: cluster-admin

授权之前

[root@master01 ~]#  kubectl get  pv  --context=magedu@kubernetes 
Error from server (Forbidden): persistentvolumes is forbidden: User "magedu" cannot list resource "persistentvolumes" in API group "" at the cluster scope

授权

[root@master01 usercerts]#  kubectl  create clusterrolebinding magedu-clusteradmin  --user=magedu --clusterrole=cluster-admin
clusterrolebinding.rbac.authorization.k8s.io/magedu-clusteradmin created
[root@master01 usercerts]#

之后

[root@master01 ~]#  kubectl get  pv  --context=magedu@kubernetes 
NAME                                       CAPACITY   ACCESS MODES   RECLAIM POLICY   STATUS   CLAIM                                      STORAGECLASS   REASON   AGE
pvc-402583b0-5020-4e79-8b56-927ecf852fb2   2Gi        RWO            Delete           Bound    default/data-demodb-2                      longhorn                4d4h
pvc-6f5cab83-43bb-43d5-b7f3-01a1880725ec   2Gi        RWO            Delete           Bound    default/data-demodb-1                      longhorn                4d4h
pvc-ab203634-4009-46c1-bd5f-74871ee1cdd9   2Gi        RWO            Delete           Bound    default/data-demodb-0                      longhorn                4d4h
pvc-d7fb4eab-2c72-4ec7-a087-4dbf35440fc5   19Gi       RWO            Delete           Bound    longhorn-system/longhorn-nfs-provisioner   longhorn                4d4h
[root@master01 ~]#

集群级别授权示例2：集群角色权限降级

[root@master01 usercerts]#  kubectl  delete clusterrolebinding magedu-clusteradmin  
clusterrolebinding.rbac.authorization.k8s.io "magedu-clusteradmin" deleted
[root@master01 usercerts]#

也就是集群角色在特定名称空间下的权限映射

[root@master01 usercerts]#  kubectl  create  rolebinding magedu-admin  --user=magedu --clusterrole=admin  -n dev
rolebinding.rbac.authorization.k8s.io/magedu-admin created
[root@master01 usercerts]#

授权

[root@master01 usercerts]#  kubectl  create  rolebinding magedu-admin  --user=magedu --clusterrole=admin  -n dev
rolebinding.rbac.authorization.k8s.io/magedu-admin created

效果

[root@master01 ~]#  kubectl get  secret   -n dev --context=magedu@kubernetes 
NAME                  TYPE                                  DATA   AGE
admin-token-tkm7w     kubernetes.io/service-account-token   3      29h
default-token-dwmh6   kubernetes.io/service-account-token   3      29h
[root@master01 ~]#  kubectl get  pv  --context=magedu@kubernetes 
Error from server (Forbidden): persistentvolumes is forbidden: User "magedu" cannot list resource "persistentvolumes" in API group "" at the cluster scope
[root@master01 ~]#  kubectl get  secret   -n dev --context=magedu@kubernetes 
NAME                  TYPE                                  DATA   AGE
admin-token-tkm7w     kubernetes.io/service-account-token   3      29h
default-token-dwmh6   kubernetes.io/service-account-token   3      29h
[root@master01 ~]#

sa 授权

sa 授权是给pod 用的。使用场景 dashboard 用户访问

格式

–serviceaccount=名称空间：sa 名称

[root@node01 ~]# kubectl  create  sa  dev-admin   -n dev
serviceaccount/dev-admin created


[root@node01 ~]# kubectl  create  rolebinding  dev-admin --clusterrole=admin --serviceaccount=dev:dev-admin
rolebinding.rbac.authorization.k8s.io/dev-admin created
[root@node01 ~]#

使用了 sa 的pod 将拥有这个名称空间管理员的权限

dashboard

文档参考

https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/

默认 https 访问可以使用secret 注入自定义证书，如果不定义，它会自动生成证书

官方的版本不是特好使的话，可以用vmware Octant–vmware

安装

kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.4.0/aio/deploy/recommended.yaml

访问，改service nodeport externalip 都行

[root@node01 ~]# cat xx.yaml 
kind: Service
apiVersion: v1
metadata:
  labels:
    k8s-app: kubernetes-dashboard
  name: kubernetes-dashboard
  namespace: kubernetes-dashboard
spec:
  ports:
    - port: 443
      targetPort: 8443
  selector:
    k8s-app: kubernetes-dashboard
  externalIPs:
    - 192.168.2.3
[root@node01 ~]# 
[root@node01 ~]# kubectl  apply -f xx.yaml

打开https://192.168.2.3/ 使用被授权的sa 的 token 登录

创建sa 会自动创建同名的secret

[root@node01 ~]# kubectl  get secret/dev-admin-token-5ljvc -n dev -o jsonpath={.data.token}|base64 -d
eyJhbGciOiJSUzI1NiIsImtpZCI6IlBLYVBkYkIxcXZsMXpnM01hN2g4SG9NanlSZDY1YTd2cUhibUNTaWdEZkkifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJkZXYiLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlY3JldC5uYW1lIjoiZGV2LWFkbWluLXRva2VuLTVsanZjIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6ImRldi1hZG1pbiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6ImMwZmUzNDE4LWYwZmMtNDU1MS05ZWUwLWUyZDE1OWY4NjVkZiIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDpkZXY6ZGV2LWFkbWluIn0.az1C0AhE1SjNMu8QipdvhOPUfSLAJTgzcjsSfAMKQxMiRhK8pgskHw07fV3e6p-mzQ-nUziDmiwxAlz0XfGfOvB7Izc9xZfIobh5EZ15zMpK48OrecBF3iNR0XTfbgJJ15ECYcvYk5nXPG7zdgOqb5OYbqHBf1tcOK8xa-oAlYyTVlAGcgk8jcAWcOLv4cUARFXMDIQVFAVxe-HyhaTea3X-hHGPdfDZMuej5krD-yKUyMBHUoSKihC6YsUsQPZb7Er9sJHBF1fPCXcVrRAksvzGiNU0-Hq9lSKdBbMe7x7gaNodU2NwJVm6pT861h1GIUVVHpstdhdd5kwtkzoviQ[root@node01 ~]#

admin 角色权限太小的话，使用cluster-admin

[root@node01 ~]#  kubectl  get secret/max-token-8s4s7 -n kubernetes-dashboard -o jsonpath={.data.token}|base64 -d
eyJhbGciOiJSUzI1NiIsImtpZCI6IlBLYVBkYkIxcXZsMXpnM01hN2g4SG9NanlSZDY1YTd2cUhibUNTaWdEZkkifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlcm5ldGVzLWRhc2hib2FyZCIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJtYXgtdG9rZW4tOHM0czciLCJrdWJlcm5ldGVzLmlvL3NlcnZpY2VhY2NvdW50L3NlcnZpY2UtYWNjb3VudC5uYW1lIjoibWF4Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQudWlkIjoiYTc3ZjM0NzAtMTU1YS00ZDJhLTg5ZjctOTRmMGFiNzNmZDU4Iiwic3ViIjoic3lzdGVtOnNlcnZpY2VhY2NvdW50Omt1YmVybmV0ZXMtZGFzaGJvYXJkOm1heCJ9.nzFSUYrpYNLwFfO20nhBuw2rgpezZTNC3sca5N0JbPizgOaQNPBV-lcRQ3eAHkZL-MqotFjK394xCY3PTKPbrORsL6sr65N83kGNH0aDT6q27AFOn2z7QTd-FmHVXPr8UdGI_6PV-nGlYVIZq5aaocK7U4pyWkwzoimu9OoEVvIk0I6cNZJX6VmBwZhljwnZ_NIjtqcikVgj8QkwG_4cbO6JXt_L9N9Hu8y4kebpVsA6h-8XD5WI5no0ohWRDYVhhcM3cxI_x3jo978KfjsVfuvhzDksPsR6ovnjHZmlbGVI5JD-gD1WwnYOLXyIkbx9HKMS5mhVJ2MBVzHb8ijdFg[root@node01 ~]#

token 制作 kubeconfig 文件

不需要用户名（[–username=basic_user] [–password=basic_password]）不需要密码

只需要 token 就行。

此时制作kubeconfig ，把 set-credentials NAME 中的NAME 认为是用户名就行。

[root@node01 ~]# kubectl config set-cluster  kubernetes --server=https://kubeapi.magedu.com:6443   --kubeconfig=/tmp/k8s-admin.conf
Cluster "kubernetes" set.
[root@node01 ~]# kubectl  config set-credentials k8s-admin --kubeconfig=/tmp/k8s-admin.conf --token=`kubectl  get secret/max-token-8s4s7 -n kubernetes-dashboard -o jsonpath={.data.token}|base64 -d`
User "k8s-admin" set.
[root@node01 ~]#  kubectl   config  set-context 'k8s-admin@kubernetes'  --user=k8s-admin --cluster=kubernetes  --kubeconfig=/tmp/k8s-admin.conf
Context "k8s-admin@kubernetes" created.
[root@node01 ~]# kubectl   config  use-context k8s-admin@kubernetes  --kubeconfig=/tmp/k8s-admin.conf
Switched to context "k8s-admin@kubernetes".
[root@node01 ~]# sz  /tmp/k8s-admin.conf

[root@node01 chapter9]# kubectl   config  view --kubeconfig=/tmp/k8s-admin.conf
apiVersion: v1
clusters:
- cluster:
    server: https://kubeapi.magedu.com:6443
  name: kubernetes
contexts:
- context:
    cluster: kubernetes
    user: k8s-admin
  name: k8s-admin@kubernetes
current-context: k8s-admin@kubernetes
kind: Config
preferences: {}
users:
- name: k8s-admin
  user:
    token: REDACTED
[root@node01 chapter9]#

token 制作 kubeconfig 文件脚本

[root@node01 chapter9]# cat  gen-kubeconfig-based-sa.sh 
# Update these to match your environment
SERVICE_ACCOUNT_NAME=$1
CONTEXT=$(kubectl config current-context)
NAMESPACE=$2

NEW_CONTEXT=$3
KUBECONFIG_FILE="kubeconfig-sa"


SECRET_NAME=$(kubectl get serviceaccount ${SERVICE_ACCOUNT_NAME} \
  --context ${CONTEXT} \
  --namespace ${NAMESPACE} \
  -o jsonpath='{.secrets[0].name}')
TOKEN_DATA=$(kubectl get secret ${SECRET_NAME} \
  --context ${CONTEXT} \
  --namespace ${NAMESPACE} \
  -o jsonpath='{.data.token}')

TOKEN=$(echo ${TOKEN_DATA} | base64 -d)

# Create dedicated kubeconfig
# Create a full copy
kubectl config view --raw > ${KUBECONFIG_FILE}.full.tmp
# Switch working context to correct context
kubectl --kubeconfig ${KUBECONFIG_FILE}.full.tmp config use-context ${CONTEXT}
# Minify
kubectl --kubeconfig ${KUBECONFIG_FILE}.full.tmp \
  config view --flatten --minify > ${KUBECONFIG_FILE}.tmp
# Rename context
kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \
  rename-context ${CONTEXT} ${NEW_CONTEXT}
# Create token user
kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \
  set-credentials ${CONTEXT}-${NAMESPACE}-token-user \
  --token ${TOKEN}
# Set context to use token user
kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \
  set-context ${NEW_CONTEXT} --user ${CONTEXT}-${NAMESPACE}-token-user
# Set context to correct namespace
kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \
  set-context ${NEW_CONTEXT} --namespace ${NAMESPACE}
# Flatten/minify kubeconfig
kubectl config --kubeconfig ${KUBECONFIG_FILE}.tmp \
  view --flatten --minify > ${KUBECONFIG_FILE}
# Remove tmp
rm ${KUBECONFIG_FILE}.full.tmp
rm ${KUBECONFIG_FILE}.tmp

[root@node01 chapter9]# sh gen-kubeconfig-based-sa.sh   max  kubernetes-dashboard k8s-admin@kubernetes
[root@node01 chapter9]# ls kubeconfig-sa 
kubeconfig-sa

准入机制

limitranger:
为Pod添加默认的计算资源需求和计算资源（cpu mem）限制；以及存储资源需求和存储资源限制；支持分别在容器和Pod级别进行限制；
resourcequota：
限制资源数量，限制计算资源总量，存储资源总量；资源类型名称ResourceQuota
podsecuritpolicy：

在集群级别限制用户能够在Pod上可配置使用的securityContext。

示例

limitranger:

[root@node01 chapter9]# cat limitrange-demo.yaml 
apiVersion: v1
kind: LimitRange
metadata:
  name: core-resource-limits
  namespace: dev
spec:
  limits:
    - type: Pod
      max:
        cpu: "4" 
        memory: "4Gi" 
      min:
        cpu: "500m" 
        memory: "16Mi" 
    - type: Container
      max:
        cpu: "4" 
        memory: "1Gi" 
      min:
        cpu: "100m" 
        memory: "4Mi" 
      default:
        cpu: "2" 
        memory: "512Mi" 
      defaultRequest:
        cpu: "500m" 
        memory: "64Mi" 
      maxLimitRequestRatio:
        cpu: "4" 
    - type: PersistentVolumeClaim
      max:
        storage: "10Gi"
      min:
        storage: "1Gi"
      default:
        storage: "5Gi"
      defaultRequest:
        storage: "1Gi"
      maxLimitRequestRatio:
        storage: "5"

只有在容器级别可以设置dafault (limit属性值使用最大资源) defaultrequest（request 属性值最小） pod 级别不支持。。就是说容器才可以设置默认的min (request)、max(request)

maxLimitRequestRatio 申请时 request 必须大过 limit/比率

查看

[root@node01 chapter9]# kubectl  get  limits/core-resource-limits  -n dev
NAME                   CREATED AT
core-resource-limits   2021-12-26T03:38:33Z

[root@node01 chapter9]# kubectl describe  limits/core-resource-limits  -n dev
Name:                  core-resource-limits
Namespace:             dev
Type                   Resource  Min   Max   Default Request  Default Limit  Max Limit/Request Ratio
----                   --------  ---   ---   ---------------  -------------  -----------------------
Pod                    cpu       500m  4     -                -              -
Pod                    memory    16Mi  4Gi   -                -              -
Container              cpu       100m  4     500m             2              4
Container              memory    4Mi   1Gi   64Mi             512Mi          -
PersistentVolumeClaim  storage   1Gi   10Gi  1Gi              5Gi            5
[root@node01 chapter9]#

验证结果：

1、request limit 为空。验证默认值

[root@node01]# kubectl run testpod-$RANDOM --image="ikubernetes/demoapp:v1.0" -n dev
[root@node01 chapter9]#  kubectl get po/testpod-21604   -n dev -oyaml


spec:
  containers:
  - image: ikubernetes/demoapp:v1.0
    imagePullPolicy: IfNotPresent
    name: testpod-21604
    resources:
      limits:
        cpu: "2"
        memory: 512Mi
      requests:
        cpu: 500m
        memory: 64Mi

2、验证限制

[root@node01 chapter9]# kubectl run testpod-$RANDOM --image="ikubernetes/demoapp:v1.0" -n dev  --limits='cpu=2,memory=1Gi' --requests='cpu=1,memory=8Mi'

Error from server (Forbidden): pods "testpod-2968" is forbidden: minimum memory usage per Pod is 16Mi, but request is 8388608
[root@node01 chapter9]#

resourcequota

k8s 资源限制分3类

k8s 资源数量限制

存储资源限制

计算资源限制

计算资源限制

ResourceQuota资源可限制名称空间中处于非终止状态的所有Pod对象的计算资源需求及计算资源限制总量。

cpu或requests.cpu：CPU资源相关请求的总量限额；
memory或requests.memory：内存资源相关请求的总量限额；
limits.cpu：CPU资源相关限制的总量限额；
limits.memory：内存资源相关限制的总量限额；

存储资源限制

ResourceQuota资源还支持为本地名称空间中的PVC存储资源的需求总量和限制总量提供限额，它能够分别从名称空间中的全部PVC、隶属于特定存储类的PVC以及基于本地临时存储的PVC三个类别分别进行定义。

requests.storage：所有PVC存储需求的总量限额；空间限制；
persistentvolumeclaims：可以创建的PVC总数限额；数量限制；
<storage-class-name>.storageclass.storage.k8s.io/requests.storage：特定的存储类上可使用的所有PVC存储需求的总量限额；
<storage-class-name>.storageclass.storage.k8s.io/persistentvolumeclaims：特定的存储类上可使用的PVC总数限额；
requests.ephemeral-storage：所有Pod可以使用的本地临时存储资源的requets总量
limits.ephemeral-storage：所有Pod可用的本地临时存储资源的limits总量。

k8s 系统资源数量限制

在v1.9版本之前的Kubernetes系统上，ResourceQuota仅支持在有限的几种资源集上设定对象计数配额，例如pods、services和configmaps等，而自v1.9版本起开始支持以count/<resource>.<group>的格式支持对所有资源类型对象的计数配额，例如count/deployments.apps、count/deployments.extensions和 count/services等。

例子

[root@node01 chapter9]# cat resourcequota-demo.yaml 
apiVersion: v1
kind: ResourceQuota
metadata:
  name: resourcequota-demo
  namespace: dev
spec:
  hard:
    pods: "5"
    count/services: "5"
    count/configmaps: "5"
    count/secrets: "5"
    count/cronjobs.batch: "2"
    requests.cpu: "2"
    requests.memory: "4Gi"
    limits.cpu: "4"
    limits.memory: "8Gi"
    count/deployments.apps: "2"
    count/statefulsets.apps: "2"
    persistentvolumeclaims: "6"
    requests.storage: "20Gi"
    longhorn.storageclass.storage.k8s.io/requests.storage: "20Gi"
    longhorn.storageclass.storage.k8s.io/persistentvolumeclaims: "6"
[root@node01 chapter9]#

[root@node01 chapter9]# kubectl describe  quota/resourcequota-demo -n dev
Name:                                                        resourcequota-demo
Namespace:                                                   dev
Resource                                                     Used   Hard
--------                                                     ----   ----
count/configmaps                                             0      5
count/cronjobs.batch                                         0      2
count/deployments.apps                                       0      2
count/secrets                                                2      5
count/services                                               0      5
count/statefulsets.apps                                      0      2
limits.cpu                                                   2      4
limits.memory                                                512Mi  8Gi
longhorn.storageclass.storage.k8s.io/persistentvolumeclaims  0      6
longhorn.storageclass.storage.k8s.io/requests.storage        0      20Gi
persistentvolumeclaims                                       0      6
pods                                                         1      5
requests.cpu                                                 500m   2
requests.memory                                              64Mi   4Gi
requests.storage                                             0      20Gi

hard 表示硬限制，不能超过，和文件系统里的一样，软限制是在宽限期内可以超出，过了宽限期就直接删除

podsecuritpolicy

默认情况下，k8s 拒绝任何未经允许的pod 运行。默认psp 控制器是没有开启的

apiVersion: policy/v1beta1  # PSP资源所属的API群组及版本
kind: PodSecurityPolicy  # 资源类型标识
metadata:
  name <string>  # 资源名称
spec:  
  allowPrivilegeEscalation  <boolean>  # 是否允许权限升级
  allowedCSIDrivers <[]Object>  #内联CSI驱动程序列表，必须在Pod规范中显式定义
  allowedCapabilities <[]string>  # 允许使用的内核能力列表，“*”表示all
  allowedFlexVolumes <[]Object>  # 允许使用的Flexvolume列表，空值表示“all
  allowedHostPaths <[]Object>  # 允许使用的主机路径列表，空值表示all
  allowedProcMountTypes <[]string> # 允许使用的ProcMountType列表，空值表示默认
  allowedUnsafeSysctls <[]string> # 允许使用的非安全sysctl参数，空值表示不允许
  defaultAddCapabilities  <[]string>  # 默认即添加到Pod对象的内核能力，可被drop
  defaultAllowPrivilegeEscalation <boolean> # 是否默认允许内核权限升级
  forbiddenSysctls  <[]string> # 禁止使用的sysctl参数，空表示不禁用
  fsGroup <Object>  # 允许在SecurityContext中使用的fsgroup，必选字段
    rule <string>  # 允许使用的FSGroup的规则，支持RunAsAny和MustRunAs
    ranges <[]Object> # 允许使用的组ID范围，需要与MustRunAs规则一同使用
      max  <integer>  # 最大组ID号
      min  <integer>  # 最小组ID号
  hostIPC <boolean> # 是否允许Pod使用hostIPC
  hostNetwork <boolean> # 是否允许Pod使用hostNetwork
  hostPID <boolean> # 是否允许Pod使用hostPID
  hostPorts <[]Object>  # 允许Pod使用的主机端口暴露其服务的范围
    max  <integer>  # 最大端口号，必选字段
    min  <integer>  # 最小端口号，必选字段
  privileged  <boolean>  # 是否允许运行特权Pod
  readOnlyRootFilesystem  <boolean>  # 是否设定容器的根文件系统为“只读”
  requiredDropCapabilities <[]string> # 必须要禁用的内核能力列表  
  runAsGroup  <Object>  # 允许Pod在runAsGroup中使用的值列表，未定义表示不限制
  runAsUser <Object> # 允许Pod在runAsUser中使用的值列表，必选字段
    rule <string>  # 支持RunAsAny、MustRunAs和MustRunAsNonRoot
    ranges <[]Object> # 允许使用的组ID范围，需要跟“MustRunAs”规则一同使用
      max  <integer>  # 最大组ID号
      min  <integer>  # 最小组ID号
  runtimeClass <Object> # 允许Pod使用的运行类，未定义表示不限制
    allowedRuntimeClassNames <[]string> # 可使用的runtimeClass列表，“*”表示all
    defaultRuntimeClassName <string> # 默认使用的runtimeClass
  seLinux <Object> # 允许Pod使用的selinux标签，必选字段
    rule <string>  # MustRunAs表示使用seLinuxOptions定义的值；RunAsAny表示可使用任意值
    seLinuxOptions  <Object>  # 自定义seLinux选项对象，与MustRunAs协作生效
  supplementalGroups  <Object> # 允许Pod在SecurityContext中使用附加组，必选字段  volumes <[]string>  # 允许Pod使用的存储卷插件列表，空表示禁用，“*”表示全部

即时我们创建了psp 资源也不会生效，必须启用psp 控制器才行

apiserver 开启 --enable-admission-plugins strings 即可。如果你开启了这个选项，但是没有psp 类型资源，你运行的所有pod 都可能被拒绝。

所以启用功能之前，必须先创建psp 资源

[root@master01 ~]#  kubectl exec kube-apiserver-master01 -it  -n kube-system  -- kube-apiserver  --help |grep '\--enable-admission-plugins strings'
      --enable-admission-plugins strings       admission plugins that should be enabled in addition to default enabled ones (NamespaceLifecycle, LimitRanger, ServiceAccount, TaintNodesByCondition, Priority, DefaultTolerationSeconds, DefaultStorageClass, StorageObjectInUseProtection, PersistentVolumeClaimResize, RuntimeClass, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, MutatingAdmissionWebhook, ValidatingAdmissionWebhook, ResourceQuota). Comma-delimited list of admission plugins: AlwaysAdmit, AlwaysDeny, AlwaysPullImages, CertificateApproval, CertificateSigning, CertificateSubjectRestriction, DefaultIngressClass, DefaultStorageClass, DefaultTolerationSeconds, DenyEscalatingExec, DenyExecOnPrivileged, EventRateLimit, ExtendedResourceToleration, ImagePolicyWebhook, LimitPodHardAntiAffinityTopology, LimitRanger, MutatingAdmissionWebhook, NamespaceAutoProvision, NamespaceExists, NamespaceLifecycle, NodeRestriction, OwnerReferencesPermissionEnforcement, PersistentVolumeClaimResize, PersistentVolumeLabel, PodNodeSelector, PodPreset, PodSecurityPolicy, PodTolerationRestriction, Priority, ResourceQuota, RuntimeClass, SecurityContextDeny, ServiceAccount, StorageObjectInUseProtection, TaintNodesByCondition, ValidatingAdmissionWebhook. The order of plugins in this flag does not matter.
[root@master01 ~]#

eg:

1、角色绑定

[root@node01 chapter9]# cat clusterrole-with-psp.yaml
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp-restricted
rules:
- apiGroups: ['policy']
  resources: ['podsecuritypolicies']
  verbs:     ['use']
  resourceNames:
  - restricted
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: psp-privileged
rules:
- apiGroups: ['policy']
  resources: ['podsecuritypolicies']
  verbs:     ['use']
  resourceNames:
  - privileged

[root@node01 chapter9]# cat clusterrolebinding-with-psp.yaml 
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
  name: privileged-psp-user
roleRef:
  apiGroup: rbac.authorization.k8s.io
  kind: ClusterRole
  name: psp-privileged
subjects:
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:masters
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:node
- apiGroup: rbac.authorization.k8s.io
  kind: Group
  name: system:serviceaccounts:kube-system
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
  name: restricted-psp-user
roleRef:
  kind: ClusterRole
  name: psp-restricted
  apiGroup: rbac.authorization.k8s.io
subjects:
- kind: Group
  apiGroup: rbac.authorization.k8s.io
  name: system:authenticated

2、创建psp 策略

对于管理员用户

[root@node01 chapter9]# cat psp-privileged 
# Pod Security Policy
# privileged policy
# Maintainer: MageEdu <mage@magedu.com>
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: privileged
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: '*'
spec:
  privileged: true
  allowPrivilegeEscalation: true
  allowedCapabilities:
  - '*'
  allowedUnsafeSysctls:
  - '*'
  volumes:
  - '*'
  hostNetwork: true
  hostPorts:
  - min: 0
    max: 65535
  hostIPC: true
  hostPID: true
  runAsUser:
    rule: 'RunAsAny'
  runAsGroup:
    rule: 'RunAsAny'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'RunAsAny'
  fsGroup:
    rule: 'RunAsAny'

对于普通用户

[root@node01 chapter9]# cat psp-restricted.yaml 
# Pod Security Policy
# Unprivileged policy
# Maintainer: MageEdu <mage@magedu.com>
---
apiVersion: policy/v1beta1
kind: PodSecurityPolicy
metadata:
  name: restricted
  annotations:
    seccomp.security.alpha.kubernetes.io/allowedProfileNames: 'docker/default'
    apparmor.security.beta.kubernetes.io/allowedProfileNames: 'runtime/default'
    seccomp.security.alpha.kubernetes.io/defaultProfileName:  'docker/default'
    apparmor.security.beta.kubernetes.io/defaultProfileName:  'runtime/default'
spec:
  privileged: false
  allowPrivilegeEscalation: false
  allowedUnsafeSysctls: []
  requiredDropCapabilities:
    - ALL
  # Allow core volume types.
  volumes:
    - 'configMap'
    - 'emptyDir'
    - 'projected'
    - 'secret'
    - 'downwardAPI'
    - 'persistentVolumeClaim'
  hostNetwork: false
  hostIPC: false
  hostPID: false
  runAsUser:
    rule: 'MustRunAsNonRoot'
  seLinux:
    rule: 'RunAsAny'
  supplementalGroups:
    rule: 'MustRunAs'
    ranges:
      # Forbid adding the root group.
      - min: 1
        max: 65535
  fsGroup:
    rule: 'MustRunAs'
    ranges:
      # Forbid adding the root group.
      - min: 1
        max: 65535
  readOnlyRootFilesystem: false
[root@node01 chapter9]#

3、启用 psp选项

K8S/Kubernetes

K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容

更多推荐

【深度】阿里巴巴万级规模 K8s 集群全局高可用体系之美

作者 | 韩堂、柘远、沉醉来源 | 阿里巴巴云原生公众号前言台湾作家林清玄在接受记者采访的时候，如此评价自己 30 多年写作生涯：“第一个十年我才华横溢，‘贼光闪现’，令周边黯然失色；第二个十年，我终于‘宝光现形’，不再去抢风头，反而与身边的美丽相得益彰；进入第三个十年，繁华落尽见真醇，我进入了‘醇光初现’的阶段，真正体味到了境界之美”。长夜有穷，真水无香。领略过了 K8s“身在江

K8S/Kubernetes

如何基于 K8s 构建下一代 DevOps 平台？

作者 | 孙健波（天元）导读：当前云原生 DevOps 体系现状如何？面临哪些挑战？如何通过 OAM 解决云原生 DevOps 场景下的诸多问题？云原生开发应用模型 OAM(Open Application Model) 社区核心成员孙健波将为大家一一解答，并分享如何基于 OAM 和 Kubernetes 打造无限能力的下一代 DevOps 平台。什么是 DevOps？为什么基于 Kub