HCIE-kubernetes(k8s)-Authentication身份验证
例如使用web来管理k8s环境时，需要调用相关的程序（pod中）有对K8s管理，当时此程序需要有相关的权限，这时候就可以创建一个SA，赋予相关的权限，并且与管理K8s的环境程序进行绑定，那么这个程序就有了SA的对应权限，可以对K8s进行管理。创建csv文件，当 API 服务器的命令行设置了 --token-auth-file=/etc/kubernetes/pki/static.csv选项时，会从
·
1、远程登录
1、kubeconfig方式
在master上都是以kubeconfig方式登录的，并不是说有一个文件叫kubeconfig。
默认使用的配置文件是~/.kube/config 这个配置文件，而这个配置文件是通过这个文件/etc/kubernetes/admin.conf
如果在node上执行命令，也需要配置该文件，否则报错如下：
[root@node1 ~]# kubectl get node
E1103 15:42:46.225278 129198 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
E1103 15:42:46.226197 129198 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
E1103 15:42:46.228511 129198 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
E1103 15:42:46.230491 129198 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
E1103 15:42:46.232909 129198 memcache.go:265] couldn't get current server API group list: Get "http://localhost:8080/api?timeout=32s": dial tcp [::1]:8080: connect: connection refused
The connection to the server localhost:8080 was refused - did you specify the right host or port?
可以在master拷贝一份/etc/kubernetes/admin.conf到node1
[root@master ~]# cd /etc/kubernetes
[root@master kubernetes]# scp admin.conf 10.1.1.201:~
在node1上执行命令的时候，直接指定该文件运行。
[root@node1 ~]# kubectl get node --kubeconfig=admin.conf
NAME STATUS ROLES AGE VERSION
master Ready control-plane 8d v1.27.0
node1 Ready <none> 8d v1.27.0
node2 Ready <none> 8d v1.27.0
如果不想指定配置文件，可以直接配置环境变量。
[root@node1 ~]# echo 'export KUBECONFIG=admin.conf' >> /etc/profile
[root@node1 ~]# source /etc/profile
[root@node1 ~]# kubectl get node
NAME STATUS ROLES AGE VERSION
master Ready control-plane 8d v1.27.0
node1 Ready <none> 8d v1.27.0
node2 Ready <none> 8d v1.27.0
这样就不需要指定配置文件了。
注意：该admin.conf文件是直接从master上拷贝过来的，所以默认具备admin管理员最大的权限的。
生产环境不建议这样操作，因为权限过大，无法保证安全性。
例如单独创建用户tom授权
1、生成私钥
[root@master kubernetes]# mkdir /ca
[root@master kubernetes]# cd /ca
[root@master ca]# openssl genrsa -out client.key 2048
Generating RSA private key, 2048 bit long modulus (2 primes)
.....................................+++++
.......+++++
e is 65537 (0x010001)
[root@master ca]# ls
client.key
2、生成tom用户证书请求文件
[root@master ca]# openssl req -new -key client.key -subj "/CN=tom" -out client.csr
[root@master ca]# ls
client.csr client.key
k8s本身使用的一些ca证书，是在/etc/kubernetes/pki/目录下
[root@master ca]# ls /etc/kubernetes/pki/
apiserver.crt apiserver-etcd-client.key apiserver-kubelet-client.crt ca.crt etcd front-proxy-ca.key front-proxy-client.key sa.pub
apiserver-etcd-client.crt apiserver.key apiserver-kubelet-client.key ca.key front-proxy-ca.crt front-proxy-client.crt sa.key
3、为tom用户颁发证书
tom如何将请求发给k8s的ca进行证书颁发呢？使用kubernetes自带的ca为tom颁发证书。
[root@master ca]# openssl x509 -req -in client.csr -CA /etc/kubernetes/pki/ca.crt -CAkey /etc/kubernetes/pki/ca.key -CAcreateserial -out client.crt -days 3650
Signature ok
subject=CN = tom
Getting CA Private Key
[root@master ca]# ls
client.crt client.csr client.key
拷贝ca证书到当前目录
[root@master ca]# cp /etc/kubernetes/pki/ca.crt .
[root@master ca]# ls
ca.crt client.crt client.csr client.key
4、创建测试命名空间及pod
[root@master ca]# kubectl create namespace hehe
namespace/hehe created
[root@master ca]# kubens hehe
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "hehe".
[root@master ca]# kubectl get pod
No resources found in hehe namespace.
[root@master ca]# kubectl run pod1 --image nginx --image-pull-policy IfNotPresent --dry-run=client -o yaml > pod1.yaml
[root@master ca]# kubectl apply -f pod1.yaml
pod/pod1 created
[root@master ca]# kubectl get pod
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 7s
5、创建角色和授权
创建角色，使其对hehe命名空间具有get/watch/list权限
[root@master ca]# kubectl create role r1 --verb get --verb watch --verb list --resource pods -n hehe
role.rbac.authorization.k8s.io/r1 created
查看角色：[root@master ca]# kubectl get role
NAME CREATED AT
r1 2023-11-03T09:07:10Z
详细查看：[root@master ca]# kubectl describe role r1
Name: r1
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get watch list]
6、将角色绑定给tom用户
[root@master ca]# kubectl create rolebinding r1binding --role r1 --user tom -n hehe
rolebinding.rbac.authorization.k8s.io/r1binding created
[root@master ca]# kubectl get rolebindings.rbac.authorization.k8s.io
NAME ROLE AGE
r1binding Role/r1 17s
[root@master ca]# kubectl describe rolebindings.rbac.authorization.k8s.io
Name: r1binding
Labels: <none>
Annotations: <none>
Role:
Kind: Role
Name: r1
Subjects:
Kind Name Namespace
---- ---- ---------
User tom
7、编辑kuberconfig文件
官网，https://kubernetes.io/zh-cn/docs/concepts/configuration/organize-cluster-access-kubeconfig/
[root@master ca]# vim kubeabc
[root@master ca]# vim kubeabc
[root@master ca]# cat kubeabc
apiVersion: v1
kind: Config
clusters:
- cluster:
name: dev1
users:
- name: tom
contexts:
- context:
name: context1
namespace: hehe
current-context: "context1"
设置集群密钥信息（嵌入集群密钥）master的ip:6443
[root@master ca]# kubectl config --kubeconfig=kubeabc set-cluster dev1 --server=https://10.1.1.200:6443 --certificate-authority=ca.crt --embed-certs=true
Cluster "dev1" set.
设置tom用户密钥信息
[root@master ca]# kubectl config --kubeconfig=kubeabc set-credentials tom --client-certificate=client.crt --client-key=client.key --embed-certs=true
User "tom" set.
设置上下文信息
[root@master ca]# kubectl config --kubeconfig=kubeabc set-context context1 --cluster=dev1 --namespace=hehe --user=tom
Context "context1" modified.
8、测试
拷贝至客户端node2并测试
[root@master ca]# scp kubeabc 10.1.1.202:~
[root@node2 ~]# kubectl get pod --kubeconfig=kubeabc
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 24m
这样客户端node2就以tom身份登录到k8s集群中，进行管理操作了。
[root@node2 ~]# export KUBECONFIG=kubeabc
[root@node2 ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 26m
2、静态token方式
官网，https://kubernetes.io/zh-cn/docs/reference/access-authn-authz/authentication/#static-token-file
创建csv文件，当 API 服务器的命令行设置了 --token-auth-file=/etc/kubernetes/pki/static.csv选项时，会从文件中读取持有者令牌。 目前，令牌会长期有效，并且在不重启 API 服务器的情况下无法更改令牌列表。
令牌文件是一个 CSV 文件，包含至少 3 个列：令牌、用户名和用户的 UID（唯一的）。 其余列被视为可选的组名。如果要设置的组名不止一个，则对应的列必须用双引号括起来，例如：token,user,uid,“group1,group2,group3”
[root@master ~]# vim /etc/kubernetes/manifests/kube-apiserver.yaml
spec:
containers:
- command:
- kube-apiserver
- --advertise-address=10.1.1.200
- --token-auth-file=/etc/kubernetes/pki/static.csv
/etc/kubernetes/pki/static.csv 该文件就是保存用户账号和密码的。建议不要更换目录，因为k8s本身是对该目录有足够权限的。
生成token令牌[root@master ~]# openssl rand -hex 10
7e96a8829598306b0171
[root@master ~]# cd /etc/kubernetes/pki/
[root@master pki]# ls
apiserver.crt apiserver-etcd-client.key apiserver-kubelet-client.crt ca.crt ca.srl front-proxy-ca.crt front-proxy-client.crt sa.key
apiserver-etcd-client.crt apiserver.key apiserver-kubelet-client.key ca.key etcd front-proxy-ca.key front-proxy-client.key sa.pub
[root@master pki]# vim abc.csv
[root@master pki]# cat abc.csv
7e96a8829598306b0171,tom,3
[root@master pki]# systemctl restart kubelet.service
重启node2将此前node2的临时环境变量清除（export KUBECONFIG=kubeabc）
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' get pods -n hehe
Unable to connect to the server: tls: failed to verify certificate: x509: certificate signed by unknown authority
提示x509加密认证
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pods -n hehe
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 42h
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pods -n default
Error from server (Forbidden): pods is forbidden: User "tom" cannot list resource "pods" in API group "" in the namespace "default"
提示该用户是没有权限的。登录过来没有权限，那如何进行授权呢？
2、角色授权
官网，https://kubernetes.io/zh-cn/docs/reference/access-authn-authz/authorization/
[root@master pki]# cat /etc/kubernetes/manifests/kube-apiserver.yaml
spec:
containers:
- command:
- kube-apiserver
- --advertise-address=10.1.1.200
- --token-auth-file=/etc/kubernetes/pki/abc.csv
- --allow-privileged=true
- --authorization-mode=Node,RBAC
注意这一行 - --authorization-mode=Node,RBAC
除了这两个，还有以下：
--authorization-mode=ABAC 基于属性的访问控制（ABAC）模式允许你使用本地文件配置策略。
--authorization-mode=RBAC 基于角色的访问控制（RBAC）模式允许你使用 Kubernetes API 创建和存储策略。把权限放在角色包里，把角色授权用户。
--authorization-mode=Webhook WebHook 是一种 HTTP 回调模式，允许你使用远程 REST 端点管理鉴权。
--authorization-mode=Node 节点鉴权是一种特殊用途的鉴权模式，专门对 kubelet 发出的 API 请求执行鉴权。
--authorization-mode=AlwaysDeny 该标志阻止所有请求。仅将此标志用于测试。
--authorization-mode=AlwaysAllow 此标志允许所有请求。仅在你不需要 API 请求的鉴权时才使用此标志。
接下来，把api里面默认用的授权方式改成 AlwaysAllow ，重启api服务，再次尝试tom用户是否可以直接登录。
[root@master pki]# vim /etc/kubernetes/manifests/kube-apiserver.yaml
[root@master pki]# cat /etc/kubernetes/manifests/kube-apiserver.yaml
spec:
containers:
- command:
- kube-apiserver
- --advertise-address=10.1.1.200
- --token-auth-file=/etc/kubernetes/pki/abc.csv
- --allow-privileged=true
- --authorization-mode=AlwaysAllow
[root@master pki]# systemctl restart kubelet.service
[root@master pki]# kubectl get nodes
NAME STATUS ROLES AGE VERSION
master Ready control-plane 10d v1.27.0
node1 Ready <none> 10d v1.27.0
node2 Ready <none> 10d v1.27.0
再次在node2上执行命令查询
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pods -n default
No resources found in default namespace.
这时候会发现，即使集群没有针对tom进行授权，也可以直接访问到该pod。
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pods -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-7bdc4cb885-4kvvb 1/1 Running 10 (159m ago) 10d
coredns-7bdc4cb885-4tw5z 1/1 Running 9 (159m ago) 10d
etcd-master 1/1 Running 7 (2d5h ago) 10d
kube-apiserver-master 0/1 Running 0 3m56s
kube-controller-manager-master 1/1 Running 39 (7m15s ago) 10d
kube-proxy-hmk7x 1/1 Running 7 (2d5h ago) 10d
kube-proxy-x7hmw 1/1 Running 8 (159m ago) 10d
kube-proxy-xr55g 1/1 Running 7 (2d5h ago) 10d
kube-scheduler-master 1/1 Running 38 (7m15s ago) 10d
metrics-server-7f5dc7b49f-nhrfh 1/1 Running 27 (2d5h ago) 10d
反之，如果api参数被改为 AlwaysDeny，那么不管该用户是否授权，一律不可以访问。当然，除了master可以，因为默认master是通过本地主机进行访问的。不受该参数影响。
role 与 rolebinding，是基于命名空间来授权的
查看集群中的角色
[root@master pki]# kubectl get clusterrole
查看当前集群中管理员角色，看他的权限。
[root@master pki]# kubectl describe clusterrole admin
角色又分为两种：role和clusterrole
role：是基于命名空间的ns1，通过ns进行隔离，只会在当前ns1里面生效。
把角色绑定给用户abc，叫rolebinding。那么abc这个用户他在ns1里面只会具备role角色定义的权限。
[root@master pki]# cat /etc/kubernetes/manifests/kube-apiserver.yaml
spec:
containers:
- command:
- kube-apiserver
- --advertise-address=10.1.1.200
- --token-auth-file=/etc/kubernetes/pki/abc.csv
- --allow-privileged=true
- --authorization-mode=Node,RBAC
[root@master pki]# systemctl restart kubelet.service
实验：创建一个ns，abc
[root@master ~]# kubectl create ns abc
namespace/abc created
[root@master ~]# kubectl get ns
NAME STATUS AGE
abc Active 20s
calico-apiserver Active 10d
calico-system Active 10d
default Active 10d
hehe Active 45h
ingress-nginx Active 5d22h
kube-node-lease Active 10d
kube-public Active 10d
kube-system Active 10d
metallb-system Active 7d
tigera-operator Active 10d
[root@master ~]# cat pod1.yaml
apiVersion: v1
kind: Pod
metadata:
creationTimestamp: null
labels:
run: pod1
app: nginx
name: pod1
spec:
containers:
- image: nginx
imagePullPolicy: IfNotPresent
name: pod1
resources: {}
dnsPolicy: ClusterFirst
restartPolicy: Always
status: {}
[root@master ~]# kubectl apply -f pod1.yaml -n abc
pod/pod1 created
[root@master ~]# kubectl get pod -n abc
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 9s
默认情况下，node2 tom用户是无法访问abc里面的pod的
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pods -n abc
Error from server (Forbidden): pods is forbidden: User "tom" cannot list resource "pods" in API group "" in the namespace "abc"
默认abc里面也没有任何角色
[root@master ~]# kubectl get role -n abc
No resources found in abc namespace.
创建一个角色
[root@master ~]# kubectl create role pod-reader01 --verb get,list,watch,delete --resource pods --dry-run=client -o yaml > role1.yaml
[root@master ~]# cat role1.yaml 删除delete权限
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: pod-reader01
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- delete
[root@master ~]# kubectl apply -f role1.yaml -n abc
role.rbac.authorization.k8s.io/pod-reader01 created
[root@master ~]# kubectl get role -n abc
NAME CREATED AT
pod-reader01 2023-11-05T06:33:18Z
[root@master ~]# kubectl describe role pod-reader01 -n abc
Name: pod-reader01
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get list watch]
这时候该角色仅仅针对pods类型具有 get list 和watch操作。这时候node2的tom可以访问吗？不可以，还没有绑定。
[root@master ~]# kubectl get rolebindings.rbac.authorization.k8s.io -n abc
No resources found in abc namespace.
绑定tom用户
注意带上 --user 和 --token ，因为是根据token认证的，所以必须带上，否则后面无法进行创建和删除pod
[root@master ~]# kubectl create rolebinding rb1-tom --role pod-reader01 --user tom --dry-run=client -o yaml > rb1.yaml
[root@master ~]# cat rb1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: RoleBinding
metadata:
creationTimestamp: null
name: rb1-tom
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: Role
name: pod-reader01
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: tom
[root@master ~]# kubectl apply -f rb1.yaml -n abc
rolebinding.rbac.authorization.k8s.io/rb1-tom created
[root@master ~]# kubectl get rolebindings.rbac.authorization.k8s.io -n abc
NAME ROLE AGE
rb1-tom Role/pod-reader01 11s
tom查看pod
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pods -n abc
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 18m
可以查看其他ns资源吗？不可以。因为角色是在abc命名空间设置的权限，而不是在kube-system里面。
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pods -n kube-system
Error from server (Forbidden): pods is forbidden: User "tom" cannot list resource "pods" in API group "" in the namespace "kube-system"
tom可以删除吗？不可以，因为授权的时候，没有授权delete
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true delete pods/pod1 -n abc
Error from server (Forbidden): pods "pod1" is forbidden: User "tom" cannot delete resource "pods" in API group "" in the namespace "abc"
现在增加一个delete权限。
[root@master ~]# vim role1.yaml
[root@master ~]# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: pod-reader01
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- delete
[root@master ~]# kubectl apply -f role1.yaml -n abc
role.rbac.authorization.k8s.io/pod-reader01 configured
这里有问题了，给delete，create，依然无法创建或删除pod。因为上面创建binding的时候，没有添加token参数。
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true delete pods/pod1 -n abc
Error from server (Forbidden): pods "pod1" is forbidden: User "tom" cannot delete resource "pods" in API group "" in the namespace "abc"
现在删除binding。重新创建。
[root@master ~]# kubectl create rolebinding rb1-tom --role pod-reader01 --user='tom' --token='7e96a8829598306b0171' --dry-run=client -o yaml > rb1.yaml
[root@master ~]# kubectl apply -f rb1.yaml -n abc
rolebinding.rbac.authorization.k8s.io/rb1-tom configured
[root@master ~]# vim role1.yaml
[root@master ~]# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: pod-reader01
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
- delete
- create
[root@master ~]# kubectl apply -f role1.yaml -n abc
role.rbac.authorization.k8s.io/pod-reader01 configured
在node2删除pod
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true delete pods/pod1 -n abc
pod "pod1" deleted
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pods -n abc
No resources found in abc namespace.
在node2创建pod
[root@master ~]# scp pod1.yaml 10.1.1.202:/root/
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true apply -f pod1.yaml -n abc
pod/pod1 created
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pods -n abc
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 3s
同样，也可以针对其他对象，比如deployment操作。
[root@master ~]# vim role1.yaml
[root@master ~]# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: pod-reader01
rules:
- apiGroups:
- ""
resources:
- pods
- deployments
verbs:
- get
- list
- watch
- delete
- create
[root@master ~]# kubectl apply -f role1.yaml -n abc
role.rbac.authorization.k8s.io/pod-reader01 configured
[root@master ~]# kubectl get deployments.apps -n abc
No resources found in abc namespace.
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get deploy -n abc
Error from server (Forbidden): deployments.apps is forbidden: User "tom" cannot list resource "deployments" in API group "apps" in the namespace "abc"
会发现，查询deploy是不可以的。
pod--apiversion v1
services--apiversion v1
deployment--apps/v1
不同的这些资源类型，他们的apiversion是不一样的。查询集群中所有的apiversion
[root@master ~]# kubectl api-versions
[root@master ~]# kubectl api-resources -o wide
这里面version分为两种，一种带斜杠，一种不带斜杠。前者是两级结构，后者一级结构。
[root@master ~]# kubectl api-resources -o wide |grep deployment
deployments deploy apps/v1 true Deployment create,delete,deletecollection,get,list,patch,update,watch all
[root@master ~]# kubectl api-resources -o wide |grep pod
pods po v1 true Pod create,delete,deletecollection,get,list,patch,update,watch all
deployment对应的apiversion为两级结构，所以，要写两级，要把父级写入到apigroup里面，才能生效。而pod和svc都是一级结构，所以父级apigroup为空。
所以改一下：
[root@master ~]# vim role1.yaml
[root@master ~]# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: pod-reader01
rules:
- apiGroups:
- ""
- apps
resources:
- pods
- deployments
verbs:
- get
- list
- watch
- delete
- create
[root@master ~]# kubectl apply -f role1.yaml -n abc
role.rbac.authorization.k8s.io/pod-reader01 configured
node2上查询deployment
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get deploy -n abc
No resources found in abc namespace.
deploy可以查看了，也可以创建deployment
[root@master ~]# kubectl create deployment web1 --image nginx --dry-run=client -o yaml > web1.yaml
[root@master ~]# vim web1.yaml
[root@master ~]# cat web1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: web1
name: web1
spec:
replicas: 1
selector:
matchLabels:
app: web1
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: web1
spec:
containers:
- image: nginx
name: nginx
imagePullPolicy: IfNotPresent
resources: {}
status: {}
[root@master ~]# scp web1.yaml 10.1.1.202:~
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true apply -f web1.yaml -n abc
deployment.apps/web1 created
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get deploy -n abc
NAME READY UP-TO-DATE AVAILABLE AGE
web1 0/1 1 0 5s
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true get pod -n abc
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 21m
web1-58d8bd576b-7j9sw 1/1 Running 0 29s
scale扩展
[root@node2 ~]# alias kubectl='kubectl --server='https://10.1.1.200:6443' --token='7e96a8829598306b0171' --insecure-skip-tls-verify=true'
[root@node2 ~]# kubectl get pod -n abc
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 23m
web1-58d8bd576b-7j9sw 1/1 Running 0 3m2s
[root@node2 ~]# kubectl get deploy -n abc
NAME READY UP-TO-DATE AVAILABLE AGE
web1 1/1 1 1 3m12s
[root@node2 ~]# kubectl scale deploy web1 --replicas=3 -n abc
Error from server (Forbidden): deployments.apps "web1" is forbidden: User "tom" cannot patch resource "deployments/scale" in API group "apps" in the namespace "abc"
这时候提示resources是无法修补资源的。把 deployments/scale 加入到resources资源里面，重新apply应用下
[root@master ~]# vim role1.yaml
[root@master ~]# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: pod-reader01
rules:
- apiGroups:
- ""
- apps
resources:
- pods
- deployments
- deployments/scale
verbs:
- get
- list
- watch
- delete
- create
[root@master ~]# kubectl apply -f role1.yaml -n abc
role.rbac.authorization.k8s.io/pod-reader01 configured[root@node2 ~]# kubectl scale deploy web1 --replicas=3 -n abc
Error from server (Forbidden): deployments.apps "web1" is forbidden: User "tom" cannot patch resource "deployments/scale" in API group "apps" in the namespace "abc"
这时候依然不行。因为role脚本里面没有给scale这个权限。所以当修改副本数的时候，实际上是更新update权限还有patch补丁权限。
[root@master ~]# vim role1.yaml
[root@master ~]# cat role1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: Role
metadata:
creationTimestamp: null
name: pod-reader01
rules:
- apiGroups:
- ""
- apps
resources:
- pods
- deployments
- deployments/scale
verbs:
- get
- list
- watch
- delete
- create
- patch
- update
[root@master ~]# kubectl apply -f role1.yaml -n abc
role.rbac.authorization.k8s.io/pod-reader01 configured
再次执行scale
[root@node2 ~]# kubectl scale deploy web1 --replicas=3 -n abc
deployment.apps/web1 scaled
[root@node2 ~]# kubectl get pod -n abc
NAME READY STATUS RESTARTS AGE
pod1 1/1 Running 0 36m
web1-58d8bd576b-7j9sw 1/1 Running 0 15m
web1-58d8bd576b-lqprd 1/1 Running 0 3s
web1-58d8bd576b-rg9dm 1/1 Running 0 3s
发现这次可以了。这时候会考虑，如何根据的不同的对象，设置不同的权限。例如：
rules:
- apiGroups: [""]
resources: ["pods"]
verbs: ["get", "list", "watch"]
rules:
- apiGroups: ["apps"]
resources: ["deployments"]
verbs: ["get", "list", "watch", "create", "update", "patch", "delete"]
以上的作用域是在ns级别。并不是整个集群的。
clusterrole 与 clusterrolebinding，是基于集群所有命名空间来授权的
clusterrole对于role来说，是全局的，整个集群命名空间都生效。且clusterrole可以进行clusterrolebinding，也可以进行rolebinding，通过 --namespace 来指定命名空间。比如[root@master ~]# kubectl create rolebinding multins --clusterrole pod-reader --user henry --token=‘46c6cc3b55f372736657’ --namespace=default --namespace=kube-system
[root@master pki]# kubens default
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "default".
[root@master ~]# cd /etc/kubernetes/pki/
[root@master pki]# openssl rand -hex 10
0be44aeaaea07a5ab624
[root@master pki]# vim abc.csv
[root@master pki]# cat abc.csv
7e96a8829598306b0171,tom,3
6c576631402edca09b93,jerry,4
[root@master pki]# systemctl restart kubelet.service
创建角色
[root@master ~]# kubectl create clusterrole pod-reader --verb=get,list,watch --resource=pods --dry-run=client -o yaml > clusterrole1.yaml
[root@master ~]# cat clusterrole1.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRole
metadata:
creationTimestamp: null
name: pod-reader
rules:
- apiGroups:
- ""
resources:
- pods
verbs:
- get
- list
- watch
[root@master ~]# kubectl apply -f clusterrole1.yaml
clusterrole.rbac.authorization.k8s.io/pod-reader created
[root@master ~]# kubectl get clusterrole |grep pod
pod-reader 2023-11-05T08:53:44Z
system:controller:horizontal-pod-autoscaler 2023-10-25T09:18:50Z
system:controller:pod-garbage-collector 2023-10-25T09:18:50Z
详细查看clusterrole的属性
[root@master ~]# kubectl describe clusterrole pod-reader
Name: pod-reader
Labels: <none>
Annotations: <none>
PolicyRule:
Resources Non-Resource URLs Resource Names Verbs
--------- ----------------- -------------- -----
pods [] [] [get list watch]
创建binding
[root@master ~]# kubectl create clusterrolebinding cluster-pods --clusterrole pod-reader --user jerry --token='6c576631402edca09b93' --dry-run=client -o yaml > rbcluster.yaml
[root@master ~]# cat rbcluster.yaml
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
creationTimestamp: null
name: cluster-pods
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: pod-reader
subjects:
- apiGroup: rbac.authorization.k8s.io
kind: User
name: jerry
[root@master ~]# kubectl apply -f rbcluster.yaml
clusterrolebinding.rbac.authorization.k8s.io/cluster-pods created
[root@master ~]# kubectl get clusterrolebindings.rbac.authorization.k8s.io | grep cluster
cluster-admin ClusterRole/cluster-admin 10d
cluster-pods ClusterRole/pod-reader 12s
[root@master ~]# kubectl describe clusterrolebindings.rbac.authorization.k8s.io cluster-pods
Name: cluster-pods
Labels: <none>
Annotations: <none>
Role:
Kind: ClusterRole
Name: pod-reader
Subjects:
Kind Name Namespace
---- ---- ---------
User jerry
这时候，直接通过jerry用户来查询default命名空间。（测试有问题）
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='6c576631402edca09b93' --insecure-skip-tls-verify=true get pod -n default
No resources found in default namespace.
[root@node2 ~]# alias kubectl='kubectl --server='https://10.1.1.200:6443' --token='6c576631402edca09b93' --insecure-skip-tls-verify=true'
[root@node2 ~]# kubectl get pod -n default
No resources found in default namespace.
[root@node2 ~]# kubectl get pod -n kube-system
NAME READY STATUS RESTARTS AGE
coredns-5bbd96d687-ddrf4 1/1 Running 1 (17m ago) 55d
coredns-5bbd96d687-jgfbv 1/1 Running 1 (17m ago) 55d
etcd-kmaster 1/1 Running 1 (17m ago) 55d
kube-apiserver-kmaster 1/1 Running 0 4m55s
kube-controller-manager-kmaster 1/1 Running 2 (4m57s ago) 55d
kube-proxy-7k2sk 1/1 Running 1 (17m ago) 55d
kube-proxy-dtplw 1/1 Running 1 (17m ago) 55d
kube-proxy-jcwrd 1/1 Running 1 (17m ago) 55d
kube-scheduler-kmaster 1/1 Running 2 (4m57s ago) 55d
其他角色权限和role是一样的。
测试有问题：
[root@node2 ~]# kubectl --server='https://10.1.1.200:6443' --token='6c576631402edca09b93' --insecure-skip-tls-verify=true get pod -n default
E1105 22:03:40.062260 344747 memcache.go:265] couldn't get current server API group list: the server has asked for the client to provide credentials
E1105 22:03:40.063660 344747 memcache.go:265] couldn't get current server API group list: the server has asked for the client to provide credentials
E1105 22:03:40.064864 344747 memcache.go:265] couldn't get current server API group list: the server has asked for the client to provide credentials
E1105 22:03:40.096103 344747 memcache.go:265] couldn't get current server API group list: the server has asked for the client to provide credentials
E1105 22:03:40.097689 344747 memcache.go:265] couldn't get current server API group list: the server has asked for the client to provide credentials
error: You must be logged in to the server (the server has asked for the client to provide credentials)
SA serviceaccount
更新：从1.24版本开始，创建sa不再自动生成secrets
https://github.com/kubernetes/kubernetes/blob/master/CHANGELOG/CHANGELOG-1.24.md#urgent-upgrade-notes
https://kubernetes.io/docs/concepts/configuration/secret/#service-account-token-secrets
RBAC分为2种账户：
user account 用户账号（UA）：用于远程登陆系统，也就是刚实验所使用的账号。
service account 服务账户（SA）：Service account是为了方便Pod里面的进程（而非人工）调用Kubernetes API或其他外部服务而设计的。例如使用web来管理k8s环境时，需要调用相关的程序（pod中）有对K8s管理，当时此程序需要有相关的权限，这时候就可以创建一个SA，赋予相关的权限，并且与管理K8s的环境程序进行绑定，那么这个程序就有了SA的对应权限，可以对K8s进行管理。
在k8s集中群，用户分为 useraccount和serviceaccount，前者用于做登录限制授权使用，后者用于给应用程序授权，不会让用户直接登录。
k8s集群中很多组件都是基于pod来运行的。比如，需要某个pod里面的应用程序来监控或操作集群资源，自动扩展、缩容，部署新的pod。
但是pod里面运行的是某个进程，该进程要去管理集群里面的资源，是否有权限吗？
这个时候，就需要通过授权指定pod里面的进程，要以哪个sa（服务账号）来运行。该sa是否有足够的权限，如果没有，需要对sa进行授权
举例：创建一个pod，假设该pod里面的进程未来要管理集群资源。
[root@master ~]# vim web1.yaml
[root@master ~]# cat web1.yaml
apiVersion: apps/v1
kind: Deployment
metadata:
creationTimestamp: null
labels:
app: web1
name: web1
spec:
replicas: 2
selector:
matchLabels:
app: web1
strategy: {}
template:
metadata:
creationTimestamp: null
labels:
app: web1
spec:
containers:
- image: nginx
name: nginx
imagePullPolicy: IfNotPresent
resources: {}
status: {}
[root@master ~]# kubectl apply -f web1.yaml
deployment.apps/web1 created
[root@master ~]# kubectl get pod
NAME READY STATUS RESTARTS AGE
web1-58d8bd576b-djctr 1/1 Running 0 8s
web1-58d8bd576b-s9cdn 1/1 Running 0 8s
[root@master ~]# kubectl get sa
NAME SECRETS AGE
default 0 11d
[root@master ~]# kubectl create sa sa1
serviceaccount/sa1 created
[root@master ~]# kubectl get sa
NAME SECRETS AGE
default 0 11d
sa1 0 3s
此时的sa1是没有任何权限的。将集群管理员角色授权binding给sa1
[root@master ~]# kubectl create clusterrolebinding rbsa1 --clusterrole admin --serviceaccount default:sa1
clusterrolebinding.rbac.authorization.k8s.io/rbsa1 created
将服务账号，给到deploy pod（里面的进程）。查看下当前deploy里面的内容：
[root@master ~]# kubectl edit deployments.apps web1
securityContext: {}
把sa1账号绑定给deploy
[root@master ~]# kubectl set serviceaccount deploy web1 sa1
deployment.apps/web1 serviceaccount updated
再次查看当前deploy里面的内容：
[root@master ~]# kubectl edit deployments.apps web1
securityContext: {}
serviceAccount: sa1
serviceAccountName: sa1
这个时候，deploy里面的pod就具备了admin的权限了。
查询sa和关联的角色
kubectl get rolebindings,clusterrolebindings \
--all-namespaces \
-o custom-columns='KIND:kind,NAMESPACE:metadata.namespace,NAME:metadata.name,SERVICE_ACCOUNTS:subjects[?(@.kind=="ServiceAccount")].name'
3、图形化管理
官方图形化管理工具：dashboard
https://kubernetes.io/zh-cn/docs/tasks/access-application-cluster/web-ui-dashboard/
kubectl apply -f https://raw.githubusercontent.com/kubernetes/dashboard/v2.7.0/aio/deploy/recommended.yaml
网络原因，先把文件拷贝过来，将代码复制重命名上传。
创建SA
[root@master ~]# kubectl create sa sa2 -n kube-system
serviceaccount/sa2 created
[root@master ~]# kubectl create clusterrolebinding rbsa2 --clusterrole admin --serviceaccount kube-system:sa2 -n kube-system
clusterrolebinding.rbac.authorization.k8s.io/rbsa2 created
[root@master ~]# kubectl get sa -n kube-system
NAME SECRETS AGE
sa2 0 44s
创建secrets
[root@master ~]# vim sa2-secret.yaml
[root@master ~]# cat sa2-secret.yaml
apiVersion: v1
kind: Secret
metadata:
name: secret-sa-sample
annotations:
kubernetes.io/service-account.name: "sa2"
type: kubernetes.io/service-account-token
[root@master ~]# kubectl apply -f sa2-secret.yaml -n kube-system
secret/secret-sa-sample created
为了方便操作，切换到kube-system
[root@master ~]# kubens kube-system
Context "kubernetes-admin@kubernetes" modified.
Active namespace is "kube-system".
[root@master ~]# kubectl get secrets
NAME TYPE DATA AGE
secret-sa-sample kubernetes.io/service-account-token 3 57s
[root@master ~]# kubectl describe secrets secret-sa-sample
Name: secret-sa-sample
Namespace: kube-system
Labels: <none>
Annotations: kubernetes.io/service-account.name: sa2
kubernetes.io/service-account.uid: 66a35af6-2586-4a04-99f2-d44a46842fec
Type: kubernetes.io/service-account-token
Data
====
ca.crt: 1099 bytes
namespace: 11 bytes
token: eyJhbGciOiJSUzI1NiIsImtpZCI6ImR2WG81dVQwNVpYcmJSbURERU1pUU1ZSTFHakh6YmlMeXZ5Y1FSNWZIcW8ifQ.eyJpc3MiOiJrdWJlcm5ldGVzL3NlcnZpY2VhY2NvdW50Iiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9uYW1lc3BhY2UiOiJrdWJlLXN5c3RlbSIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VjcmV0Lm5hbWUiOiJzZWNyZXQtc2Etc2FtcGxlIiwia3ViZXJuZXRlcy5pby9zZXJ2aWNlYWNjb3VudC9zZXJ2aWNlLWFjY291bnQubmFtZSI6InNhMiIsImt1YmVybmV0ZXMuaW8vc2VydmljZWFjY291bnQvc2VydmljZS1hY2NvdW50LnVpZCI6IjY2YTM1YWY2LTI1ODYtNGEwNC05OWYyLWQ0NGE0Njg0MmZlYyIsInN1YiI6InN5c3RlbTpzZXJ2aWNlYWNjb3VudDprdWJlLXN5c3RlbTpzYTIifQ.BlN1WaAli40bdl0lRCOSo3b2Cf3gaGXabEKQhVFgBFGc8RYqguX9xyMOzqhDYasXNXYTnka0vLHBvK1TM4nuJg3MskYKgoRogNtfyebFa9OjcGsCBH_yA3IWUqbXWxAkl1a4yHnjh2-Ck1FxlBcUCaCxJw-2w56iANGBv95hS5mdtAgslbt3LIr25N3vGTdOfpwWsh_CWcZQRz8xTGowUpqO1dWR9qzhYIWT7405soqHW8DbJr5A9JsMZsEAF94WCG32CU4UphkxTpWNv_LOGkZNyO1XBcCImxya4YfT6LYtH-LH-w7n0On3-7-yEBy4dJffuvvEPu8LrEW5stY8tQ
这个token就是admin角色的值。后面登入要用到的。
开始安装dashboard
https://kubernetes.io/docs/tasks/access-application-cluster/web-ui-dashboard/
[root@master ~]# vim recommended.yaml
[root@master ~]# grep image recommended.yaml
image: kubernetesui/dashboard:v2.7.0
imagePullPolicy: Always
image: kubernetesui/metrics-scraper:v1.0.8
修改文件更换成阿里云的镜像地址
registry.cn-hangzhou.aliyuncs.com/cloudcs/dashboard:v2.7.0
registry.cn-hangzhou.aliyuncs.com/cloudcs/metrics-scraper:v1.0.8
[root@master ~]# grep image recommended.yaml
image: registry.cn-hangzhou.aliyuncs.com/cloudcs/dashboard:v2.7.0
imagePullPolicy: IfNotPresent
image: registry.cn-hangzhou.aliyuncs.com/cloudcs/metrics-scraper:v1.0.8
imagePullPolicy: IfNotPresent
[root@master ~]# kubectl apply -f recommended.yaml
[root@master ~]# kubectl get ns 发现多了个kubernetes-dashboard命名空间
NAME STATUS AGE
calico-apiserver Active 55d
calico-system Active 55d
default Active 55d
kube-node-lease Active 55d
kube-public Active 55d
kube-system Active 55d
kubernetes-dashboard Active 35s
tigera-operator Active 55d
[root@master ~]# kubectl get pods -n kubernetes-dashboard
NAME READY STATUS RESTARTS AGE
dashboard-metrics-scraper-c848b4fb5-gpnrx 1/1 Running 0 2m57s
kubernetes-dashboard-76bc5cbb8f-lj85t 1/1 Running 0 2m57s
[root@master ~]# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.101.141.223 <none> 8000/TCP 3m25s
kubernetes-dashboard ClusterIP 10.106.44.184 <none> 443/TCP 3m25s
[root@master ~]# kubectl edit svc kubernetes-dashboard -n kubernetes-dashboard
service/kubernetes-dashboard edited
sessionAffinity: None
type: NodePort
[root@master ~]# kubectl get svc -n kubernetes-dashboard
NAME TYPE CLUSTER-IP EXTERNAL-IP PORT(S) AGE
dashboard-metrics-scraper ClusterIP 10.101.141.223 <none> 8000/TCP 5m12s
kubernetes-dashboard NodePort 10.106.44.184 <none> 443:30360/TCP 5m12s
访问集群，集群内任一节点ip加端口（注意https）
https://10.1.1.200:30360/#/login
可以选择config方式登入，这里选择token方式登录（用上面生成的token）。
关于token如何创建，也可以参考官方链接：
https://github.com/kubernetes/dashboard/blob/master/docs/user/access-control/creating-sample-user.md
类似于其他的三方工具：
https://www.rancher.com/
https://github.com/kubesphere/kubesphere/blob/master/README_zh.md
K8S/Kubernetes社区为您提供最前沿的新闻资讯和知识内容
更多推荐
0
0- 0
已为社区贡献3条内容
所有评论(0)