Kubernetes 1.29使用群辉dsm7 NFS存储创建StorageClass持久化

目前单机创建NFS服务器高可用数据不好实现，使用CEPH数据存储又可能比较麻烦，还需要额外维护单独CEPH集群，本次采用群辉NFS存储进行模拟测试

群辉配置

登录群辉，需要搭建可以参考下面的文章

开启NFS Server服务

创建共享目录，打开控制面板

创建共享目录

设置目录名称nfs-k8s

按照下面的配置，设置NFS Server权限

开启File Station 匿名用户访问权限

挂载目录可以在共享目录中看到

Kubernetes 存储卷创建

• 所有k8s节点安装nfs (我这里以Ubuntu为例)

1. sudo apt update
2. sudo apt install nfs-kernel-server

执行showmount测试nfs挂载情况

1. root@k8s-master-01:~# showmount -e 192.168.21.27
2. Export list for 192.168.21.27:
3. /volume1/nfs-k8s *
4. root@k8s-master-01:~#

接下来为nfs创建rbac权限

1. apiVersion: v1
2. kind: ServiceAccount
3. metadata:
4.   name: nfs-client-provisioner
5.   # replace with namespace where provisioner is deployed
6.   namespace: default
7. ---
8. kind: ClusterRole
9. apiVersion: rbac.authorization.k8s.io/v1
10. metadata:
11.   name: nfs-client-provisioner-runner
12. rules:
13.   - apiGroups: [""]
14.     resources: ["nodes"]
15.     verbs: ["get", "list", "watch"]
16.   - apiGroups: [""]
17.     resources: ["persistentvolumes"]
18.     verbs: ["get", "list", "watch", "create", "delete"]
19.   - apiGroups: [""]
20.     resources: ["persistentvolumeclaims"]
21.     verbs: ["get", "list", "watch", "update"]
22.   - apiGroups: ["storage.k8s.io"]
23.     resources: ["storageclasses"]
24.     verbs: ["get", "list", "watch"]
25.   - apiGroups: [""]
26.     resources: ["events"]
27.     verbs: ["create", "update", "patch"]
28. ---
29. kind: ClusterRoleBinding
30. apiVersion: rbac.authorization.k8s.io/v1
31. metadata:
32.   name: run-nfs-client-provisioner
33. subjects:
34.   - kind: ServiceAccount
35.     name: nfs-client-provisioner
36.     # replace with namespace where provisioner is deployed
37.     namespace: default
38. roleRef:
39.   kind: ClusterRole
40.   name: nfs-client-provisioner-runner
41.   apiGroup: rbac.authorization.k8s.io
42. ---
43. kind: Role
44. apiVersion: rbac.authorization.k8s.io/v1
45. metadata:
46.   name: leader-locking-nfs-client-provisioner
47.   # replace with namespace where provisioner is deployed
48.   namespace: default
49. rules:
50.   - apiGroups: [""]
51.     resources: ["endpoints"]
52.     verbs: ["get", "list", "watch", "create", "update", "patch"]
53. ---
54. kind: RoleBinding
55. apiVersion: rbac.authorization.k8s.io/v1
56. metadata:
57.   name: leader-locking-nfs-client-provisioner
58.   # replace with namespace where provisioner is deployed
59.   namespace: default
60. subjects:
61.   - kind: ServiceAccount
62.     name: nfs-client-provisioner
63.     # replace with namespace where provisioner is deployed
64.     namespace: default
65. roleRef:
66.   kind: Role
67.   name: leader-locking-nfs-client-provisioner
68.   apiGroup: rbac.authorization.k8s.io

创建Deployment

1. apiVersion: apps/v1
2. kind: Deployment
3. metadata:
4.   name: nfs-client-provisioner
5.   labels:
6.     app: nfs-client-provisioner
7.   # replace with namespace where provisioner is deployed
8.   namespace: default
9. spec:
10.   replicas: 1
11.   strategy:
12.     type: Recreate
13.   selector:
14.     matchLabels:
15.       app: nfs-client-provisioner
16.   template:
17.     metadata:
18.       labels:
19.         app: nfs-client-provisioner
20.     spec:
21.       serviceAccountName: nfs-client-provisioner
22.       containers:
23.         - name: nfs-client-provisioner
24.           image: dockerproxy.frps.fun/dyrnq/nfs-subdir-external-provisioner:v4.0.1
25.           volumeMounts:
26.             - name: nfs-client-root
27.               mountPath: /persistentvolumes
28.           env:
29.             - name: PROVISIONER_NAME
30.               value: k8s-sigs.io/nfs-subdir-external-provisioner
31.             - name: NFS_SERVER
32.               value: 192.168.21.27
33.             - name: NFS_PATH
34.               value: /volume1/nfs-k8s
35.       volumes:
36.         - name: nfs-client-root
37.           nfs:
38.             server: 192.168.21.27
39.             path: /volume1/nfs-k8s

我这里提供下面镜像
dockerproxy.frps.fun/dyrnq/nfs-subdir-external-provisioner:v4.0.1
官方默认镜像地址为dyrnq/nfs-subdir-external-provisioner:v4.0.1

这里是创建StorageClass

1. apiVersion: storage.k8s.io/v1
2. kind: StorageClass
3. metadata:
4.   name: dsm7-nfs-client
5. provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
6. parameters:
7.   archiveOnDelete: "false"

查看所有已创建的资源

1. root@k8s-master-01:~/nfs# kubectl get pod,sa,sc
2. NAME READY STATUS RESTARTS AGE
3. pod/nfs-client-provisioner-594f79d67b-wvx5s 1/1 Running 0 23m
5. NAME SECRETS AGE
6. serviceaccount/default 0 22h
7. serviceaccount/nfs-client-provisioner 0 26m
9. NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
10. storageclass.storage.k8s.io/dsm7-nfs-client k8s-sigs.io/nfs-subdir-external-provisioner Delete Immediate false 13s

将dsm7-nfs-client设置为默认storageclass

1. root@k8s-master-01:~/nfs# kubectl patch storageclass dsm7-nfs-client -p '{"metadata": {"annotations":{"storageclass.kubernetes.io/is-default-class":"true"}'

查看设置default状态

1. root@k8s-master-01:~/nfs# kubectl get sc
2. NAME PROVISIONER RECLAIMPOLICY VOLUMEBINDINGMODE ALLOWVOLUMEEXPANSION AGE
3. dsm7-nfs-client (default) k8s-sigs.io/nfs-subdir-external-provisioner Delete Immediate false 5m

测试pvc如下

1. apiVersion: v1
2. kind: PersistentVolumeClaim
3. metadata:
4. namespace: default
5. name: abcdocker-test-pvc
6. labels: {}
7. spec:
8. accessModes:
9. - ReadWriteMany
10. resources:
11. requests:
12. storage: 10Gi
13. storageClassName: dsm7-nfs-client

这里提示已绑定，代表PVC验证完毕

1. root@k8s-master-01:~/nfs# kubectl apply -f test-pvc.yaml
2. persistentvolumeclaim/abcdocker-test-pvc created
3. root@k8s-master-01:~/nfs# kubectl get pvc
4. NAME STATUS VOLUME CAPACITY ACCESS MODES STORAGECLASS VOLUMEATTRIBUTESCLASS AGE
5. abcdocker-test-pvc Bound pvc-ba81924e-3d04-4173-9abf-d17972f6fa5b 10Gi RWX dsm7-nfs-client <unset> 5s