Trying to install Crunchydata postgres-operator.
My pgo-deploy pod is failing with error.
I have setup default nfs storage running the following commands,
# kubectl create -f rbac.yaml
the content is,
apiVersion: v1
kind: ServiceAccount
metadata:
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: pgo
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: nfs-client-provisioner-runner
rules:
- apiGroups: [""]
resources: ["nodes"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["persistentvolumes"]
verbs: ["get", "list", "watch", "create", "delete"]
- apiGroups: [""]
resources: ["persistentvolumeclaims"]
verbs: ["get", "list", "watch", "update"]
- apiGroups: ["storage.k8s.io"]
resources: ["storageclasses"]
verbs: ["get", "list", "watch"]
- apiGroups: [""]
resources: ["events"]
verbs: ["create", "update", "patch"]
---
kind: ClusterRoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: run-nfs-client-provisioner
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: pgo
roleRef:
kind: ClusterRole
name: nfs-client-provisioner-runner
apiGroup: rbac.authorization.k8s.io
---
kind: Role
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: pgo
rules:
- apiGroups: [""]
resources: ["endpoints"]
verbs: ["get", "list", "watch", "create", "update", "patch"]
---
kind: RoleBinding
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: leader-locking-nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: pgo
subjects:
- kind: ServiceAccount
name: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: pgo
roleRef:
kind: Role
name: leader-locking-nfs-client-provisioner
apiGroup: rbac.authorization.k8s.io
# kubectl create -f class.yaml
the content:
apiVersion: storage.k8s.io/v1
kind: StorageClass
metadata:
name: managed-nfs-storage
annotations:
storageclass.kubernetes.io/is-default-class: "true"
provisioner: k8s-sigs.io/nfs-subdir-external-provisioner # or choose another name, must match deployment's env PROVISIONER_NAME'
parameters:
archiveOnDelete: "false"
# kubectl create -f deployment.yaml
the content:
apiVersion: apps/v1
kind: Deployment
metadata:
name: nfs-client-provisioner
labels:
app: nfs-client-provisioner
# replace with namespace where provisioner is deployed
namespace: pgo
spec:
replicas: 1
strategy:
type: Recreate
selector:
matchLabels:
app: nfs-client-provisioner
template:
metadata:
labels:
app: nfs-client-provisioner
spec:
serviceAccountName: nfs-client-provisioner
containers:
- name: nfs-client-provisioner
image: k8s.gcr.io/sig-storage/nfs-subdir-external-provisioner:v4.0.2
volumeMounts:
- name: nfs-client-root
mountPath: /persistentvolumes
env:
- name: PROVISIONER_NAME
value: k8s-sigs.io/nfs-subdir-external-provisioner
- name: NFS_SERVER
value: 192.168.10.114
- name: NFS_PATH
value: /var/nfs/general
volumes:
- name: nfs-client-root
nfs:
server: 192.168.10.114
path: /var/nfs/general
Now when I apply # kubectl apply -f postgres-operator.yml with my configuration:
apiVersion: v1
kind: ServiceAccount
metadata:
name: pgo-deployer-sa
namespace: pgo
---
kind: ClusterRole
apiVersion: rbac.authorization.k8s.io/v1
metadata:
name: pgo-deployer-cr
rules:
- apiGroups:
- ''
resources:
- namespaces
verbs:
- get
- list
- create
- patch
- delete
- apiGroups:
- ''
resources:
- pods
verbs:
- list
- apiGroups:
- ''
resources:
- secrets
verbs:
- list
- get
- create
- delete
- apiGroups:
- ''
resources:
- configmaps
- services
- persistentvolumeclaims
verbs:
- get
- create
- delete
- list
- apiGroups:
- ''
resources:
- serviceaccounts
verbs:
- get
- create
- delete
- patch
- list
- apiGroups:
- apps
- extensions
resources:
- deployments
- replicasets
verbs:
- get
- list
- watch
- create
- delete
- apiGroups:
- apiextensions.k8s.io
resources:
- customresourcedefinitions
verbs:
- get
- create
- delete
- apiGroups:
- rbac.authorization.k8s.io
resources:
- clusterroles
- clusterrolebindings
- roles
- rolebindings
verbs:
- get
- create
- delete
- bind
- escalate
- apiGroups:
- rbac.authorization.k8s.io
resources:
- roles
verbs:
- create
- delete
- apiGroups:
- batch
resources:
- jobs
verbs:
- delete
- list
- apiGroups:
- crunchydata.com
resources:
- pgclusters
- pgreplicas
- pgpolicies
- pgtasks
verbs:
- delete
- list
---
apiVersion: v1
kind: ConfigMap
metadata:
name: pgo-deployer-cm
namespace: pgo
data:
values.yaml: |-
# =====================
# Configuration Options
# More info for these options can be found in the docs
# https://access.crunchydata.com/documentation/postgres-operator/latest/installation/configuration/
# =====================
archive_mode: "true"
archive_timeout: "60"
backrest_aws_s3_bucket: ""
backrest_aws_s3_endpoint: ""
backrest_aws_s3_key: ""
backrest_aws_s3_region: ""
backrest_aws_s3_secret: ""
backrest_aws_s3_uri_style: ""
backrest_aws_s3_verify_tls: "true"
backrest_gcs_bucket: ""
backrest_gcs_endpoint: ""
backrest_gcs_key_type: ""
backrest_port: "2022"
badger: "false"
ccp_image_prefix: "registry.developers.crunchydata.com/crunchydata"
ccp_image_pull_secret: ""
ccp_image_pull_secret_manifest: ""
ccp_image_tag: "centos8-13.3-4.7.0"
create_rbac: "true"
crunchy_debug: "false"
db_name: ""
db_password_age_days: "0"
db_password_length: "24"
db_port: "5432"
db_replicas: "0"
db_user: "testuser"
default_instance_memory: "128Mi"
default_pgbackrest_memory: "48Mi"
default_pgbouncer_memory: "24Mi"
default_exporter_memory: "24Mi"
delete_operator_namespace: "false"
delete_watched_namespaces: "false"
disable_auto_failover: "false"
disable_fsgroup: "false"
reconcile_rbac: "true"
exporterport: "9187"
metrics: "false"
namespace: "pgo"
namespace_mode: "dynamic"
pgbadgerport: "10000"
pgo_add_os_ca_store: "false"
pgo_admin_password: "examplepassword"
pgo_admin_perms: "*"
pgo_admin_role_name: "pgoadmin"
pgo_admin_username: "admin"
pgo_apiserver_port: "8443"
pgo_apiserver_url: "https://postgres-operator"
pgo_client_cert_secret: "pgo.tls"
pgo_client_container_install: "false"
pgo_client_install: "true"
pgo_client_version: "4.7.0"
pgo_cluster_admin: "false"
pgo_disable_eventing: "false"
pgo_disable_tls: "false"
pgo_image_prefix: "registry.developers.crunchydata.com/crunchydata"
pgo_image_pull_secret: ""
pgo_image_pull_secret_manifest: ""
pgo_image_tag: "centos8-4.7.0"
pgo_installation_name: "devtest"
pgo_noauth_routes: ""
pgo_operator_namespace: "pgo"
pgo_tls_ca_store: ""
pgo_tls_no_verify: "false"
pod_anti_affinity: "preferred"
pod_anti_affinity_pgbackrest: ""
pod_anti_affinity_pgbouncer: ""
scheduler_timeout: "3600"
service_type: "ClusterIP"
sync_replication: "false"
backrest_storage: "nfsstorage"
backup_storage: "nfsstorage"
primary_storage: "nfsstorage"
replica_storage: "nfsstorage"
pgadmin_storage: "nfsstorage"
wal_storage: ""
storage1_name: "default"
storage1_access_mode: "ReadWriteOnce"
storage1_size: "1G"
storage1_type: "dynamic"
storage2_name: "hostpathstorage"
storage2_access_mode: "ReadWriteMany"
storage2_size: "1G"
storage2_type: "create"
storage3_name: "nfsstorage"
storage3_access_mode: "ReadWriteMany"
storage3_size: "10Gi"
storage3_type: "create"
storage3_supplemental_groups: "65534"
storage4_name: "nfsstoragered"
storage4_access_mode: "ReadWriteMany"
storage4_size: "1G"
storage4_match_labels: "crunchyzone=red"
storage4_type: "create"
storage4_supplemental_groups: "65534"
storage5_name: "storageos"
storage5_access_mode: "ReadWriteOnce"
storage5_size: "5Gi"
storage5_type: "dynamic"
storage5_class: "fast"
storage6_name: "primarysite"
storage6_access_mode: "ReadWriteOnce"
storage6_size: "4G"
storage6_type: "dynamic"
storage6_class: "primarysite"
storage7_name: "alternatesite"
storage7_access_mode: "ReadWriteOnce"
storage7_size: "4G"
storage7_type: "dynamic"
storage7_class: "alternatesite"
storage8_name: "gce"
storage8_access_mode: "ReadWriteOnce"
storage8_size: "300M"
storage8_type: "dynamic"
storage8_class: "standard"
storage9_name: "rook"
storage9_access_mode: "ReadWriteOnce"
storage9_size: "1Gi"
storage9_type: "dynamic"
storage9_class: "rook-ceph-block"
---
apiVersion: rbac.authorization.k8s.io/v1
kind: ClusterRoleBinding
metadata:
name: pgo-deployer-crb
roleRef:
apiGroup: rbac.authorization.k8s.io
kind: ClusterRole
name: pgo-deployer-cr
subjects:
- kind: ServiceAccount
name: pgo-deployer-sa
namespace: pgo
---
apiVersion: batch/v1
kind: Job
metadata:
name: pgo-deploy
namespace: pgo
spec:
backoffLimit: 0
template:
metadata:
name: pgo-deploy
spec:
serviceAccountName: pgo-deployer-sa
restartPolicy: Never
containers:
- name: pgo-deploy
image: registry.developers.crunchydata.com/crunchydata/pgo-deployer:centos8-4.7.0
imagePullPolicy: IfNotPresent
env:
- name: DEPLOY_ACTION
value: install
volumeMounts:
- name: deployer-conf
mountPath: "/conf"
volumes:
- name: deployer-conf
configMap:
name: pgo-deployer-cm
I get the following error:
# kubectl get pods -n pgo
NAME READY STATUS RESTARTS AGE
nfs-client-provisioner-7d485f5b8d-cnt57 1/1 Running 0 28m
pgo-deploy--1-ppzkw 0/1 Error 0 10m
# kubectl describe pod -n pgo pgo-deploy--1-ppzkw returns the following error:
Events:
Type Reason Age From Message
---- ------ ---- ---- -------
Normal Scheduled 9m13s default-scheduler Successfully assigned pgo/pgo-deploy--1-ppzkw to dfsworker1
Normal Pulled 9m11s kubelet Container image "registry.developers.crunchydata.com/crunchydata/pgo-deployer:centos8-4.7.1" already present on machine
Normal Created 9m10s kubelet Created container pgo-deploy
Normal Started 9m10s kubelet Started container pgo-deploy
Warning FailedMount 8m58s (x3 over 9m) kubelet MountVolume.SetUp failed for volume "deployer-conf" : object "pgo"/"pgo-deployer-cm" not registered
even tried with # kubectl apply -f https://raw.githubusercontent.com/CrunchyData/postgres-operator/v4.7.1/installers/kubectl/postgres-operator.yml
it gives the same error. # kubectl -n pgo logs -f pgo-deploy--1-ppzkw gives the following error:
TASK [pgo-operator : Create PGClusters CRD] ************************************
fatal: [localhost]: FAILED! => {"changed": true, "cmd": ["kubectl", "create", "-f", "/ansible/postgres-operator/roles/pgo-operator/files/crds/pgclusters-crd.yaml"], "delta": "0:00:02.599141", "end": "2021-08-09 08:24:50.295545", "msg": "non-zero return code", "rc": 1, "start": "2021-08-09 08:24:47.696404", "stderr": "error: unable to recognize \"/ansible/postgres-operator/roles/pgo-operator/files/crds/pgclusters-crd.yaml\": no matches for kind \"CustomResourceDefinition\" in version \"apiextensions.k8s.io/v1beta1\"", "stderr_lines": ["error: unable to recognize \"/ansible/postgres-operator/roles/pgo-operator/files/crds/pgclusters-crd.yaml\": no matches for kind \"CustomResourceDefinition\" in version \"apiextensions.k8s.io/v1beta1\""], "stdout": "", "stdout_lines": []}
PLAY RECAP *********************************************************************
localhost : ok=21 changed=5 unreachable=0 failed=1 skipped=17 rescued=0 ignored=0
Can anyone help me to solve this? All my machines are ubuntu 20.04. It was all working with the same configurations and steps a few days ago until I deleted the pgo namespace and followed all my past procedures.
My kubernetes version: v1.22.0.
The error you provided says what is wrong:
error: unable to recognize \"/ansible/postgres-operator/roles/pgo-operator/files/crds/pgclusters-crd.yaml\": no matches for kind \"CustomResourceDefinition\" in version \"apiextensions.k8s.io/v1beta1\"
CustomResourceDefinition is no longer in beta API:
kubectl explain CustomResourceDefinition
KIND: CustomResourceDefinition
VERSION: apiextensions.k8s.io/v1
Ideally, the editor in charge of that operator already ships with some up-to-date CustomResourceDefinitions. In your case, the last copy seems to be available over here.
Though if your CRD is outdated: there may be other changes you would want to pull out of Crunchy latest release.
Otherwise, we may consider rewriting those objects ourselves:
apiVersion to apiextensions.k8s.io/v1spec complying with the last schemaspec.additionalPrinterColumns, spec.subresources or spec.validation would need to move into a spec.versions array. You no longer have to define a schema for your resources metadata - if you did configure a schema in your CRD.
The new layout would look something like this:
apiVersion: apiextensions.k8s.io/v1
kind: CustomResourceDefinition
metadata:
name: crname.api-group
spec:
group: api-group
names:
kind: CrName
listKind: CrNameList
plural: crnames
singular: crname
scope: Namespaced
versions:
- name: v1
additionalPrinterColumns:
- name: Age
type: date
jsonPath: .metadata.creationTimestamp
schema:
openAPIV3Schema:
properties:
apiVersion:
type: string
kind:
type: string
spec:
properties:
[...]
type: object
served: true
storage: true
subresources:
status: {}
- name: v1beta1
[...]