i try to run a mongodb within a kubernetes cluster secured with a keyFile. For this, i created a simple statefulset and a configmap, where i stored the keyfile:
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: mongodb
spec:
serviceName: mongodb
replicas: 1
selector:
matchLabels:
app: mongodb
template:
metadata:
labels:
app: mongodb
spec:
containers:
- name: mongodb
image: mongo:4.4
args:
- --bind_ip
- '0.0.0.0,::'
- --replSet
- MySetname01
- --auth
- --keyFile
- /etc/mongodb/keyfile/keyfile
env:
- name: MONGO_INITDB_ROOT_USERNAME
value: MyUsername
- name: MONGO_INITDB_ROOT_PASSWORD
value: MyPassword
ports:
- containerPort: 27017
name: mongodb
volumeMounts:
- name: mongodb-persistent-storage
mountPath: /data/db
- name: mongodb-keyfile
mountPath: /etc/mongodb/keyfile
readOnly: True
volumes:
- name: mongodb-keyfile
configMap:
name: mongodb-keyfile
volumeClaimTemplates:
- metadata:
name: mongodb-persistent-storage
spec:
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 1Gi
---
apiVersion: v1
kind: Service
metadata:
name: mongodb
labels:
app: mongodb
spec:
ports:
- port: 27017
selector:
app: mongodb
---
apiVersion: v1
kind: ConfigMap
metadata:
name: mongodb-keyfile
data:
keyfile: |
+PN6gXEU8NeRsyjlWDnTesHCoPOn6uQIEI5pNorDkphREi6RyoSHCIaXOzLrUpPq
jpSGhSc5/MZj17R7K5anjerhvR6f5JtWjBuQcrjdJdNBceck71F2bly/u9ICfCOy
STFzv6foMQJBJTBYqLwtfyEO7CQ9ywodM0K5r9jtT7x5BiJaqso+F8VN/VFtIYpe
vnzKj7uU3GwDbmw6Yduybgv6P88BGXyW3w6HG8VLMgud5aV7wxIIPE6nAcr2nYmM
1BqC7wp8G6uCcMiHx5pONPA5ONYAIF+u3zj2wAthgMe2UeQxx2L2ERx8Zdsa9HLR
qYOmy9XhfolwdCTwwYvqYRO+RqXGoPGczenC/CKJPj14yfkua+0My5NBWvpL/fIB
osu0lQNw1vFu0rcT1/9OcaJHuwFWocec2qBih9tk2C3c7jNMuxkPo3dxjv8J/lex
vN3Et6tK/wDsQo2+8j6uLYkPFQbHZJQzf/oQiekV4RaC6/pejAf9fSAo4zbQXh29
8BIMpRL3fik+hvamjrtS/45yfqGf/Q5DQ7o8foI4HYmhy+SU2+Bxyc0ZLTn659zl
myesNjB6uC9lMWtpjas0XphNy8GvJxfjvz+bckccPUVczxyC3QSEIcVMMH9vhzes
AcQscswhFMgzp1Z0fbNKy0FqQiDy1hUSir06ZZ3xBGLKeIySRsw9D1Pyh1Y11HlH
NdGwF14cLqm53TGVd9gYeIAm2siQYMKm8rEjxmecc3yGgn0B69gtMcBmxr+z3xMU
X256om6l8L2BJjm3W1zUTiZABuKzeNKjhmXQdEFPQvxhubvCinTYs68XL76ZdVdJ
Q909MmllkOXKbAhi/TMdWmpV9nhINUCBrnu3F08jAQ3UkmVb923XZBzcbdPlpuHe
Orp11/f3Dke4x0niqATccidRHf6Hz+ufVkwIrucBZwcHhK4SBY/RU90n233nV06t
JXlBl/4XjWifB7iJi9mxy/66k
Problem is: MongoDb stays in a Crashloopbackoff , because the permissions on the keyfile are too open:
{"t":{"$date":"2022-12-19T12:41:41.399+00:00"},"s":"I", "c":"CONTROL", "id":23285, "ctx":"main","msg":"Automatically disabling TLS 1.0, to force-enable TLS 1.0 specify --sslDisabledProtocols 'none'"}
{"t":{"$date":"2022-12-19T12:41:41.402+00:00"},"s":"I", "c":"NETWORK", "id":4648601, "ctx":"main","msg":"Implicit TCP FastOpen unavailable. If TCP FastOpen is required, set tcpFastOpenServer, tcpFastOpenClient, and tcpFastOpenQueueSize."}
{"t":{"$date":"2022-12-19T12:41:41.402+00:00"},"s":"I", "c":"ACCESS", "id":20254, "ctx":"main","msg":"Read security file failed","attr":{"error":{"code":30,"codeName":"InvalidPath","errmsg":"permissions on /etc/mongodb/keyfile/keyfile are too open"}}}
For what i dont have a explanation.
I already set the volumemount of the configmap on readonly (You see in the mongo statefulset). Also i tried around with commands or lifecyclehooks to chmod 600/400 the file. I tried differnt versions of mongodb, but got always the same error. For sure i also tried if the configmap is included correctly, it is (I uncommented the args and Username/Password for that one).
Permissions are shown:
lrwxrwxrwx 1 root root 14 Dec 19 12:50 keyfile -> ..data/keyfile
Maybe its related to that fact that the file is shown as linked?
I expect a kubernetes yaml which is able to start with a keyfile. Thank you very much.
EDIT: I tried to mount the file directly, not as a link with subpath. Now i got the following permissions:
-rw-r--r-- 1 root root 1001 Dec 19 13:34 mongod.key
But sadly the db will not start with that one too, it's still crashing with the same error.
EDIT2:
Adding defaultMode: 0600 to the volume in the statefulset led at least to the correct permissions, but also another error (already mentioned in one of my comments):
file: /var/lib/mongo/mongod.key: bad file"
So i tried to mount on different places in the Pod (You see here /var/lib/) for example and i tried to include the keyfile as secret. But none is working.
this works for me:
ConfigMap
---
apiVersion: v1
kind: ConfigMap
metadata:
name: common-mongodb
data:
keyfile: |
<hash>
StatefulSet
---
apiVersion: apps/v1
kind: StatefulSet
metadata:
name: common-mongodb
namespace: default
spec:
selector:
matchLabels:
app: common-mongodb
serviceName: "common-mongodb"
podManagementPolicy: Parallel
replicas: 1
template:
metadata:
labels:
app: common-mongodb
spec:
terminationGracePeriodSeconds: 10
containers:
- name: common-mongodb
image: mongo
imagePullPolicy: IfNotPresent
args:
- --bind_ip
- '0.0.0.0,::'
- --replSet
- rpl0
- --auth
- --keyFile
- /etc/mongo/keyfile
env:
- name: MONGO_INITDB_ROOT_USERNAME
value: user
- name: MONGO_INITDB_ROOT_PASSWORD
value: password
ports:
- containerPort: 27017
name: common-mongodb
volumeMounts:
- name: common-mongodb-data
mountPath: /etc/mongo
initContainers:
- name: change-ownership-container
image: busybox
imagePullPolicy: IfNotPresent
command: ["/bin/sh"]
args:
- -c
- >-
/bin/cp /tmp/keyfile /etc/mongo/keyfile &&
/bin/chown 999:999 /etc/mongo/keyfile &&
/bin/chmod 400 /etc/mongo/keyfile
volumeMounts:
- name: common-mongodb
mountPath: /tmp/
- name: common-mongodb-data
mountPath: /etc/mongo
volumes:
- name: common-mongodb
defaultMode: 0400
configMap:
name: common-mongodb
volumeClaimTemplates:
- metadata:
name: common-mongodb-data
namespace: default
spec:
storageClassName: rook-ceph-block
accessModes: [ "ReadWriteOnce" ]
resources:
requests:
storage: 50Gi
Maybe not the best solution, but it works