I am trying to run a workflow (https://github.com/argoproj/argo-workflows/blob/master/examples/memoize-simple.yaml) with limited permissions in Argo. I am specifying a serviceaccount with the requisite permissions in the execution command and in the workflow itself, but the workflow controller logs show a different serviceaccount.
This is the execution command
argo submit -n argo --serviceaccount dma --watch whalesay.yaml
Here are the Configmaps
apiVersion: v1
data:
containerRuntimeExecutor: emissary
workflowDefaults: '{"annotations": {"workflows.argoproj.io/version": ">= 3.1.0"},
"metadata": { }, "spec": {"artifactRepositoryRef": {"configMap": "my-config",
"key": "whalesay-cache"}, "entrypoint": "entrypoint", "parallelism": 3, "podGC":
{"strategy": "OnWorkflowSuccess"}, "securityContext": {"fsGroup": 2000, "runAsGroup":
3000, "runAsUser": 1000}, "ttlStrategy": {"secondsAfterSuccess": 5}}}'
kind: ConfigMap
metadata:
annotations:
kubectl.kubernetes.io/last-applied-configuration: |
{"apiVersion":"v1","data":{"containerRuntimeExecutor":"emissary","workflowDefaults":"{\"annotations\": {\"workflows.argoproj.io/version\": \"\u003e= 3.1.0\"}, \"metadata\": { }, \"spec\": {\"artifactRepositoryRef\": {\"configMap\": \"my-config\", \"key\": \"whalesay-cache\"}, \"entrypoint\": \"entrypoint\", \"parallelism\": 3, \"podGC\": {\"strategy\": \"OnWorkflowSuccess\"}, \"securityContext\": {\"fsGroup\": 2000, \"runAsGroup\": 3000, \"runAsUser\": 1000}, \"ttlStrategy\": {\"secondsAfterSuccess\": 5}}}"},"kind":"ConfigMap","metadata":{"annotations":{},"name":"workflow-controller-configmap","namespace":"argo"}}
creationTimestamp: "2021-10-15T11:53:11Z"
name: workflow-controller-configmap
namespace: argo
resourceVersion: "1928507"
uid: bc8c16b8-e5cd-4a31-b354-1627cdf3296c
and the workflow itself
apiVersion: argoproj.io/v1alpha1
kind: Workflow
metadata:
generateName: memoized-simple-workflow-
spec:
entrypoint: whalesay
serviceAccountName: dma
artifactRepositoryRef:
configMap: my-config # default is "artifact-repositories"
key: whalesay-cache # default can be set by the annotation
arguments:
parameters:
- name: message
value: test-6
templates:
- name: whalesay
inputs:
parameters:
- name: message
memoize:
key: whalesay-cache
maxAge: "10s"
cache:
configMap:
name: my-config
container:
image: docker/whalesay:latest
command: [sh, -c]
args: ["cowsay {{inputs.parameters.message}} > /tmp/hello_world.txt"]
outputs:
parameters:
- name: hello
valueFrom:
path: /tmp/hello_world.txt
but the logs show a different serviceaccount name when the workflow is actually run
time="2021-10-13T14:32:27.424Z" level=info msg="Update leases 200"
time="2021-10-13T14:32:32.441Z" level=info msg="Get leases 200"
time="2021-10-13T14:32:32.457Z" level=info msg="Update leases 200"
time="2021-10-13T14:32:32.505Z" level=info msg="Processing workflow" namespace=argo workflow=memoized-simple-workflow-c5bp5
time="2021-10-13T14:32:32.511Z" level=info msg="Updating node memoized-simple-workflow-c5bp5 exit code 0" namespace=argo workflow=memoized-simple-workflow-c5bp5
time="2021-10-13T14:32:32.511Z" level=info msg="Setting node memoized-simple-workflow-c5bp5 outputs: {\"parameters\":[{\"name\":\"hello\",\"value\":\" ________ \\n\\u003c test-6 \\u003e\\n -------- \\n \\\\\\n \\\\\\n \\\\ \\n ## . \\n ## ## ## == \\n ## ## ## ## === \\n /\\\"\\\"\\\"\\\"\\\"\\\"\\\"\\\"\\\"\\\"\\\"\\\"\\\"\\\"\\\"\\\"___/ === \\n ~~~ {~~ ~~~~ ~~~ ~~~~ ~~ ~ / ===- ~~~ \\n \\\\______ o __/ \\n \\\\ \\\\ __/ \\n \\\\____\\\\______/ \",\"valueFrom\":{\"path\":\"/tmp/hello_world.txt\"}}]}" namespace=argo workflow=memoized-simple-workflow-c5bp5
time="2021-10-13T14:32:32.511Z" level=info msg="Updating node memoized-simple-workflow-c5bp5 status Pending -> Succeeded" namespace=argo workflow=memoized-simple-workflow-c5bp5
time="2021-10-13T14:32:32.511Z" level=info msg="Saving ConfigMap cache entry" key=whalesay name=my-config namespace=argo nodeId=memoized-simple-workflow-c5bp5
time="2021-10-13T14:32:32.516Z" level=info msg="Get configmaps 200"
time="2021-10-13T14:32:32.519Z" level=info msg="Update configmaps 403"
time="2021-10-13T14:32:32.521Z" level=error msg="Failed to save node outputs to cache" error="error creating cache entry: configmaps \"my-config\" is forbidden: User \"system:serviceaccount:argo:argo\" cannot update resource \"configmaps\" in API group \"\" in the namespace \"argo\". Please check out this page for help: https://argoproj.github.io/argo-workflows/memoization/#faqs" namespace=argo nodeID=memoized-simple-workflow-c5bp5 workflow=memoized-simple-workflow-c5bp5
time="2021-10-13T14:32:32.521Z" level=info msg="TaskSet Reconciliation" namespace=argo workflow=memoized-simple-workflow-c5bp5
time="2021-10-13T14:32:32.521Z" level=info msg=reconcileAgentPod namespace=argo workflow=memoized-simple-workflow-c5bp5
my permissions can be found here
https://github.com/wdma/Hippocampus-Analytics/blob/main/manifests/policies.yaml
The first few lines of the workflow-controller pod logs show that the Configmap was loaded correctly
time="2021-10-15T19:08:12Z" level=info msg="index config" indexWorkflowSemaphoreKeys=true
time="2021-10-15T19:08:12Z" level=info msg="cron config" cronSyncPeriod=10s
time="2021-10-15T19:08:12.689Z" level=info msg="not enabling pprof debug endpoints"
time="2021-10-15T19:08:12.703Z" level=info msg="config map" name=workflow-controller-configmap
time="2021-10-15T19:08:12.746Z" level=info msg="Get configmaps 200"
time="2021-10-15T19:08:12.764Z" level=info msg="Configuration:\nartifactRepository: {}\ncontainerRuntimeExecutor: emissary\ninitialDelay: 0s\nmetricsConfig: {}\nnodeEvents: {}\npodSpecLogStrategy: {}\ntelemetryConfig: {}\nworkflowDefaults:\n metadata:\n creationTimestamp: null\n spec:\n arguments: {}\n artifactRepositoryRef:\n configMap: my-config\n key: whalesay-cache\n entrypoint: entrypoint\n parallelism: 3\n podGC:\n strategy: OnWorkflowSuccess\n securityContext:\n fsGroup: 2000\n runAsGroup: 3000\n runAsUser: 1000\n ttlStrategy:\n secondsAfterSuccess: 5\n status:\n finishedAt: null\n startedAt: null\n"
time="2021-10-15T19:08:12.766Z" level=info msg="Persistence configuration disabled"
time="2021-10-15T19:08:12.771Z" level=info msg="Starting Workflow Controller" version=v3.2.0
time="2021-10-15T19:08:12.772Z" level=info msg="Workers: workflow: 32, pod: 32, pod cleanup: 4"
time="2021-10-15T19:08:12.784Z" level=info msg="List workflows 200"
Can anyone tell me how to fix this?
The workflow-controller itself, which is running in the argo namespace with the argo ServiceAccount, needs permissions to patch the ConfigMap. The workflow-controller is modifying the ConfigMap on behalf of the Workflow. The Workflow itself does not modify the ConfigMap.