Search code examples
kubernetesamazon-eksvolumes

EKS - How to add a Volume with Kyverno


Issue #1 - Solved

I have an application running in EKS, and using Helm and ArgoCD in conjunction with it. I'm trying to add a volume to certain Pods in my namespace by using a Kyverno Policy resource, but when I try to create a resource like Sample Kyverno Resource, I keep getting the below error:

api-versions wgpolicyk8s.io/v1alpha2/ClusterPolicyReport --api-versions wgpolicyk8s.io/v1alpha2/PolicyReport --include-crds` failed exit status 1: Error: parse error at (deployment/templates/kyverno2.yaml:30): function "request" not defined

I've tried a few attempts such as using "Escaping Variables" like in this link

I've gotten the example in this page to work without making any references to "request". I tried a lot of different ways to escape the curly brackets '{{' to get the syntax to work, but none seem to help. Any advice is appreciated!

Edit: I was able to get the "request" object defined by referencing the request object with this syntax:

"{{`{{ request.object.metadata.annotations.inject }}`}}"

Issue #2 - Solved

But now I'm working through this issue:

admission webhook "validate-policy.kyverno.svc" denied the request: policy contains invalid variables: variable substitution failed for rule add-volume: failed to resolve request.object.metadata.annotations.inject at path /preconditions/all/0/key: invalid JMESPath query request.object.metadata.annotations.inject-certs: SyntaxError: Unexpected token at the end of the expression: tNumber

Edit: Because "inject" wasn't a defined variable, I needed to put it in quotes. This would be the proper syntax, and the empty string at the end is needed in case a resource does not have this "inject" in its annotation.

"{{`{{ request.object.metadata.annotations.\"inject\" || \"\" }}`}}"

Issue #3 - Solved

Using Json6904 worked, but when using this instead, it fails:

mutate:
  foreach:
  - list: "{{`{{ request.object.spec.containers }}`}}"
    patchStrategicMerge:

admission webhook "mutate.kyverno.svc-fail" denied the request: mutation policy add-volume error: failed to apply policy add-volume rules [autogen-add-ssl-certs]

Edit: This list field value should not be in curly brackets but instead in quotes:

  - list: "request.object.spec.containers"

I was able to get the Kyverno Policy to work.

Thank you,

Prem


Solution

  • I've added each issue I faced in the post above and their answer.

    The answer to the original question was to use another set of curly brackets and escape any strings with \ on the quotes:

    "{{`{{ request.object.metadata.annotations.\"inject\" || \"\" }}`}}"
    

    The last issue was when iterating over a list, I should not do the above, but just have the iteration in a string.

      - list: "request.object.spec.containers"