Search code examples
azurekubernetescontainerspodman

Run Azure Self-hosted agent with podman as unpriviliged container on Kubernetes

I am deploying an Azure Self hosted agent on a Kubernetes Cluster 1.22+ following steps in: https://learn.microsoft.com/en-us/azure/devops/pipelines/agents/docker?view=azure-devops#linuxInstructions

I am adding podman to self hosted agent as container manager, following code is added to self hosted agent Dockerfile:

# install podman
ENV VERSION_ID=20.04

RUN apt-get update -y && apt-get install curl wget gnupg2 -y && . ./etc/os-release && sh -c "echo 'deb http://download.opensuse.org/repositories/devel:/kubic:/libcontainers:/stable/xUbuntu_${VERSION_ID}/ /' > /etc/apt/sources.list.d/devel:kubic:libcontainers:stable.list" && wget -nv https://download.opensuse.org/repositories/devel:kubic:libcontainers:stable/xUbuntu_${VERSION_ID}/Release.key -O- | apt-key add - && apt-get update -y && apt-get -y install podman && podman --version

Everything runs smoothly when running the container in privileged mode.

...
        securityContext:
          privileged: true
...

When swith to privileged: false and try to connect to podman, I get following error

level=warning msg="\"/\" is not a shared mount, this could cause issues or missing mounts with rootless containers"
Error: mount /var/lib/containers/storage/overlay:/var/lib/containers/storage/overlay, flags: 0x1000: permission denied

the Command I use for connecting is:

podman login private.container.registry \
    --username $USER \
    --password $PASS \
    --storage-opt mount_program=/usr/bin/fuse-overlayfs

How can I use podman with unprivileged mode ?

Solution

  • Issue was related to Containerd's apparmor profile denying the mount syscall, I fixed it for now by disabling apparmor for the container while running unprivileged mode

    ...
  template:
    metadata:
      labels:
        app: vsts-agent-2
      annotations:
        container.apparmor.security.beta.kubernetes.io/kubepodcreation: unconfined
...
        securityContext:
          privileged: false #true

    A better way would be creating an apparmor profile that allows the mount and apply it to the container