Search code examples
asp.netkubernetes

How do I run an ASP.Net Core container in Kubernetes as non-root user?

I am trying to run an ASP.Net docker container on Kubernetes as a non-root user. I have this dockerfile:

FROM mcr.microsoft.com/dotnet/core/aspnet:3.1-buster-slim AS base
WORKDIR /app
EXPOSE 8443

FROM mcr.microsoft.com/dotnet/core/sdk:3.1-buster AS build
WORKDIR /src
COPY ["MyProgram.API/MyProgram.API.csproj", "MyProgram.API/"]
COPY ["MyProgram.Services/MyProgram.Services.csproj", "MyProgram.Services/"]
COPY ["MyProgram.Core/MyProgram.Core.csproj", "MyProgram.Core/"]
COPY ["MyProgram.Data/MyProgram.Data.csproj", "MyProgram.Data/"]
RUN dotnet restore "MyProgram.API/MyProgram.API.csproj"
COPY . .
WORKDIR "/src/MyProgram.API"
RUN dotnet build "MyProgram.API.csproj" -c Release -o /app/build

FROM build AS publish
RUN dotnet publish "MyProgram.API.csproj" -c Release -o /app/publish

FROM base AS final
WORKDIR /app
COPY --from=publish /app/publish .
ENTRYPOINT ["dotnet", "MyProgram.API.dll"]

When I run it locally, I can go to https://localhost:8443 and use my app succesfully. When I deploy it to Kubernetes using this file:

apiVersion: apps/v1 
kind: Deployment
# snip
    spec:
       securityContext:
         fsGroup: 2000
         runAsNonRoot: true
         runAsUser: 1000
      containers:
      - name: myprogram
        image: mycompany/myprogram:develop
        imagePullPolicy: "Always"
        env:
        - name: "ASPNETCORE_ENVIRONMENT"
          value: "Kubernetes"
        ports:
        - containerPort: 8443
          name: "myprogram"
         securityContext:
           allowPrivilegeEscalation: false
      imagePullSecrets:
      - name: privatereposecret

---

apiVersion: v1
kind: Service
#snip
spec:
  type: NodePort
  ports:
  - protocol: TCP
    port: 8081
    targetPort: 8443
    nodePort: 31999
  selector:
   app: myprogram

My container won't start and gives these gives these log files:

[13:13:30 FTL] Unable to start Kestrel.
System.Net.Sockets.SocketException (13): Permission denied
   at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, String callerName)
   at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
   at System.Net.Sockets.Socket.Bind(EndPoint localEP)
   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.<>c__DisplayClass21_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.AnyIPListenOptions.BindAsync(AddressBindContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IServerAddressesFeature addresses, KestrelServerOptions serverOptions, ILogger logger, Func`2 createBinding)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
[13:13:30 FTL] Application start-up failed
System.Net.Sockets.SocketException (13): Permission denied
   at System.Net.Sockets.Socket.UpdateStatusAfterSocketErrorAndThrowException(SocketError error, String callerName)
   at System.Net.Sockets.Socket.DoBind(EndPoint endPointSnapshot, SocketAddress socketAddress)
   at System.Net.Sockets.Socket.Bind(EndPoint localEP)
   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketConnectionListener.Bind()
   at Microsoft.AspNetCore.Server.Kestrel.Transport.Sockets.SocketTransportFactory.BindAsync(EndPoint endpoint, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.<>c__DisplayClass21_0`1.<<StartAsync>g__OnBind|0>d.MoveNext()
--- End of stack trace from previous location where exception was thrown ---
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindEndpointAsync(ListenOptions endpoint, AddressBindContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.ListenOptions.BindAsync(AddressBindContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.AnyIPListenOptions.BindAsync(AddressBindContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.AddressesStrategy.BindAsync(AddressBindContext context)
   at Microsoft.AspNetCore.Server.Kestrel.Core.Internal.AddressBinder.BindAsync(IServerAddressesFeature addresses, KestrelServerOptions serverOptions, ILogger logger, Func`2 createBinding)
   at Microsoft.AspNetCore.Server.Kestrel.Core.KestrelServer.StartAsync[TContext](IHttpApplication`1 application, CancellationToken cancellationToken)
   at Microsoft.AspNetCore.Hosting.GenericWebHostService.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.Internal.Host.StartAsync(CancellationToken cancellationToken)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.RunAsync(IHost host, CancellationToken token)
   at Microsoft.Extensions.Hosting.HostingAbstractionsHostExtensions.Run(IHost host)
   at MyProgram.API.Program.Main(String[] args) in /src/MyProgram.API/Program.cs:line 30

If I try the exact same deployment without a SecurityContext the container works perfectly. What's going wrong?

Solution

  • Kestrel is trying to bind to port 80 and/or port 443 because that's its default unless you tell it otherwise, and you can't do that unless priviledged.

    You need to specify the ports, usually via environment variables, and expose them, e.g.

    # Declare ports above 1024, as an unprivileged non-root user cannot bind to ports <= 1024
ENV ASPNETCORE_URLS http://+:8000;https://+:8443
EXPOSE 8000
EXPOSE 8443