How to start ASP.NET Core application in Docker to be open in CI Playwright tests?

Question

We have ASP.NET Core MVC microservice and want to add automated UI tests for web forms using Playwright.

We already have a GitHub Actions setup for web application build (including running unit tests for C# backend code). Playwright tests should be in the same repository and tests should be run as a part of CI before the application will be deployed.

To run the Playwright tests, we need to start the microservice locally on the build server and tests should run on URL e.g. http://localhost:5000.

Is it possible to do in a single Dockerfile at the build stage invoked from GitHub Actions?

Or does it have to be done as multiple GitHub Actions steps:

1. Build Docker image of ASP.NET web site
2. Run the image locally on CI build server on the local port, e.g. http://localhost:5000
3. Build and run Playwright project with the tests against http://localhost:5000.

Ideally it makes sense to test and fail build of broken application before pushing it to Docker registry. We do something similar in the single Dockerfile for React with Cypress projects:

run npm run test; # Jest tests
run npm run e2e-tests # Cypress tests

However I would appreciate advice, why it is not good/feasible.

Playwright documentation only shows how to build and run Playwright project, not how to start the website under the test.

I found the article Elevate Your CI/CD: Dockerized E2E Tests with GitHub Actions that describes multi-step approach, but for different technologies - Express and React containers

Can anyone give recommendations/suggest an example for ASP.NET Core site and Playwright C# tests to be run using GitHub Actions and Docker?

Answer 1

I've got a suggestion from playwright-dotnet/issues [Docs]: How to start application in docker to be open in CI Playwright tests?

We recommend e.g. starting it in the background via & and waiting it to be available via e.g. npx wait-on or starting it inside AssemblyInitialize.

AssemblyInitialize code

[TestClass]
public static class ServiceTestSetup
{
    public static Process ServiceProcess { get;  set; }

    // ignore warning  UTA031: class ServiceTestSetup does not have valid TestContext property. TestContext must be of type TestContext, must be non-static, public and must not be read-only. For example: public TestContext TestContext.
    public static TestContext TestContext { get; set; } 

    [AssemblyInitialize]
    public static void Initialize(TestContext context)
    {
        // Path to your .NET MVC service executable
        string servicePath = "MyProgram.exe";
#if !DEBUG
        servicePath = "MyProgram"; //in Linux docker app, there is no  "MyProgram.exe", just the name
#endif
        Console.WriteLine($"ServiceTestSetup.Initialize starts for {servicePath}");
        // Ensure the executable exists
        if (!File.Exists(servicePath))
        {
            var currentDirectory=System.IO.Directory.GetCurrentDirectory();
            var message= $"The specified file was not found {servicePath}" + Environment.NewLine; ;
            message+= $"In the following directory:{currentDirectory}" + Environment.NewLine;
            foreach (string file in Directory.EnumerateFiles(currentDirectory, "*.*", SearchOption.TopDirectoryOnly))
            {
                message+=$"{file}"+Environment.NewLine;
            }
            throw new FileNotFoundException(message, servicePath);
        }
        //If required to kill process see https://www.revisitclass.com/networking/how-to-kill-a-process-which-is-using-port-8080-in-windows/
        // netstat -ano | findstr 5000
        // In Admin    taskkill /F /PID 12345  
       
        var useShellExecute = false;  //use true, if want to debug output locally 
        StartService(servicePath, "", useShellExecute);
    }
    private static void StartService(string servicePath, string dllPath, bool useShellExecute=true)
    {
        bool redirectOutput = !useShellExecute;
        ServiceProcess = new Process
        {
            StartInfo = new ProcessStartInfo
            {
                FileName = servicePath,
                Arguments = dllPath, 
                UseShellExecute = useShellExecute,
                RedirectStandardOutput = redirectOutput,
                RedirectStandardError = redirectOutput,
                CreateNoWindow = false
            }
        };
         
        Console.WriteLine("serviceProcess.Start");
        ServiceProcess.Start();

        Thread.Sleep(1000); // Adjust the delay as necessary based on your service's startup time
        Console.WriteLine("CheckIfRunningLogOutput");
        CheckIfRunningLogOutput(ServiceProcess, redirectOutput);
    }
}

TestInitialize code

I've also added a call from [TestInitialize] if the start in [AssemblyInitialize] failed

public class PlaywrightTests : PageTest
{

    private string _baseUrl = "http://localhost:5000/";// TestContext.Parameters["BaseUrl"]

    [TestInitialize] //should be after base PageTest.PageSetup https://stackoverflow.com/questions/18511269/multiple-testinitialize-attributes-in-mstest
    public async Task OpenPage()
    {
        // Set the static TestContext property,allows  to indirectly log initialization messages through each test’s TestContext.
        ServiceTestSetup.TestContext = TestContext;
        if (ServiceTestSetup.ServiceProcess?.HasExited ?? true)
        { //try to start for the test instead of AssemblyInitialize
            ServiceTestSetup.Initialize(TestContext);
            if (ServiceTestSetup.ServiceProcess?.HasExited??true)
            {
                Assert.Fail("ServiceProcess has exited. See logs for the reason");
            }
        }
    }
}

Dockerfile changes

Commands in Dockerfile to run UI TESTS are the following

COPY test/PlayWright.Tests/*.runsettings /app/ 
RUN if [ -z "$SKIP_UI_TESTS" ] ; then \
        dotnet restore test/PlayWright.Tests/ && \
        dotnet publish test/PlayWright.Tests/PlayWright.Tests.sln -c Release -o /app  && \
        pwsh playwright.ps1 install && \
        pwsh playwright.ps1 install-deps chromium && \
        dotnet test PlayWright.Tests.dll  --settings:dockerfile.runsettings ; \
    else \
        echo "Skipping UI tests." ; \
    fi

There are more details in my example repo https://github.com/MNF/PlayWright.Min/