Mocking a system dependencies

I'm working on a system that calls external APIs, some are owned by my company and others are not.

My system is composed of an HTTP interface that takes orders and publishes them into a message queue in order to run an operation chain. My system is composed of 3 NodeJS processes (1 for HTTP, 2 message queue consumers), 2 databases and a message queue.

As I develop my application it becomes hard to test all the scenarios covered by my system (even tho I have unit tests). To ensure that all the components are working together, I am writing specifications using the Gherkin language and cucumber js.

To test the system, I want to be as close as the deployment environment, so I start all my system including the databases, the NodeJS processes and the message queue with docker-compose. All of the components of the system are communicating through a docker network defined in the docker-compose configuration.

The issue is that I can't make sure that all of the external APIs are in the right state ready to accept my request and that they will respond in a way that's interesting for my test steps.

So, I thought about using a Mock server for each of my dependencies and discovered pact.io. As I understand, Pact allows me to write contracts and start a mock server so my system can then run HTTP requests against the mock server. Pact also allows me to give the contract to the service provider so it can also run the contract against the real app to see if it really works.

I saw the examples, in javascript, and I am able to start a mock service, provide it an interaction, verify the interaction and close the mock service. (JS with mocha example)

My issue is that I want my system to be as close as the production so I want it to access the Pact mock service through my docker network. I saw a Pact CLI docker image to run the pact mock service (Pact CLI docker image) but once my mock server is dockerized, I lose the control that I had with the JS wrapper to add new interactions.

Also, I don't want to write pact files, I want to add interactions at the time my tests are running otherwise I will declare the test data twice (once in the cucumber tests scenarios and once in the pact files).

My questions are:

Is there a way to bind the JS wrapper to an existing mock service, a dockerize one? When using the docker pact image, is there a way to add interaction at run time? Is pact the right tool to use since I just need a mock service?

Edit

I just create a sandbox env to see what could be done with the NodeJS wrapper. It seems that you can create a mock service using docker and them control it via the NodeJS wrapper.

# Starts the docker container 

docker run -dit \
  --rm \
  --name pact-mock-service \
  -p 1234:1234 \
  -v <YOUR_PATH>/docker_pacts/:/tmp/pacts \
  pactfoundation/pact-cli:latest \
  mock-service \
  -p 1234 \
  --host 0.0.0.0 \
  --pact-dir /tmp/pacts

const {Pact, MockService, } = require('@pact-foundation/pact') 
const axios = require('axios')

const pact = new Pact({
  consumer: "my client",
  provider: "some provider",
  // Those two are ignored since we override the inner mock service
  port: 1234,
  host: 'localhost'
})


const mockService = new MockService(
  // You need to duplicate those data, normally they are passed
  // by the pact object when calling `pact.setup()`.
  'my client', 
  'provider', 
  // the port and host to the docker container
  1234, 
  'localhost'
)

pact.mockService = mockService

async function run () {
  await pact.addInteraction({
    state: "some data is created",
    withRequest: {
      method: "GET",
      path: "/hello"
    },
    willRespondWith: {
      status: 200,
      body: {
        hello: 'hello world'
      }
    },
    uponReceiving: ''
  })

  const response = await axios.get('http://localhost:1234/hello')

  console.log(response.data) // { "hello": "hello world" }
}

run().catch(console.error)

Edit 2

I will probably follow Matthew Fellows's answer, and test my system using some sort of unit tests with Pact mocking the external interactions of my system.

Solution

So, I thought about using a Mock server for each of my dependencies and discovered pact.io. As I understand, Pact allows me to write contracts and start a mock server so my system can then run HTTP requests against the mock server. Pact also allows me to give the contract to the service provider so it can also run the contract against the real app to see if it really works.

Yes, that's correct. Pact can probably be considered as a unit testing tool for your API client, that uses contract testing to ensure the mocks are valid.

If you're using Pact just for mocking, you're missing out on all of the key benefits though.

Using Pact at such a high level test is considered bad practice, and as you can see, is difficult to do (you're working against the way it is intended to be used).

I would worry less about overlapping tests (end-to-end tests will always double up with other layers testing by design), and concern yourself more about ensuring each of the API contracts are covered by Pact. These tests will run much faster, are more precise in what the test and are less flakey.

This way, you can reduce the scope of your end-to-end BDD scenarios to the key ones and that should reduce the maintenance cost.