Monkeypatching/mocking the HTTPX external requests

Question

I'm trying to monkeypatch the external request. Here is the code of a web endpoint:

import httpx, json
...
@app.get('/test')
async def view_test(request):
    async with httpx.AsyncClient() as client:
# sending external request
        api_response = await client.get(
            f'https://jsonplaceholder.typicode.com/todos/1',
            timeout=10,
        )
        resp = api_response.json()
# modifying the result
        resp['foo'] = 0
# forwarding the modified result back to the user
        return HTTPResponse(json.dumps(resp), 200)

When user sends a GET request to /test, it requests an external API (JSONPlaceholder), gets the JSON result and adds 'foo' = 0 to it. After that it forwards the result back to the user. Here is the Postman result:

{
    "userId": 1,
    "id": 1,
    "title": "delectus aut autem",
    "completed": false,
    "foo": 0
}

Next, here is my pytest code:

import httpx, pytest
...
# The `client` parameter is the fixture of web app
def test_view_test(client, monkeypatch):
    async def return_mock_response(*args, **kwargs):
        return httpx.Response(200, content=b'{"response": "response"}')

    monkeypatch.setattr(httpx.AsyncClient, 'get', return_mock_response)
    _, response = client.test_client.get('/test')
    assert response.json == {'response': 'response', 'foo': 0}
    assert response.status_code == 200

I used pytest's monkeypatch fixture to mock the HTTPX request's result with {"response": "response"}. So basically what I expected is that endpoint adds 'foo' = 0 to my mocked result. But instead it returned {"response": "response"} unmodified. Here's the traceback of pytest -vv command:

>       assert response.json == {'response': 'response', 'foo': 0}
E       AssertionError: assert {'response': 'response'} == {'response': 'response', 'foo': 0}
E         Common items:
E         {'response': 'response'}
E         Right contains 1 more item:
E         {'foo': 0}
E         Full diff:
E         - {'foo': 0, 'response': 'response'}
E         ?  ----------
E         + {'response': 'response'}

Can someone help me with why the endpoint doesn't modify httpx.AsyncClient().get mocked result? I used sanic==22.9.0 for backend, httpx==0.23.0 for requests, and pytest==7.2.0 for testing.

Expected to get {'response': 'response', 'foo': 0} instead got {"response": "response"} - an unmodified result of mocked httpx response.

Answer 1

The issue is that sanic-testing uses httpx under the hood. So, when you are monkeypatching httpx you are also impacting the test client. Since you only want to mock the outgoing calls we need to exempt those from being impacted.

My comment to @srbssv on Discord was to monkeypatch httpx with a custom function that would inspect the location of the request. If it was the internal Sanic app, proceed as is. If not, then return with a mock object.

Basically, something like this:

from unittest.mock import AsyncMock

import pytest
from httpx import AsyncClient, Response
from sanic import Sanic, json


@pytest.fixture
def httpx():
    orig = AsyncClient.request
    mock = AsyncMock()

    async def request(self, method, url, **kwargs):
        if "127.0.0.1" in url:
            return await orig(self, method, url, **kwargs)
        return await mock(method, url, **kwargs)

    AsyncClient.request = request
    yield mock
    AsyncClient.request = orig


@pytest.fixture
def app():
    app = Sanic("Test")

    @app.post("/")
    async def handler(_):
        async with AsyncClient() as client:
            resp = await client.get("https://httpbin.org/get")
            return json(resp.json(), status=resp.status_code)

    return app


def test_outgoing(app: Sanic, httpx: AsyncMock):
    httpx.return_value = Response(201, json={"foo": "bar"})

    _, response = app.test_client.post("")
    assert response.status == 201
    assert response.json == {"foo": "bar"}
    httpx.assert_awaited_once()