Unicode special character U+FFFD returning from my Nest.JS API, but only sometimes

I am experiencing the following issue in my Nest.Js API: About 3 to 5 times a day, for periods of about 1 hour each the API responses are not handling UTF-8 special characters correctly. For instance, a JSON response like

{ "name":"João Pessoa" } returns as { "name":"Jo�o Pessoa " }.

While debugging, I concluded that the characters become corrupted after the API logic, maybe in the nest serializer. For example, in this code snippet:

console.log(sendResourceDto);
const { data } = await firstValueFrom(
  this.httpService
    .post(
      process.env.OTHER_SERVICE + '/resource',
      sendResourceDto,
      {
        headers: {
          'Content-Type': 'application/json; charset=utf-8',
        },
      }
    ).pipe( /* ... */ )
);

The characters are still correct in the console log, but as soon as the request reaches OTHER_SERVICE (also our code base), they are corrupted. This also happens within regular controller calls.

I have compared these periods with possible usage spikes in the database, server, or external services (in my case, an AWS Lambda), but I found no correlation.

Another option that I haven’t ruled out, but could be likely, is that the issue might be with NGINX. However, I believe this is unlikely since the OTHER_SERVICE and my API communicate locally, without the server’s reverse proxy configuration.

Anything related could help me

I have tested dozens of possibilities in development, but I couldn’t reproduce the problem. Despite everything, I tried to force the error in production, but it doesn't occur. It only happens at arbitrary times."

Solution

Had this exact same thing happen with a NestJS app running on Docker. What made it harder to debug is that the service ran fine when the container booted, but after a random time (somewhere around 30 minutes) it started to fail and return corrupted characters in the API JSON response exactly as you have described.

Once it started returning corrupted characters, it remained that way until the container was rebooted.

While I can't give you the exact reason, the cause was related to the node image. node:21.6.1-bookworm-slim works as expected, but upgrading to node:22.7.0-bookworm-slim we got this error.