Can't access to containerlized SvelteKit app from different hosts in Docker Swarm

I have a SvelteKit app and a few other services running on multiple hosts in Docker Swarm. The SvelteKit app service is named sveltekit-frontend-app on swarm's overlay network. The app is run with node-adapter with the command node /path/to/build.

I was expecting it to be accessed over HTTP as http://sveltekit-frontend-app:3000. The app works fine and returns a valid response when another container on the same host accesses it. However, if a container from different hosts try to access, the app doesn't return a response and wait infinitely until timeout comes.

The output of curl sveltekig-frontend-app:3000/home looks like this:

*  Trying 10.0.0.1:3000...
* Connected to sveltekit-frontend-app (10.0.0.1) port 3000 (#0)
> GET /home HTTP/1.1
> Host: sveltekit-frontend-app:3000
> User-Agent: curl/7.81.0
> Accept: */*

* Recv failure: Connection reset by peer
* Closing connection 1
curl: (56) Recv failure: Connection reset by peer

I think Docker Swarm is not the cause of the problem, since other container services can talk with each other between different hosts without any problem. I thought the problem is node, so I had created a simple node server and tried to access it from different hosts, but it worked fine.

The sveltekit app actually accepts the requests and handles redirects if necessary, but just don't return responses. See the following output:

*  Trying 10.0.0.1:3000...
* Connected to sveltekit-frontend-app (10.0.0.1) port 3000 (#0)
> GET / HTTP/1.1
> Host: sveltekit-frontend-app:3000
> User-Agent: curl/7.81.0
> Accept: */*
> 
* Mark bundle as not supporting multiuse
< HTTP/1.1 308 Permanent Redirect
< location: /home
< Date: Fri, 17 Feb 2023 07:23:18 GMT
< Connection: keep-alive
< Keep-Alive: timeout=5
< Transfer-Encoding: chunked
< 
* Ignoring the response-body
* Connection #0 to host frontend left intact
* Issue another request to this URL: 'http://sveltekit-frontend-app:3000/home'
* Found bundle for host sveltekit-frontend-app: 0x123d0a39aef0 [serially]
* Can not multiplex, even if we wanted to!
* Re-using existing connection! (#0) with host sveltekit-frontend-app
* Connected to sveltekit-frontend-app (10.0.0.1) port 3000 (#0)
> GET /home HTTP/1.1
> Host: sveltekit-frontend-app:3000
> User-Agent: curl/7.81.0
> Accept: */*

I guess this problem happens in other orchestration environments as long as you containerlize sveltekit app and try to access it on the network from different hosts too, like Kubernetes, though not sure and haven't tried it. Does anyone know how to solve this issue?

Thanks in advance.

Update (2023-03-07)

I have created a repository to replicate my issue.

I've also found the cause of the issue. It was neither SvelteKit nor Docker Swarm.

I was managing a swarm cluster on Vultr's VPC network. This issue happens only when I use their VPC, not when I use their global ipv4 addresses. I've also tested VLAN network on Linode and it worked completely fine.

Solution

I've sent this post to Vultr's support and they gave me the solution.

This issue is solved by setting docker's mtu to 1450, same as Vultr's VPC network mtu size.

Modify network's section in stack.yml like this:

networks:
  app-overlay-net:
    driver: overlay
    driver_opts:
      com.docker.network.driver.mtu: 1450