Nginx DNS Resolver in docker depend on the conditions of the transformation

Question

I want to understand why this happens. When I use proxy_pass without regular expressions, everything works great, the DNS is resolved, and so on.

    location /images/ {
        proxy_pass http://sfs-filer:8888;
        proxy_set_header Host $host;
    }

But as soon as I add regular expressions, it starts throwing an ERROR.

[error] 29#29: *442 no resolver defined to resolve sfs-filer, client: 172.18.0.1, server: , request: "GET /9395123a-39bd-4951-acd9-e759fae2e7f4.webp HTTP/1.1", host: "localhost"

    location ~ ([a-zA-Z0-9-]+\.(jpg|jpeg|png|gif|webp|bmp|avif))$ {
        proxy_pass http://sfs-filer:8888/$1;
        proxy_set_header Host $host;
    }

Please note that in proxy_pass I do not change the domain at all, it remains 1 to 1 as in the first example. How is this even possible? Why does the resolver depend on regular expressions?

Yes, I can add resolver 127.0.0.11 valid=10s; and it will work. But the essence of it is an absolute mystery to me.

Answer 1

When your proxy_pass directive argument include variables, nginx does not resolve your upstream IP at the startup time, as it usually does, but tries to do it during the request processing, thus needing a resolver to be defined. Your question is almost the same with the How to use nginx proxy_pass with $remote_addr? question, and the answer can be found in a blog post referenced by the second answer: Proxy pass and resolver:

Linux, POSIX and the like offer only one way to get an IP from a name: gethostbyname. If you take time to read the man page (always a safe thing to do... ;)) you'll realise there is a lot to do to resolve a name: open files, ask NIS or YP what they think about it, ask a DNS server (may be in a few different ways). And all this is synchronous. Now that you are used to the nginx way, you know how bad this is and you don't want to go down the ugly synchronous way. So Igor, faithful to himself, reimplemented a DNS lookup (with an in-memory cache, mind you) just to avoid calling this ugly blocking gethostbyname... And that's why we have this extra resolver directive.

I have my own answer there too, showing how the problem can be solved without using the resolver directive. The only difference is that you are using regex location, and you need named captures to use the same technique, since any numbered captures will be overwritten during the rewrite directive execution:

    location ~ (?<path>[a-zA-Z0-9-]+\.(jpg|jpeg|png|gif|webp|bmp|avif))$ {
        rewrite ^ /$path;
        proxy_pass http://sfs-filer:8888;
        proxy_set_header Host $host;
    }

There are other ways to avoid the resolver directive. For example, the following configuration should not require it:

upstream sfs_filer_upstream {
    server  sfs-filer:8888;
}
server {
    ...
    location ~ ([a-zA-Z0-9-]+\.(jpg|jpeg|png|gif|webp|bmp|avif))$ {
        proxy_pass http://sfs_filer_upstream/$1;
        proxy_set_header Host $host;
    }

(here the sfs-filer backend server IP address will be resolved at the nginx startup.)

Defining your backend server using IP address instead of the domain name also won't require resolver to be defined. For example, the following directive

proxy_pass http://localhost/$1;

will require a resolver, while the

proxy_pass http://127.0.0.1/$1;

directive will not require it.