Search code examples
gohttp-proxygo-modules

go mod returns 404 while curl returns 200 OK or how to debug go mod


I have a problem that may seems trivial, but I spend hours solving it and nothing helps. First of all, I cannot reproduce it anywhere but on my machine.

For some reason go mod gets 404 instead of 200 when it tries to receive metadata about a module.

$ go mod download -x gitlab.com/secret/secret-repo/v2@latest
# get https://gitlab.com/secret/secret-repo/v2?go-get=1
# get https://gitlab.com/secret/secret-repo/v2?go-get=1: 404 Not Found (0.256s)

If I run curl in the same bash session, I got plain 200 OK

$ curl -o /dev/null -s -w "%{http_code}\n"  https://gitlab.com/secret/secret-repo/v2?go-get=1
200

Do you have any ideas on how to debug such thing? -x doesn't help much

I tried reproducing this problem on another machine and I cannot reproduce it. Even with this very same machine when I run a docker image, go mod gets 200 OK

$ go mod download -x  gitlab.com/secret/secret-repo/v2@latest
# get https://proxy.golang.org/gitlab.com/secret/secret-repo/v2/@v/list
# get https://proxy.golang.org/gitlab.com/secret/secret-repo/v2/@v/list: 404 Not Found (2.142s)
# get https://gitlab.com/secret/secret-repo/v2?go-get=1
# get https://gitlab.com/secret/secret-repo/v2?go-get=1: 200 OK (0.215s)

I just don't know how to debug go mod further. Also, it ignores http_proxy or HTTP_PROXY variables, so I don't know how to proerly make it work with mitmproxy for example.

Other things that I've tried: trying the same command for another user, and it works returning 200 OK.

When I login into my user shell, even with non envasive su myuser -s /bin/sh having almost identical env, it still returns 404. I just don't know how to debug it further.


Solution

  • It looks you have some invalid HTTP credentials in your $HOME/.netrc, so when you go mod download your private module, the go command authenticates with those credentials giving you a 404. On the other side, curl doesn't use your .netrc credentials by default (and the other environment you tested and the Docker build don't have access to your .netrc), so you get a 200 response there.