How to record http requests and time cost in containers?

Question

Here's the situation, I'm using docker to build some projects in containers, I want to record the request urls of these containers to optimize these jobs.

So I find a way to run a Nginx container as forward proxy called proxy and run the other building jobs in container with http_proxy.

proxy:

docker run -d -p 8090:8090 proxy

jobs:

docker run --env http_proxy="http://127.0.0.1:8090" --network host jobs 

But I can't find the correct Nginx config to do this trick.

➜ cat nginx.conf                                                                                
worker_processes  1;

events {
    worker_connections  1024;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    gzip  on;

    server {
        listen       80;
        listen       443;
        server_name _;
        # forward proxy for CONNECT request
        proxy_connect;
        proxy_connect_allow            443 563;
        proxy_connect_connect_timeout  10s;
        proxy_connect_read_timeout     10s;
        proxy_connect_send_timeout     10s;

        location / {
            resolver 8.8.8.8;
            proxy_pass $scheme://$host$request_uri;
        }
    }
}

I also try to use Envoy to proxy the containers, and I read the doc Front Proxy and it seems that's not forward proxy, so what's the recommended way to record the http requests and time cost in containers?

Any help would be great appreciate.

Answer 1

I solved this issue by using Nginx, actually, it's easy to use Nginx as transparent forward proxy to do this trick, Nginx needs ngx_http_proxy_connect_module to proxy HTTPS requests, and the author also contributed this module to Tengine. So I try to use Tengine.

worker_processes  1;

events {
    worker_connections  65536;
}

http {
    include       mime.types;
    default_type  application/octet-stream;

    sendfile        on;
    keepalive_timeout  65;

    gzip  on;

    server {
        listen       *:80; # fix 99 address not available
        listen       *:443;# fix 99 address not available
        server_name localhost;
        resolver 10.10.10.10 ipv6=off;
        resolver_timeout 30s;


        # forward proxy for CONNECT request
        proxy_connect;
        proxy_connect_allow            443 563;
        proxy_connect_connect_timeout  30s;
        proxy_connect_read_timeout     30s;
        proxy_connect_send_timeout     30s;

        location / {
            proxy_pass $scheme://$host$request_uri;
        }
        access_log /tmp/access.log;
        error_log /tmp/error.log;
    }
}

The above conf is my Nginx.conf. To avoid connection error while connecting to upstream, I disable ipv6 option. It works.