I want to implement a proxy server to forward traffic to upstream proxy server.
client <-> proxy server <-> upstream proxy server <-> ..... <-> target server
After searching for a long time, and ask GTP for help, below is http/https tunnel proxy
use tokio::io::{self, AsyncReadExt, AsyncWriteExt};
use tokio::net::{TcpListener, TcpStream};
async fn handle_connection(mut inbound: TcpStream, target_addr: &str) -> io::Result<()> {
let mut buffer = [0; 1024];
let n = inbound.read(&mut buffer).await?;
let request = String::from_utf8_lossy(&buffer[..n]);
println!("request {:?}", request);
let mut outbound = TcpStream::connect(target_addr).await?;
println!("outbound {:?}", outbound);
if request.starts_with("CONNECT") {
inbound
.write_all(b"HTTP/1.1 200 Connection Established\r\n\r\n")
.await?;
}
let (mut ri, mut wi) = inbound.split();
let (mut ro, mut wo) = outbound.split();
let initial_data = &buffer[..n];
let write_initial = async {
if !initial_data.is_empty() {
wo.write_all(initial_data).await?;
}
io::copy(&mut ri, &mut wo).await
};
tokio::try_join!(write_initial, io::copy(&mut ro, &mut wi))?;
// tokio::try_join!(io::copy(&mut ri, &mut wo), io::copy(&mut ro, &mut wi))?;
Ok(())
}
#[tokio::main]
async fn main() -> io::Result<()> {
let listener = TcpListener::bind("127.0.0.1:8181").await?;
loop {
let (inbound, _) = listener.accept().await?;
tokio::spawn(async move {
let target_addr = "127.0.0.1:12345";
if let Err(e) = handle_connection(inbound, target_addr).await {
eprintln!("Failed to forward connection: {}", e);
}
});
}
}
This work for http traffic
$ curl -v 'http://httpbin.org/anything' -x http://localhost:8181
* Trying 127.0.0.1:8181...
* Connected to (nil) (127.0.0.1) port 8181 (#0)
> GET http://httpbin.org/anything HTTP/1.1
> Host: httpbin.org
> User-Agent: curl/7.81.0
> Accept: */*
> Proxy-Connection: Keep-Alive
>
* Mark bundle as not supporting multiuse
< HTTP/1.1 200 OK
< Content-Length: 373
< Access-Control-Allow-Credentials: true
< Access-Control-Allow-Origin: *
< Connection: keep-alive
< Content-Type: application/json
< Date: Mon, 02 Sep 2024 12:39:53 GMT
< Keep-Alive: timeout=4
< Proxy-Connection: keep-alive
< Server: gunicorn/19.9.0
<
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Accept-Encoding": "gzip",
"Host": "httpbin.org",
"User-Agent": "curl/7.81.0",
"X-Amzn-Trace-Id": "Root=1-66d5b219-3fc519364e7b7fc100db86cc"
},
"json": null,
"method": "GET",
"origin": "x.y.z.k",
"url": "http://httpbin.org/anything"
}
* Connection #0 to host (nil) left intact
Not work on https traffic
$ curl -v 'https://httpbin.org/anything' -x http://localhost:8181
* Trying 127.0.0.1:8181...
* Connected to (nil) (127.0.0.1) port 8181 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to httpbin.org:443
> CONNECT httpbin.org:443 HTTP/1.1
> Host: httpbin.org:443
> User-Agent: curl/7.81.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection Established
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* (5454) (IN), , Unknown (72):
* error:0A00010B:SSL routines::wrong version number
* Closing connection 0
curl: (35) error:0A00010B:SSL routines::wrong version number
I don't know where I went wrong, but when I asked gtp multiple times, it gave thesame and incorrect answers :neutral_face:
I am very grateful for any suggestions.
[edit] Sorry, I also post this question on rust forum. https://users.rust-lang.org/t/tunnel-proxy-for-https-traffic/116955
BTW, 127.0.0.1:12345 is working for https traffic.
$ curl -v 'https://httpbin.org/anything' -x http://localhost:12345
* Trying 127.0.0.1:12345...
* Connected to (nil) (127.0.0.1) port 12345 (#0)
* allocate connect buffer!
* Establish HTTP proxy tunnel to httpbin.org:443
> CONNECT httpbin.org:443 HTTP/1.1
> Host: httpbin.org:443
> User-Agent: curl/7.81.0
> Proxy-Connection: Keep-Alive
>
< HTTP/1.1 200 Connection established
<
* Proxy replied 200 to CONNECT request
* CONNECT phase completed!
* ALPN, offering h2
* ALPN, offering http/1.1
* CAfile: /etc/ssl/certs/ca-certificates.crt
* CApath: /etc/ssl/certs
* TLSv1.0 (OUT), TLS header, Certificate Status (22):
* TLSv1.3 (OUT), TLS handshake, Client hello (1):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.3 (IN), TLS handshake, Server hello (2):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Certificate (11):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server key exchange (12):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Server finished (14):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Client key exchange (16):
* TLSv1.2 (OUT), TLS header, Finished (20):
* TLSv1.2 (OUT), TLS change cipher, Change cipher spec (1):
* TLSv1.2 (OUT), TLS header, Certificate Status (22):
* TLSv1.2 (OUT), TLS handshake, Finished (20):
* TLSv1.2 (IN), TLS header, Finished (20):
* TLSv1.2 (IN), TLS header, Certificate Status (22):
* TLSv1.2 (IN), TLS handshake, Finished (20):
* SSL connection using TLSv1.2 / ECDHE-RSA-AES128-GCM-SHA256
* ALPN, server accepted to use h2
* Server certificate:
* subject: CN=httpbin.org
* start date: Aug 20 00:00:00 2024 GMT
* expire date: Sep 17 23:59:59 2025 GMT
* subjectAltName: host "httpbin.org" matched cert's "httpbin.org"
* issuer: C=US; O=Amazon; CN=Amazon RSA 2048 M02
* SSL certificate verify ok.
* Using HTTP2, server supports multiplexing
* Connection state changed (HTTP/2 confirmed)
* Copying HTTP/2 data in stream buffer to connection buffer after upgrade: len=0
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* Using Stream ID: 1 (easy handle 0x562af2195eb0)
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
> GET /anything HTTP/2
> Host: httpbin.org
> user-agent: curl/7.81.0
> accept: */*
>
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection state changed (MAX_CONCURRENT_STREAMS == 128)!
* TLSv1.2 (OUT), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* TLSv1.2 (IN), TLS header, Supplemental data (23):
< HTTP/2 200
< date: Tue, 03 Sep 2024 00:21:55 GMT
< content-type: application/json
< content-length: 342
< server: gunicorn/19.9.0
< access-control-allow-origin: *
< access-control-allow-credentials: true
<
{
"args": {},
"data": "",
"files": {},
"form": {},
"headers": {
"Accept": "*/*",
"Host": "httpbin.org",
"User-Agent": "curl/7.81.0",
"X-Amzn-Trace-Id": "Root=1-66d656a3-3b632abc5c2a7cff6f66e16d"
},
"json": null,
"method": "GET",
"origin": "a.b.e.f",
"url": "https://httpbin.org/anything"
}
* TLSv1.2 (IN), TLS header, Supplemental data (23):
* Connection #0 to host (nil) left intact
TL;DR: your proxy must forward the CONNECT request from the client to the upstream proxy, not just the inner traffic.
Your HTTP proxy implementation at 127.0.0.1:8181 is having a fixed upstream proxy 127.0.0.1:12345. Both of these proxies expect a CONNECT request for HTTPS.
But, while your proxy at 127.0.0.18181 handles a CONNECT from the client it will not send a CONNECT to the upstream proxy at 127.0.0.1:12345. Instead it will directly try to pass through everything after the CONNECT request, i.e. it will treat the upstream as HTTPS server (expecting direct TLS) and not a HTTP proxy (expecting CONNECT request).
Since the upstream proxy does not get a proper HTTP request (i.e. the expected CONNECT) but instead seemingly gibberish (the start of the inner TLS handshake from the client - specifically the TLS ClientHello) it will respond with a HTTP error. This HTTP error is passed through the client which will treat it as part of the inner TLS handshake - since this (specifically the TLS ServerHello) is what be sent in the expectations of the client.
The seemingly strange error "wrong version number" comes from the client trying to interpret the HTTP error from the upstream proxy as a TLS record, specifically trying to interpret the second and third byte as the TLS version - see structure of TLS record layer. But instead of these bytes being something like 0x0303 (TLS 1.2) they will be more like 0x5454, i.e. the start of a HTTP response (which starts with HTTP/1..) and thus totally unexpected.
To fix the problem your proxy must not only handle CONNECT request by the client as you already do, but also forward the CONNECT request to the upstream proxy - which you don't.