I cannot get any website to accept my CONNECT request. I was trying to create a simple forward proxy server but how come I can't even get to make CONNECT work? If websites actually don't allow CONNECT (like, httpbin doesn't) then death to all HTTP proxies I guess?
As seen on https://reqbin.com/Article/HttpConnect I tried to do a CONNECT request and miserably failed...
def test4():
# https://reqbin.com/Article/HttpConnect
url = r"https://reqbin.com:443"
headers = {
"Host": "reqbin.com:443"
}
with httpx.Client() as client:
response = client.request("CONNECT", url, headers=headers)
print(response,response.text)
The print outputted a whole HTML page so I will only tell the relevant parts:
<Response [400 Bad Request]>
<title>HTTP CONNECT is not supported | reqbin.com | Cloudflare</title>
So like, what? Does that literally mean CONNECT is not allowed so death to all HTTP proxies? Is CONNECT deprecated? What happened? I don't get it... I want to create a proxy server but I am literally trying websites to accept CONNECT requests right now... Why is this even an issue? Not only with reqbin, google also gives me a HTTP 400 with "malformed request" and httpbin literally says "method not supported".
I also tried a socket approach. You can see it below if you want to I guess...
def test5():
import socket
# Useless, almost same as test4 actually... HTTP 400 by cloudflare.
import socket
target_host = "reqbin.com"
target_port = 443
# Create a socket connection to the target server
with socket.create_connection((target_host, target_port)) as sock:
# Send the raw CONNECT request taken from https://reqbin.com/Article/HttpConnect exactly
connect_request = f"CONNECT {target_host}:{target_port} HTTP/1.1\r\nHost: {target_host}:{target_port}\r\n\r\n"
sock.sendall(connect_request.encode())
# Receive and print the response
response = sock.recv(4096)
print(response.decode())
A CONNECT request is done from the client to the forward proxy in order to establish a connection to the target by the proxy. Within this connection the HTTPS traffic from the client is then forwarded to the target and the response traffic back, without additional encapsulation by a new CONNECT request to the target site.
Thus CONNECT is only relevant between client and forward proxy. Since arbitrary websites are not forward proxies they don't support the CONNECT request.