Search code examples
delphisslhttp-status-code-403indyanalyzer

Delphi, Indy TLS 1.2 and TLS 1.3 gettin status code 403


i have one problem, that i dont know how to fix. since few days i can't more login via indy to server (https://auth.bmwgroup.com, https://aos.bmwgroup.com, https://carver.bmwgroup.com).

TLS 1.2 and TLS 1.3 are enabled on this server.

So i started HttpAnalyzer to check where is the problem and my programm start to working, than i close HttpAnalyzer and my programm doesnt work anymore.

Can someone help to fix the problem.

i am using this code :

procedure TForm1.Server_NewInit;
var
    List: TStringList;
    URL, AuthID, ID_USER, ID_URL : AnsiString;
    JSON: TStringStream;
    l,p : string;
begin
    idHttpC := TIdHTTP.Create(nil);
    idHttpC.ConnectTimeout := 80000;
    idHttpC.ReadTimeout := 80000;
    idHttpC.AllowCookies := true;

// config Redirect's
    idHttpC.RedirectMaximum := 35;
    idHttpC.HandleRedirects := true;
    idHttpC.HTTPOptions := [hoKeepOrigProtocol, hoTreat302Like303];
    idHttpC.OnRedirect := IdHTTP1Redirect;

// create Cookie's
    idCookieC := TIdCookieManager.Create(IdHttpC);
    idHttpC.CookieManager := idCookieC;

// create OpenSSL
    lIOHandlerC := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
    lIOHandlerC.SSLOptions.Mode := sslmClient;
    lIOHandlerC.SSLOptions.SSLVersions := [sslvTLSv1, sslvTLSv1_1, sslvTLSv1_2];
    idHttpC.IOHandler := lIOHandlerC;

// Nr1 execute redirect
..
// Nr2 serverinfo
..
// Nr3 getSessionInfo
..
// Nr4 internetb2x empty post to get AuthId
..
// Nr5 internetb2x POST JSON - authorisation nr1
..
// Nr6 getSessionInfo after login
..
// check for authorisation give me ok, so, pass and login are ok
..
// Nr7 step idFromSession
..
// Nr8 step get info about user id
..
// Nr9 execute new ID_URL
..
// Nr10 authorisation Nr 2
idHttpC.Request.Referer := idHttpC.URL.GetFullURI([]);
idHttpC.Request.CacheControl := 'no-cache';
idHttpC.Request.ContentType := 'application/x-www-form-urlencoded';
...
here i am getting problem

the steps from 1 to 9 working without any problem, i compare headers and content data, they are same as with launched analyzer.

but step nr10 give me with launched analyzer status

**RT HTTP/1.1 200 OK
RC 200
CT text/html**

Date: Mon, 22 Nov 2021 21:09:10 GMT
Server: Apache
Last-Modified: Thu, 27 May 2021 09:09:24 GMT
ETag: "1b8-5c34c1d88f900"
Accept-Ranges: bytes
Strict-Transport-Security: max-age=8640000; includeSubDomains
Content-Security-Policy: default-src 'self' 'unsafe-inline' 'unsafe-eval'
X-Content-Security-Policy: default-src 'self' 'unsafe-inline' 'unsafe-eval'
X-WebKit-CSP: default-src 'self' 'unsafe-inline' 'unsafe-eval'
X-Permitted-Cross-Domain-Policies: none
X-XSS-Protection: 1; mode=block
X-Frame-Options: sameorigin
Keep-Alive: timeout=5, max=99
Connection: Keep-Alive
content-type: text/html; charset=iso-8859-1
Content-Length: 440
refresh: 0; URL=/carver_www/carverMain.jsp

and without analyzer

**RT HTTP/1.1 403 Forbidden
RC 403
CT text/html** 

Date: Mon, 22 Nov 2021 21:10:20 GMT
Server: Apache
Vary: accept-language,accept-charset
Accept-Ranges: bytes
Strict-Transport-Security: max-age=8640000; includeSubDomains
Content-Security-Policy: default-src 'self' 'unsafe-inline' 'unsafe-eval'
X-Content-Security-Policy: default-src 'self' 'unsafe-inline' 'unsafe-eval'
X-WebKit-CSP: default-src 'self' 'unsafe-inline' 'unsafe-eval'
X-Permitted-Cross-Domain-Policies: none
X-XSS-Protection: 1; mode=block
X-Frame-Options: sameorigin
Keep-Alive: timeout=5, max=100
Connection: Keep-Alive
Transfer-Encoding: chunked
Content-Type: text/html; charset=utf-8
Content-Language: de

part2 :

here request nr9

// prepare new Request for ID_URL
    idHttpC.Request.Clear;
    idHttpC.Request.CustomHeaders.Clear;
    idHttpC.Request.Accept := 'text/html, application/xhtml+xml, */*';
    idHttpC.Request.AcceptLanguage := 'de-DE';
    idHttpC.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko';
    idHttpC.Request.Host := 'auth.bmwgroup.com';
    idHttpC.Request.CustomHeaders.Values['Connection'] := 'Keep-Alive';
    idHttpC.Request.CustomHeaders.Values['DNT'] := '1';
    idHttpC.Request.AcceptEncoding := 'gzip, deflate';

// Nr9 execute new ID_URL
    try
        res1 := idHttpC.Get(ID_URL);
        HTML_SaveToFile(res1, 'sFile9.txt');
        idHttpC.Response.RawHeaders.SaveToFile('RH9.txt');
    except
        on e:EIdSocketError do
            ShowMessage('EIdSocketError: ' + e.Message);
        on e:EIdReadTimeout do
            ShowMessage('EIdReadTimeout: ' + e.Message);
        on e:EIDHttpProtocolException do
            ShowMessage('EIDHttpProtocolException: ' + IntToStr(e.ErrorCode));
        on e:Exception do
            ShowMessage('Exception: ' + e.Message);
    end;
    Memo4.Lines.Add('Nr9 done');

with this request i got document :

<html lang="en">
    <head>
        <meta charset="utf-8">
        <meta http-equiv="X-UA-Compatible" content="IE=edge">
        <meta name="viewport" content="width=device-width, initial-scale=1">
        <meta name="description" content="OAuth 2.0 Form Post">
        <title>Submit This Form</title>
    </head>
    <body onload="javascript:document.forms[0].submit()">
        <form method="post" action="https://carver.bmwgroup.com:443/agent/cdsso-oauth2">
            <input type="hidden" name="id_token" value="...."/><input type="hidden" name="state" value="...."/>
        </form>
    </body>
</html>

this code i execute in step 10 :

// Nr10 authorisation Nr 2
    idHttpC.Request.Clear;
    idHttpC.Request.Accept := 'text/html, application/xhtml+xml, */*';
    idHttpC.Request.Referer := idHttpC.URL.GetFullURI([]);
    idHttpC.Request.AcceptLanguage := 'de-DE';
    idHttpC.Request.UserAgent := 'Mozilla/5.0 (Windows NT 6.1; WOW64; Trident/7.0; rv:11.0) like Gecko';
    idHttpC.Request.ContentType := 'application/x-www-form-urlencoded';
    idHttpC.Request.Host := 'carver.bmwgroup.com';
    idHttpC.Request.CustomHeaders.Values['Connection'] := 'Keep-Alive';
    idHttpC.Request.CustomHeaders.Values['DNT'] := '1';
    idHttpC.Request.CacheControl := 'no-cache';
    idHttpC.Request.AcceptEncoding := 'gzip, deflate';

    List := TStringList.Create;
    List.Add('id_token='+JSON_GET_ID(res1, 3, 0));
    List.Add('state='+JSON_GET_ID(res1, 4, 0));
    try
        res1 := idHttpC.Post('https://carver.bmwgroup.com/agent/cdsso-oauth2', List);
        FReeAndnil(List);
        Memo4.Lines.Add('RT' + idHttpC.Response.ResponseText);
        Memo4.Lines.Add('RC' + IntToStr(idHttpC.Response.ResponseCode));
        Memo4.Lines.Add('CT' + idHttpC.Response.ContentType);
        HTML_SaveToFile(res1, 'sFile10.txt');
        idHttpC.Response.RawHeaders.SaveToFile('RH10.txt');
    except
        on e:EIdSocketError do
            ShowMessage('EIdSocketError: ' + e.Message);
        on e:EIdReadTimeout do
            ShowMessage('EIdReadTimeout: ' + e.Message);
        on e:EIDHttpProtocolException do
            ShowMessage('EIDHttpProtocolException: ' + IntToStr(e.ErrorCode));
        on e:Exception do
            ShowMessage('Exception: ' + e.Message);
    end;
    Memo4.Lines.Add('Nr10 done');

Solution

  • The problem was on step nr9, where i got in indy compressed data :

    Accept-Ranges: bytes
    Content-Encoding: gzip
    

    Why the code was working with httpAnalyzer ? Because httpAnalyzer logging network protocol via owned library and in this case httpAnalyzer decompress data for me.

    after i added

    idHttpC.Compressor := TIdCompressorZLib.Create(idHttpC);
    

    the code work again.