Posting webform data when the names are random with indy

Question

I'm trying to create a login to https://umod.org/login but I'm having a hard time to understand exactly how this works. Why do I want to create a login? Their site API has a rate limit of 40 for guests and a 100 for members. I basically want to keep a logged in session while I'm working with the api.

My understanding is that I need to somehow login and get the cookie.

I have been looking at some posts and I'm getting frustrated because I simply cannot get it to work. I have found questions like this: Log in to website from Delphi and Delphi - logging in to webpage from code that makes be somewhat have a little understanding.

So what have I done? I have used inspect element to get the field names of the input fields on the login page. The email field has a id called email but the name for both fields change as soon as I hit refresh.

I have written the following code but with no success:

procedure Login(const aUsername, aPassword: string);
var
  http: TIdHTTP;
  ssl: TIdSSLIOHandlerSocketOpenSSL;
  params: TStringList;
begin
  http := TIdHTTP.Create(nil);
  try
    ssl := TIdSSLIOHandlerSocketOpenSSL.Create(nil);
    try
      ssl.SSLOptions.SSLVersions := [sslvTLSv1, sslvTLSv1_1, sslvTLSv1_2];
      http.IOHandler := ssl;
      http.HandleRedirects := True;
      http.Request.UserAgent := 'Mozilla/5.0 (Windows NT 10.0; Win64; x64; rv:61.0) Gecko/20100101 Firefox/61.0';
      http.AllowCookies := True;

      http.Get('https://umod.org/login');

      params := TStringList.Create;
      try
        params.Add('email=' + aUsername);
        params.Add('password=' + aPassword);
        params.Add('remember=1');
        params.Add('redirect=https://umod.org/');

        http.Post('https://umod.org/login', params);
      finally
        params.Free;
      end;
    finally
      ssl.Free;
    end;
  finally
    http.Free;
  end;
end;

I'm getting the following error: EIdHTTPProtocolException: HTTP/1.1 419 unknown status

Answer 1

After you Get() the HTML for the webform page, you need to parse the HTML to locate the appropriate <form> element and extract all of its <input> elements that you will need to submit back to the server.

That parsing will give you the actual names (particularly randomly-generated ones) and non-user-provided values that you need to put into your TStringList.

Any <input> element (especially "hidden" ones) that have a non-empy value attribute (if not user-provided) MUST be submitted.

This is exactly what a web browser would do, and is what the server is expecting you to do. Indy will not help you with that parsing, you need to handle it yourself. There are plenty of HTML parsers available if you look around.