I have an application written in Delphi 6 that uses the Indy 9.0.18 HTTP client component to drive a robot. If I recall correctly when I did the Indy install, version 10 and newer does not work with Delphi 6 so I am using 9.0.18. On newer desktops the program runs completely fine. But I am having some problems on older laptops. Note in all case I am getting absolutely no errors or Exceptions.
The robot is an HTTP server that responds to HTTP requests to drive it. To obtain continuous motion, you have to send a drive command (e.g. - move forward) in a continuous loop. A drive command is an HTTP request to the numeric IP address the robot responds to, no domain name is involved with the URL used in the request so that rules out domain name resolution as a problem (I believe). Once you get the response from the last HTTP request, you turn around immediately and send the next one. You can tell when a system is having trouble keeping up with the loop because the robot makes small jerky moves, never able to reach the continuous momentum needed for smooth motion because the motors have time to settle down and stop.
The two systems that are having trouble are laptop computers and have the following CPU and memory and are running Windows XP SP3:
Both of these systems can not obtain smooth motion except transiently as indicated below.
The reason I say it's a laptop problem is because the two systems above are laptops. In contrast, I have an old Pentium 4 single core with Hyperthreading (2.8 GHz, 2.5 GB of memory). It can obtain smooth motion. However, although continuous the robot moves noticeably slower indicating that there's still a slight delay between the HTTP requests, but not enough to completely stop the motors and therefore the motion is still continuous, albeit noticeably slower than on my quad core or a dual core desktop.
I am aware that the other discriminant in the data points is that the old Pentium 4 desktop has 2.5 times memory over the laptops, despite being a nearly archaic PC. Perhaps the real culprit is some memory thrashing? Every now and then the robot does run smoothly but soon reverts back to stuttering again, indicating that without whatever is mucking up the interaction over the socket, smooth motion is occasionally possible. Note, the robot is also streaming audio both ways to and from the PC and streaming video to the PC (but not the other direction), so there's a fair amount of processing going on along with driving the robot.
The Indy HTTP client is created and runs on a background thread, not on the main Delphi thread, in a tight loop with no sleep states. It does do a PeekMessage() call in the loop to see if any new commands have come in that should be looped instead of the currently looping one. The reason for the GetMessage() call in the loop is so that the thread blocks when the robot is supposed to be idle, that is, no HTTP requests should be sent to it until the user decides to drive it again. In that case, posting a new command to the thread unblocks the GetMessage() call and the new command is looped.
I tried raising the thread priority to THREAD_PRIORITY_TIME_CRITICAL but that had absolutely zero effect. Note I did use GetThreadPriority() to make sure the priority was indeed raised and it returned a value of 15 after initially returning 0 before the SetThreadPriority() call.
1) So what can I do to improve the performance on these older low power systems, since several of my best users have them?
2) The other question I have is does anyone know if Indy has to rebuild the connection each HTTP request or does it cache the socket connection intelligently so that would not be the problem? Would it make a difference if I resorted to using a lower level Indy client socket and did the HTTP request crafting myself? I'd like to avoid that possibility since it would be a significant rewrite, but if that's a high certainty solution, let me know.
I have included the loop for the background thread below in case you can see anything inefficient. Commands to be executed are posted to the thread via an asynchronous PostThreadMessage() operation from the main thread.
// Does the actual post to the robot.
function doPost(
commandName, // The robot command (used for reporting purposes)
// commandString, // The robot command string (used for reporting purposes)
URL, // The URL to POST to.
userName, // The user name to use in authenticating.
password, // The password to use.
strPostData // The string containing the POST data.
: string): string;
var
RBody: TStringStream;
bRaiseException: boolean;
theSubstituteAuthLine: string;
begin
try
RBody := TStringStream.Create(strPostData);
// Custom HTTP request headers.
FIdHTTPClient.Request.CustomHeaders := TIdHeaderList.Create;
try
FIdHTTPClient.Request.Accept := 'text/html,application/xhtml+xml,application/xml;q=0.9,*/*;q=0.8';
FIdHTTPClient.Request.ContentType := 'application/xml';
FIdHTTPClient.Request.ContentEncoding := 'utf-8';
FIdHTTPClient.Request.CacheControl := 'no-cache';
FIdHTTPClient.Request.UserAgent := 'RobotCommand';
FIdHTTPClient.Request.CustomHeaders.Add('Connection: keep-alive');
FIdHTTPClient.Request.CustomHeaders.Add('Keep-Alive: timeout=30, max=3 header');
// Create the correct authorization line for the commands stream server.
theSubstituteAuthLine :=
basicAuthenticationHeaderLine(userName, password);
FIdHTTPClient.Request.CustomHeaders.Add(theSubstituteAuthLine);
Result := FIdHTTPClient.Post(URL, RBody);
// Let the owner component know the HTTP operation
// completed, whether the response code was
// successful or not. Return the response code in the long
// parameter.
PostMessageWithUserDataIntf(
FOwner.winHandleStable,
WM_HTTP_OPERATION_FINISHED,
POSTMESSAGEUSERDATA_LPARAM_IS_INTF,
TRovioCommandIsFinished.Create(
FIdHttpClient.responseCode,
commandName,
strPostData,
FIdHttpClient.ResponseText)
);
finally
FreeAndNil(RBody);
end; // try/finally
except
{
Exceptions that occur during an HTTP operation should not
break the Execute() loop. That would render this thread
inactive. Instead, call the background Exception handler
and only raise an Exception if requested.
}
On E: Exception do
begin
// Default is to raise an Exception. The background
// Exception event handler, if one exists, can
// override this by setting bRaiseException to
// FALSE.
bRaiseException := true;
FOwner.doBgException(E, bRaiseException);
if bRaiseException then
// Ok, raise it as requested.
raise;
end; // On E: Exception do
end; // try/except
end;
// The background thread's Excecute() method and loop (excerpted).
procedure TClientThread_roviosendcmd_.Execute;
var
errMsg: string;
MsgRec : TMsg;
theHttpCliName: string;
intfCommandTodo, intfNewCommandTodo: IRovioSendCommandsTodo_indy;
bSendResultNotification: boolean;
responseBody, S: string;
dwPriority: DWORD;
begin
// Clear the current command todo and the busy flag.
intfCommandTodo := nil;
FOwner.isBusy := false;
intfNewCommandTodo := nil;
// -------- BEGIN: THREAD PRIORITY SETTING ------------
dwPriority := GetThreadPriority(GetCurrentThread);
{$IFDEF THREADDEBUG}
OutputDebugString(PChar(
Format('Current thread priority for the the send-commands background thread: %d', [dwPriority])
));
{$ENDIF}
// On single CPU systems like our Dell laptop, the system appears
// to have trouble executing smooth motion. Guessing that
// the thread keeps getting interrupted. Raising the thread priority
// to time critical to see if that helps.
if not SetThreadPriority(GetCurrentThread, THREAD_PRIORITY_TIME_CRITICAL) then
RaiseLastOSError;
dwPriority := GetThreadPriority(GetCurrentThread);
{$IFDEF THREADDEBUG}
OutputDebugString(PChar(
Format('New thread priority for the the send-commands background thread after SetThreadPriority() call: %d', [dwPriority])
));
{$ENDIF}
// -------- END : THREAD PRIORITY SETTING ------------
// try
// Create the client Indy HTTP component.
theHttpCliName := '(unassigned)';
theHttpCliName := FOwner.Name + '_idhttpcli';
// 1-24-2012: Added empty component name check.
if theHttpCliName = '' then
raise Exception.Create('(TClientThread_roviosendcmd_.Execute) The client HTTP object is nameless.');
FIdHTTPClient := TIdHTTP.Create(nil);
{ If GetMessage retrieves the WM_QUIT, the return value is FALSE and }
{ the message loop is broken. }
while not Application.Terminated do
begin
try
bSendResultNotification := false;
// Reset the variable that detects new commands to do.
intfNewCommandTodo := nil;
{
If we are repeating a command, use PeekMessage so that if
there is nothing in the queue, we do not block and go
on repeating the command. Note, intfCommandTodo
becomes NIL after we execute a single-shot command.
If we are not repeating a command, use GetMessage so
it will block until there is something to do or we
quit.
}
if Assigned(intfCommandTodo) then
begin
// Set the busy flag to let others know we have a command
// to execute (single-shot or looping).
// FOwner.isBusy := true;
{
Note: Might have to start draining the queue to
properly handle WM_QUIT if we have problems with this
code.
}
// See if we have a new command todo.
if Integer(PeekMessage(MsgRec, 0, 0, 0, PM_REMOVE)) > 0 then
begin
// WM_QUIT?
if MsgRec.message = WM_QUIT then
break // We're done.
else
// Recover the command todo if any.
intfNewCommandTodo := getCommandToDo(MsgRec);
end; // if Integer(PeekMessage(MsgRec, FWndProcHandle, 0, 0, PM_REMOVE)) > 0 then
end
else
begin
// Not repeating a command. Block until something new shows
// up or we quit.
if GetMessage(MsgRec, 0, 0, 0) then
// Recover the command todo if any.
intfNewCommandTodo := getCommandToDo(MsgRec)
else
// GetMessage() returned FALSE. We're done.
break;
end; // else - if Assigned(intfCommandTodo) then
// Did we get a new command todo?
if Assigned(intfNewCommandTodo) then
begin
// ----- COMMAND TODO REPLACED!
// Update/Replace the command todo variable. Set the
// busy flag too.
intfCommandTodo := intfNewCommandTodo;
FOwner.isBusy := true;
// Clear the recently received new command todo.
intfNewCommandTodo := nil;
// Need to send a result notification after this command
// executes because it is the first iteration for it.
// (repeating commands only report the first iteration).
bSendResultNotification := true;
end; // if Assigned(intfNewCommandTodo) then
// If we have a command to do, make the request.
if Assigned(intfCommandTodo) then
begin
// Check for the clear command.
if intfCommandTodo.commandName = 'CLEAR' then
begin
// Clear the current command todo and the busy flag.
intfCommandTodo := nil;
FOwner.isBusy := false;
// Return the response as a simple result.
// FOwner.sendSimpleResult(newSimpleResult_basic('CLEAR command was successful'), intfCommandToDo);
end
else
begin
// ------------- SEND THE COMMAND TO ROVIO --------------
// This method makes the actual HTTP request via the TIdHTTP
// Post() method.
responseBody := doPost(
intfCommandTodo.commandName,
intfCommandTodo.cgiScriptName,
intfCommandTodo.userName_auth,
intfCommandTodo.password_auth,
intfCommandTodo.commandString);
// If this is the first or only execution of a command,
// send a result notification back to the owner.
if bSendResultNotification then
begin
// Send back the fully reconstructed response since
// that is what is expected.
S := FIdHTTPClient.Response.ResponseText + CRLF + FIdHTTPClient.Response.RawHeaders.Text + CRLF + responseBody;
// Return the response as a simple result.
FOwner.sendSimpleResult(newSimpleResult_basic(S), intfCommandToDo);
end; // if bSendResultNotification then
// If it is not a repeating command, then clear the
// reference. We don't need it anymore and this lets
// us know we already executed it.
if not intfCommandTodo.isRepeating then
begin
// Clear the current command todo and the busy flag.
intfCommandTodo := nil;
FOwner.isBusy := false;
end; // if not intfCommandTodo.isRepeating then
end; // if intfCommandTodo.commandName = 'CLEAR' then
end
else
// Didn't do anything this iteration. Yield
// control of the thread for a moment.
Sleep(0);
except
// Do not let Exceptions break the loop. That would render the
// component inactive.
On E: Exception do
begin
// Post a message to the component log.
postComponentLogMessage_error('ERROR in client thread for socket(' + theHttpCliName +'). Details: ' + E.Message, Self.ClassName);
// Return the Exception to the current EZTSI if any.
if Assigned(intfCommandTodo) then
begin
if Assigned(intfCommandTodo.intfTinySocket_direct) then
intfCommandTodo.intfTinySocket_direct.sendErrorToRemoteClient(exceptionToErrorObjIntf(E, PERRTYPE_GENERAL_ERROR));
end; // if Assigned(intfCommandTodo) then
// Clear the command todo interfaces to avoid looping an error.
intfNewCommandTodo := nil;
// Clear the current command todo and the busy flag.
intfCommandTodo := nil;
FOwner.isBusy := false;
end; // On E: Exception do
end; // try
end; // while not Application.Terminated do
To make use of HTTP keep-alives correctly, use FIdHTTPClient.Request.Connection := 'keep-alive' instead of FIdHTTPClient.Request.CustomHeaders.Add('Connection: keep-alive'), or set FIdHTTPClient.ProtocolVersion := pv1_1. At least that is how it works in Indy 10. I will double check Indy 9 when I get a chance.
Regardless of which version you use, the robot has to support keep-alives in the first place, or else TIdHTTP has no choice but to make a new socket connection for each request. If the robot sends an HTTP 1.0 response that does not include a Connection: keep-alive header, or an HTTP 1.1 response that includes a Connection: close header, then keep-alives are not supported.