how to read non Latin Charactes inside memory stream that sent from iDTCPServer?

Question

I have this code that insert some none latin charcter and then send it from IDTCPserver To idTcpclient

LVU := TStringList.Create;
LVU.Add('مرحبا');
try

  if (LVU.Count > 0) then
  begin
    memorylist := TMemoryStream.Create;
    try
      LVU.SaveToStream(memorylist);
      memorylist.Position := 0;
      AContext.Connection.IOHandler.DefStringEncoding := IndyTextEncoding_UTF8;
      AContext.Connection.IOHandler.LargeStream := true;
      AContext.Connection.IOHandler.Write(memorylist, 0, true);
    finally
      memorylist.Free;
    end;
  end;
finally
  LVU.Free;
end;

but when client receive it, it recive the text inside ms as '???????' what could be the issue ? I already set connection encoding to utf8

I run this server on Ubuntu. When I try the server on windows I got the text normally.

And this is how I read stream on client side

FMSFROMCALL := TMemoryStream.Create;
FTCP.Socket.LargeStream := True;
FTCP.Socket.ReadStream(FMSFROMCALL, -1, false);
FMSFROMCALL.Position := 0;

Answer 1

Your assignment of DefStringEncoding does not apply to the TIdIOHandler.Write(TStream) method. It only applies to IOHandler methods that convert between string <-> bytes, but your TStream is already in raw bytes so there is no string conversion involved.

To do what you are attempting, you need to pass TEncoding.UTF8 to the TStringList.SaveToStream() method:

LVU := TStringList.Create;
try
  LVU.Add('مرحبا');
  memorylist := TMemoryStream.Create;
  try
    LVU.SaveToStream(memorylist, TEncoding.UTF8);
    memorylist.Position := 0;
    AContext.Connection.IOHandler.LargeStream := true;
    AContext.Connection.IOHandler.Write(memorylist, 0, true);
  finally
    memorylist.Free;
  end;
finally
  LVU.Free;
end;

When you do not pass an encoding to SaveToStream(), it gets encoded using a default encoding, which in your case is one that does not support Arabic, so that is where the ??????? comes from.

An alternative is to send a TIdBytes instead of a TMemoryStream:

 Bytes := IndyTextEncoding_UTF8.GetBytes('مرحبا');
 AContext.Connection.IOHandler.WriteInt64(Length(Bytes));
 AContext.Connection.IOHandler.Write(Bytes);

Or, just send the string as-is with DefStringEncoding assigned:

s := 'مرحبا';
AContext.Connection.IOHandler.DefStringEncoding := IndyTextEncoding_UTF8;
AContext.Connection.IOHandler.WriteInt64(IndyTextEncoding_UTF8.GetByteCount(s));
AContext.Connection.IOHandler.Write(s);