Strange behaviour with DrawTextA, Courier New and Japanese locale

Question

I found some strange behaviour when using DrawTextA in combination with the Courier New-font with a Japanese locale. Consider the following Delphi XE2 code:

procedure PaintTexts(aPaintBox: TPaintBox; aCharset: Byte);
var
  A: AnsiString;
  S: string;
  R: TRect;
begin
  aPaintBox.Font.Charset := aCharset;

  A := '[DrawTextA] The word "Japan" in Japanese: 日本';
  R := Rect(0, 0, aPaintBox.Width, aPaintBox.Height);
  DrawTextA(aPaintBox.Canvas.Handle, PAnsiChar(A), Length(A), R, 0);

  S := '[DrawTextW] The word "Japan" in Japanese: 日本';
  R := Rect(0, 20, aPaintBox.Width, aPaintBox.Height);
  DrawTextW(aPaintBox.Canvas.Handle, PWideChar(S), Length(S), R, 0);
end;

procedure TForm1.PaintBox1Paint(Sender: TObject);
begin
  PaintTexts(PaintBox1, DEFAULT_CHARSET);
end;

procedure TForm1.PaintBox2Paint(Sender: TObject);
begin
  PaintTexts(PaintBox2, SHIFTJIS_CHARSET);
end;

In this code, Form1 contains two paintboxes (PaintBox1 and PaintBox2). The font of Form1 is set to Courier New, the two paintboxes have set ParentFont to True. The non-Unicode locale of Windows is set to Japanese (Japan), so it is working with codepage 932.

This what the result looks like:

The first paintbox shows the output of a DrawTextA and a DrawTextW call with a Charset property CHARSET_DEFAULT. This is the default value of the font's charset property. Note that the japanese word 日本 is not shown correctly when passed to DrawTextA. However, DrawTextW draws it perfectly.

The second paintbox shows the same texts, but only with the Charset property changed to SHIFTJIS_CHARSET. Now both calls show the correct japanese characters. But the font has changed to a variable width font!

When I change the font of Form1 to Tahoma, both DrawTextA and DrawTextW show the same correct texts.

Does anyone know why DrawTextA behaves different than DrawTextW when my non-Unicode locale is set to Japanese and my font is set to Courier New?

I always thought that the only difference between the Ansi- and Wide-versions of Windows API's was that the Ansi-versions handled the conversion to and from Unicode.

I have tried this in combination with Windows XP and Windows 7, and Delphi 7 and Delphi XE2. All combinations show the same behaviour.

Update: After David Heffernan posted his answer, I started reading Micheal Kaplan's blog. There I found a similar topic and also more information about this topic.

Answer 1

DrawTextA does not convert the text to Unicode. Instead the selected font's charset is used to interpret the supplied text. This is indeed somewhat more complex than the typical A and W suffixed API functions.

The use of the font charset allowed non-Unicode programs to display text in multiple character sets. For a Unicode program this is a complete non-issue because Unicode can encode all characters.

According to Michael Kaplan in this forum thread, DEFAULT_FONTSET should not be used. He says:

Do not use DEFAULT_CHARSET at all. It is evil.

If you need to specify a charset, you should do the following:

1. Call GetACP to obtain the active code page.
2. Call TranslateCharsetInfo passing the code page and specify the TCI_SRCCODEPAGE flag.

The charset info information that is returned is the appropriate charset to use for the active code page. Wrap it up like this:

function CharsetFromCP(CP: UINT): UINT;
var
  csi: TCharsetInfo;
begin
  Win32Check(TranslateCharsetInfo(CP, csi, TCI_SRCCODEPAGE));
  Result := csi.ciCharset;
end;

And then you can write:

aPaintBox.Font.Charset := CharsetFromCP(GetACP);

Of course, if you know the text is Japanese then you can write SHIFTJIS_CHARSET directly. And even more obviously, you can simply use the Unicode API and avoid all this nonsense.