Rounding the corners of a tooltip window with DWM_WINDOW_CORNER_PREFERENCE makes the window a lot higher

Question

In my background app, I use the tiny tooltip window without the title or borders.

FormCreate() contains this code:

SetWindowLong(Application.Handle, GWL_EXSTYLE,GetWindowLong(Application.Handle, GWL_EXSTYLE) or WS_EX_TOOLWINDOW);
BorderIcons := BorderIcons - [biMaximize];

I am currently rounding the corners with these 2 code lines below, but the window is not 100% smooth, please see the sides of the sample window picture below (you may need to zoom the picture).

 RRShape := CreateRoundRectRgn(1, 1, Width - 1, height - 1, 100, 100);
 SetWindowRgn(Handle,RRShape,True);

I would now prefer to use the following code to round the corners under Win11 instead.

The problem is that after calling SetRoundedCorners(), my window becomes twice higher than in the picture above, a square, see below:

Why did this happen, and is there a solution? All I need is the smooth rounded corners, so the accepted answer is to fix the FormCreate() or the following helper function:

unit delphi_rounded_corners;

interface

uses
  Windows;

type
  TRoundedWindowCornerType = (RoundedCornerDefault, RoundedCornerOff, RoundedCornerOn, RoundedCornerSmall);

////////////////////////////////////////////////////////////////////////////
//
// Originally written by Ian Barker
//            https://github.com/checkdigits
//            https://about.me/IanBarker
//            [email protected]
//
// Based on an example in an answer during the RAD Studio 11 Launch Q & A
//
//
// Free software - use for any purpose including commercial use.
//
////////////////////////////////////////////////////////////////////////////
//
// Set or prevent Windows 11 from rounding the corners or your application
//
// Usage:
//         SetRoundedCorners(Self.Handle, RoundedCornerSmall);
//
////////////////////////////////////////////////////////////////////////////
///
procedure SetRoundedCorners(const TheHandle: HWND; const CornerType: TRoundedWindowCornerType);

implementation

uses
  Dwmapi;

const

  //
  // More information:
  //      https://docs.microsoft.com/en-us/windows/apps/desktop/modernize/apply-rounded-corners
  //      https://docs.microsoft.com/en-us/windows/win32/api/dwmapi/ne-dwmapi-dwmwindowattribute
  //      https://docs.microsoft.com/en-us/windows/win32/api/dwmapi/nf-dwmapi-dwmsetwindowattribute
  //

  DWMWCP_DEFAULT    = 0; // Let the system decide whether or not to round window corners
  DWMWCP_DONOTROUND = 1; // Never round window corners
  DWMWCP_ROUND      = 2; // Round the corners if appropriate
  DWMWCP_ROUNDSMALL = 3; // Round the corners if appropriate, with a small radius

  DWMWA_WINDOW_CORNER_PREFERENCE = 33; // [set] WINDOW_CORNER_PREFERENCE, Controls the policy that rounds top-level window corners

procedure SetRoundedCorners(const TheHandle: HWND; const CornerType: TRoundedWindowCornerType);
var
  DWM_WINDOW_CORNER_PREFERENCE: DWORD;
begin
  case CornerType of
    RoundedCornerOff:     DWM_WINDOW_CORNER_PREFERENCE := DWMWCP_DONOTROUND;
    RoundedCornerOn:      DWM_WINDOW_CORNER_PREFERENCE := DWMWCP_ROUND;
    RoundedCornerSmall:   DWM_WINDOW_CORNER_PREFERENCE := DWMWCP_ROUNDSMALL;
  else
    DWM_WINDOW_CORNER_PREFERENCE := DWMWCP_DEFAULT;
  end;

  // or simpler, since TRoundedWindowCornerType and
  // the DWM_WINDOW_CORNER_PREFERENCE enum have the
  // same numeric values:
  //
  // DWM_WINDOW_CORNER_PREFERENCE := Ord(CornerType);

  Dwmapi.DwmSetWindowAttribute(TheHandle, DWMWA_WINDOW_CORNER_PREFERENCE, @DWM_WINDOW_CORNER_PREFERENCE, sizeof(DWM_WINDOW_CORNER_PREFERENCE));
end;

end.

Answer 1

The classic Win32 approach

In my background app, I use the tiny tooltip window without the title or borders.

I assume you have set BorderStyle = bsNone at design time? Why, then, do you do BorderIcons := BorderIcons - [biMaximize]; in the form's OnCreate handler?

Firstly, if BorderStyle = bsNone, there is no title bar or system menu, so the BorderIcons property has no effect. Secondly, even if it did have an effect, why set BorderStyle at design time and BorderIcons at runtime?

It's also not clear why you feel the need to apply the WS_EX_TOOLWINDOW style if you have no window border. It won't make any difference, right? (Also, Application.Handle should be Self.Handle, or, equivalently, just Handle.)

If I create a new VCL app and a new form with BorderStyle = bsNone set at design time and only the following code in the form's OnCreate handler,

var RRShape := CreateRoundRectRgn(1, 1, Width - 1, Height - 1, 100, 100);
SetWindowRgn(Handle, RRShape, True);

then I do get something with perfectly rounded corners:

There's no anti-aliasing, because SetWindowRgn is an API from the late 1990s, and GUIs weren't very advanced back then. Remember the Windows 95 look? No anti-aliasing there.

Here's a close-up. No anti-aliasing here:

You don't define what you mean by "100% smooth", but IMHO, this is "100% smooth", at least if you accept the fact that there is no anti-aliasing.

The DWM approach

If, instead, I only use the following code in the form's OnCreate handler,

SetRoundedCorners(Handle, RoundedCornerOn)

then the modern DWM will create nice, rounded corners for me, and also a subtle border and shadow:

Close-up:

Here you can clearly see that the rounded corner is nicely anti-aliased. You can also see the subtle border line and the drop shadow.

Notice that I don't get any unexpected size change of my form.

Bonus question

Also, as a bonus, please think about your magic constant 100. What happens if the end user has a 200% DPI-scaled desktop?