Stripping effects on Delphi toolbuttons (TToolbutton)

Question

I'm attempting to switch from using Toolbar2000 to the regular toolbar because there doesn't seem to be a Delphi XE2 version and it looks like it uses some Assembly and I just don't really want to deal with it if I don't have to. (and I really like the fade-in effect with the Delphi Toolbar)

But, what I don't like is that the background of the button gets the regular blueish button treatment. I know how to change the color, but can I just not make the color change and not have a border painted around the button?

I've implemented the 'OnAdvancedCustomDrawButton' but the flags available don't seem to work right and I'm not sure how they interact with the gradient color and the hot track color and I wind up having some weird flashing or weird black backgrounds.

Here's how I'm creating the Toolbar

ToolBar1 := TToolBar.Create(Self);
ToolBar1.DoubleBuffered := true;
ToolBar1.OnAdvancedCustomDrawButton := Toolbar1CustomDrawButton;
ToolBar1.Transparent := false;
ToolBar1.Parent := Self;
ToolBar1.GradientEndColor := $7ca0c2; //RGB(194, 160, 124);
ToolBar1.GradientStartColor := $edeeed; //RGB(237, 238, 124);
ToolBar1.Indent := 5;
ToolBar1.Images := Normal;
ToolBar1.DrawingStyle := dsGradient;
ToolBar1.HotImages := Over;
ToolBar1.AutoSize := True;
ToolBar1.Visible := False;

and here's how I'm creating the buttons (in a loop):

ToolButton := TToolButton.Create(ToolBar1);
ToolButton.Parent := ToolBar1;
ToolButton.ImageIndex := ToolButtonImages[Index].ImageIndex;
ToolButton.OnClick := ToolButtonClick;

and here's my AdvancedCustomDrawButton function

procedure TMyForm.Toolbar1CustomDrawButton(Sender: TToolBar; Button: TToolButton;
    State: TCustomDrawState; Stage: TCustomDrawStage;
    var Flags: TTBCustomDrawFlags; var DefaultDraw: Boolean);
begin
  Flags := [tbNoEdges, tbNoOffset];
  DefaultDraw := True;
end;

Answer 1

Set drawing style of the toolbar to dsNormal and set Flags to [tbNoEdges] in custom draw handler.

update:

While the above works for 2K and XP, Vista and 7 seem to not to draw the border when button background is not drawn. Unfortunately achieving this with the VCL supplied TTBCustomDrawFlags is impossible, so we cannot get rid of the borders in a custom drawing handler.

If the toolbar is on the form itself we can put a handler for WM_NOTIFY since notification messages are sent to the parent window:

type
  TForm1 = class(TForm)
    ..
  private
    procedure WMNotify(var Msg: TWMNotify); message WM_NOTIFY;
    ..
  ..

procedure TForm1.WMNotify(var Msg: TWMNotify);
begin
  inherited;
  if (Msg.NMHdr.code = NM_CUSTOMDRAW) and
      Assigned(Toolbar1) and (Toolbar1.HandleAllocated) and
      (Msg.NMHdr.hwndFrom = ToolBar1.Handle) then

    case PNMTBCustomDraw(Msg.NMHdr).nmcd.dwDrawStage of
      CDDS_PREPAINT: Msg.Result := Msg.Result or CDRF_NOTIFYITEMDRAW;
      CDDS_ITEMPREPAINT: Msg.Result := TBCDRF_NOEDGES or TBCDRF_NOBACKGROUND;
                             // NOEDGES for 2K, XP, // NOBACKGROUND for Vista 7
    end;
end;

If the toolbar is parented in another window, like a panel, then we need to subclass the toolbar:

type
  TForm1 = class(TForm)
    ..
  private
    FSaveToolbarWndProc: TWndMethod;
    procedure ToolbarWndProc(var Msg: TMessage);
    ..
..

uses
  commctrl;

procedure TForm1.FormCreate(Sender: TObject);
begin
  ..
  FSaveToolbarWndProc := ToolBar1.WindowProc;
  ToolBar1.WindowProc := ToolbarWndProc;
end;

procedure TForm1.ToolbarWndProc(var Msg: TMessage);
begin
  FSaveToolbarWndProc(Msg);

  if (Msg.Msg = CN_NOTIFY) and
      (TWMNotify(Msg).NMHdr.hwndFrom = ToolBar1.Handle) and
      (TWMNotify(Msg).NMHdr.code = NM_CUSTOMDRAW) then begin

    case PNMTBCustomDraw(TWmNotify(Msg).NMHdr)^.nmcd.dwDrawStage of
      CDDS_PREPAINT: Msg.Result := CDRF_NOTIFYITEMDRAW;
      CDDS_ITEMPREPAINT: Msg.Result := TBCDRF_NOEDGES or TBCDRF_NOBACKGROUND;
    end;
  end;
end;

(Note that drawing style still needs to be dsNormal.)

With this solution you don't need to put a handler for custom drawing. But if you need/want to anyway, you might need to 'or' the Msg.Result with the one VCL's window procedure returns, i.e the 'case' would look like:

  CDDS_PREPAINT: Msg.Result := Msg.Result or CDRF_NOTIFYITEMDRAW;
  CDDS_ITEMPREPAINT: Msg.Result :=
                    Msg.Result or TBCDRF_NOEDGES or TBCDRF_NOBACKGROUND;

same goes for when we handle WM_NOTIFY on the form.

There may be other ways to achieve the same, custom drawing is a broad topic. If you want to delve into it, I suggest you to start from the links below for the problem at hand:

About Custom Draw
NM_CUSTOMDRAW (toolbar) notification code
NMCUSTOMDRAW structure
NMTBCUSTOMDRAW structure