Windows API: Simple window does not redraw correctly

I'm working on a Lua binding for Windows API. So far I've been able to create a basic window with a list box control:

require('Alien')
package.path = 'libs\\?.lua;libs\\?\\init.lua;' .. package.path

local function printf(...) io.write(string.format(...)) end

Windows = require('Windows')
local W = Windows

print("loaded OK")
print("Command line=", W.GetCommandLine())


local windowClassName = "testWindow"
local windowClass, window
local hInstance = assert(W.GetModuleHandle(nil))

assert(W.WM_CREATE)
assert(W.WM_CLOSE)
assert(W.WM_DESTROY)
assert(W.DefWindowProc)
assert(W.DestroyWindow)
assert(W.PostQuitMessage)

local msgID = {}
for k, v in pairs(Windows) do
    if k:match('^WM_') then msgID[v] = k end
end

local function wndProc(hWnd, msg, wParam, lParam)
    local id = msgID[msg] or ('%08X'):format(msg)
    printf('wndProc(%08X, %20s, %08X, %08X\n',
        hWnd,id, wParam, lParam)

    if msg == W.WM_CREATE then
        printf("window %08X create\n", hWnd)

    elseif msg == W.WM_CLOSE   then W.DestroyWindow(hWnd)
    elseif msg == W.WM_DESTROY then W.PostQuitMessage(0)
    else return W.DefWindowProc(hWnd, msg, wParam, lParam)
    end
    return 0
end


local wndProcCB = alien.callback(wndProc,
    assert(W.LRESULT), assert(W.HWND),
    assert(W.UINT), assert(W.WPARAM), assert(W.LPARAM))


windowClass = assert(W.WNDCLASSEX:new {
    cbSize        = assert(W.WNDCLASSEX.size),
    style         = bit.bor(W.CS_HREDRAW, W.CS_VREDRAW),
    lpfnWndProc   = assert(wndProcCB),
    cbClsExtra    = 0,
    cbWndExtra    = 0,
    hInstance     = assert(hInstance),
    hIcon         = nil,
    hCursor       = nil,
    hbrBackground = assert(W.COLOR_APPWORKSPACE)+1,
    lpszMenuName  = nil,
    lpszClassName = assert(windowClassName),
    hIconSm       = nil,
})
assert(W.RegisterClassEx(windowClass))


local dwExStyle = bit.bor(
    W.WS_EX_TOOLWINDOW,
    W.WS_EX_OVERLAPPEDWINDOW)

local dwStyle = bit.bor(
    W.WS_OVERLAPPEDWINDOW,
    W.WS_CLIPSIBLINGS,
    W.WS_CLIPCHILDREN)

window = assert(W.CreateWindowEx(
    assert(dwExStyle),        --dwExStyle
    assert(windowClassName),  --lpClassName
    "Test Window",            --lpWindowName
    assert(dwStyle),          --dwStyle
    assert(W.CW_USEDEFAULT),  --x
    assert(W.CW_USEDEFAULT),  --y
    420, 314,         --width, height
    nil, nil, assert(hInstance), nil)) --hWndParent, hMenu, hInstance, lpParam


local SWP_SHOW_ONLY = bit.bor(
    W.SWP_ASYNCWINDOWPOS,
    W.SWP_SHOWWINDOW,
    W.SWP_NOACTIVATE,
    W.SWP_NOMOVE,
    W.SWP_NOSIZE,
    W.SWP_NOZORDER)
W.SetWindowPos(assert(window), nil, 0, 0, 0, 0, assert(SWP_SHOW_ONLY))


local msg = assert(W.MSG:new())
while W.GetMessage(msg, 0, 0, 0) do
    W.TranslateMessage(msg)
    W.DispatchMessage(msg)
end

It creates and displays a window, but the window doesn't redraw correctly. From the console output I can see lots of WM_PAINT, WM_ERASEBKGND, WM_NCPAINT etc, which I pass on to DefWindowProc, but it seems to be not handling them.

Example screenshots: Example

When the window first appears, it either takes the image of whatever was behind it (as in this case) or appears solid gray. As it's dragged around, it keeps that image and anything that drags over it leaves behind a ghost image. (Can be seen on the titlebar, where I have a small white button following the active window around.) If I drag it off screen a bit, it starts to get even more glitchy but never successfully repaints itself.

I've tried various methods of handling WM_PAINT, WM_ERASEBKGND, etc but none have had any effect. Examples don't show these messages being handled either. Am I doing something wrong, or is there something else I need to be doing?

Solution

Well, I found the issue (I hope) with the help of Rohitab API Monitor. It was indeed a problem with the Lua binding: it had DefWindowProc declared as taking four int parameters. That's mostly fine, until we get a WM_ERASEBKGND message with a wParam (a handle to a device context) which is > 0x7FFFFFFF, and the call to DefWindowProc fails at converting that to an int and passes the wrong value. So the window never renders certain items to the correct device context. My debug logging (which wraps Windows API functions to print their parameters to the console) didn't catch this either, since it was logging the value before it got mangled.

Changing the declaration to use uint seems to have resolved it. (And maybe I can use API Monitor's definition files for my bindings, since the ones I have seem to be somewhat inaccurate.)