Highlight NSWindow under mouse cursor

Since this is quite a lot of code and it probably helps if there is a sample project where you can better understand the current problem I made a simple sample project which you can find on GitHub here: https://github.com/dehlen/Stackoverflow

I want to implement some functionality pretty similar what the macOS screenshot tool does. When the mouse hovers over a window the window should be highlighted. However I am having issues only highlighting the part of the window which is visible to the user.

Here is a screenshot of what the feature should look like:

My current implementation however looks like this:

My current implementation does the following:

1. Get a list of all windows visible on screen

static func all() -> [Window] {
        let options = CGWindowListOption(arrayLiteral: .excludeDesktopElements, .optionOnScreenOnly)
        let windowsListInfo = CGWindowListCopyWindowInfo(options, CGMainDisplayID()) //current window
        let infoList = windowsListInfo as! [[String: Any]]
        return infoList
            .filter { $0["kCGWindowLayer"] as! Int == 0 }
            .map { Window(
                frame: CGRect(x: ($0["kCGWindowBounds"] as! [String: Any])["X"] as! CGFloat,
                       y: ($0["kCGWindowBounds"] as! [String: Any])["Y"] as! CGFloat,
                       width: ($0["kCGWindowBounds"] as! [String: Any])["Width"] as! CGFloat,
                       height: ($0["kCGWindowBounds"] as! [String: Any])["Height"] as! CGFloat),
                applicationName: $0["kCGWindowOwnerName"] as! String)}
    }

2. Get the mouse location

private func registerMouseEvents() {
        NSEvent.addLocalMonitorForEvents(matching: [.mouseMoved]) {
            self.mouseLocation = NSEvent.mouseLocation
            return $0
        }
        NSEvent.addGlobalMonitorForEvents(matching: [.mouseMoved]) { _ in
            self.mouseLocation = NSEvent.mouseLocation
        }
    }

3. Highlight the window at the current mouse location:

static func window(at point: CGPoint) -> Window? {
        // TODO: only if frontmost
        let list = all()
        return list.filter { $0.frame.contains(point) }.first
    }
var mouseLocation: NSPoint = NSEvent.mouseLocation {
        didSet {
            //TODO: don't highlight if its the same window
            if let window = WindowList.window(at: mouseLocation), !window.isCapture {
                highlight(window: window)
            } else {
                removeHighlight()
            }
        }
    }

 private func removeHighlight() {
        highlightWindowController?.close()
        highlightWindowController = nil
    }

    func highlight(window: Window) {
        removeHighlight()
        highlightWindowController = HighlightWindowController()
        highlightWindowController?.highlight(frame: window.frame, animate: false)
        highlightWindowController?.showWindow(nil)
    }

class HighlightWindowController: NSWindowController, NSWindowDelegate {
    // MARK: - Initializers
    init() {
        let bounds = NSRect(x: 0, y: 0, width: 100, height: 100)
        let window = NSWindow(contentRect: bounds, styleMask: .borderless, backing: .buffered, defer: true)
        window.isOpaque = false
        window.level = .screenSaver
        window.backgroundColor = NSColor.blue
        window.alphaValue = 0.2
        window.ignoresMouseEvents = true
        super.init(window: window)
        window.delegate = self
    }

    // MARK: - Public API
    func highlight(frame: CGRect, animate: Bool) {
        if animate {
            NSAnimationContext.current.duration = 0.1
        }
        let target = animate ? window?.animator() : window
        target?.setFrame(frame, display: false)
    }
}

As you can see the window under the cursor is highlighted however the highlight window is drawn above other windows which might intersect.

Possible Solution I could iterate over the available windows in the list and only find the rectangle which does not overlap with other windows to draw the highlight rect only for this part instead of the whole window.

I am asking myself whether the would be a more elegant and more performant solution to this problem. Maybe I could solve this with the window level of the drawn HighlightWindow? Or is there any API from Apple which I could leverage to get the desired behavior?

Solution

I messed around with your code, and @Ted is correct. NSWindow.order(_:relativeTo) is exactly what you need.

Why NSWindow.level wont work:

Using NSWindow.level will not work for you because normal windows (like the ones in your screenshot) all have a window level of 0, or .normal. If you simply adjusted the window level to, say "1" for instance, your highlight view would appear above all the other windows. On the contrary, if you set it to "-1" your highlight view would appear below all normal windows, and above the desktop.

Problems to be introduced using NSWindow.order(_: relativeTo)

No great solution comes without caveats right? In order to use this method you will have to set the window level to 0 so it can be layerd among the other windows. However, this will cause your highlighting window to be selected in your WindowList.window(at: mouseLocation) method. And when it's selected, your if-statement removes it because it believes it's the main window. This will cause a flicker. (a fix for this is included in the TLDR below)

Also, if you attempt to highlight a window that does not have a level of 0, you will run into issues. To fix such issues you need to find the window level of the window you are highlighting and set your highlighting window to that level. (my code didn't include a fix for this problem)

In addition to the above problems, you need to consider what happens when the user hovers over a background window, and clicks on it without moving the mouse. What will happen is the background window will become front.. without moving the highlight window. A possible fix for this would be to update the highlight window on click events.

Lastly, I noticed you create a new HighlightWindowController + window every time the user moves their mouse. It may be a bit lighter on the system if you simply mutate the frame of an already exsisting HighlightWindowController on mouse movement (instead of creating one). To hide it you could call the NSWindowController.close() function, or even set the frame to {0,0,0,0} (not sure about the 2nd idea).

TLDR; Show us some code

Here's what I did.

1. Change your window struct to include a window number:

struct Window {
    let frame: CGRect
    let applicationName: String
    let windowNumber: Int

    init(frame: CGRect, applicationName: String, refNumber: Int) {
        self.frame = frame.flippedScreenBounds
        self.applicationName = applicationName
        self.windowNumber = refNumber
    }

    var isCapture: Bool {
        return applicationName.caseInsensitiveCompare("Capture") == .orderedSame
    }
}

2. In your window listing function ie static func all() -> [Window], include the window number:

refNumber: $0["kCGWindowNumber"] as! Int

3. In your window highlighting function, after highlightWindowController?.showWindow(nil), order the window relative to the window you are highlighting!

highlightWindowController!.window!.order(.above, relativeTo: window.windowNumber)

4. In your highlight controller, make sure to set the window level back to normal:

window.level = .normal

5. The window will now flicker, to prevent this, update your view controller if-statement:

    if let window = WindowList.window(at: mouseLocation) {
        if !window.isCapture {
            highlight(window: window)
        }
    } else {
        removeHighlight()
    }

Best of luck and have fun swifting!

Edit:

I forgot to mention, my swift version is 4.2 (haven't upgraded yet) so the syntax may be ever so slightly different.