I'm working on a USB device that will talk to a single application. It looks like we may want to have a Windows driver that presents a nicer software interface to the application. (As opposed to having the application itself send lower-level commands to the device via WinUSB.)
Is it possible to use WinUSB from within a DLL? Choosing a driver model for developing a USB client driver doesn't address that specifically.
Are there reasons in this situation that I should instead consider writing a UMDF-based or KMDF-based driver, or a hybrid driver that calls WDM routines?
Using WinUSB is probably the right way to go. You can definitely use WinUSB within a DLL. In general, you can write a DLL that calls functions in another DLL, and there is nothing special about winusb.dll that prevents you from doing that. Also, it is already done in other projects like libusb and libusbp, which compile to a DLL that uses winusb.dll.
I would also encourage you to make your code cross-platform: don't call WinUSB directly from your DLL, but instead use a USB abstraction library such as libusb or libusbp. Even if you only want to support Windows, these libraries are lot easier to use than SetupAPI and WinUSB, so they should save development time. They will also save a lot of time if you ever want your code to work on different operating systems.
I think the only reason to write your own UMDF or KMDF driver in a situation like this is if you need advanced features of the Windows USB stack that are not supported by WinUSB. For instance, if you needed to switch your device to a different USB configuration, or do tricky stuff with power management, or allow multiple applications to use the device at once. If you just want to send some data back and forth, WinUSB is a fine choice.