It wasn't totally clear from the developer site but can I use the API access with the self-hosted version of Quickbooks enterprise (on a Linux server in this case)? I see APIs for the cloud version and section on the desktop version but nothing about the server installed version.
No. You will need to use the QBXML SDK to access QB desktop data.
The integration will have to run on one of your Windows clients, not directly on the Linux machine. The Linux machine just hosts the data file, it doesn't really do anything else beyond that.