I can't seem to find anywhere where I can restrict access for an application to read/write to a specific file/folder.
Within the permissions its just for all. - https://learn.microsoft.com/en-us/graph/permissions-reference
I have am Umbraco website, where on form submissios, it would write to a file in Sharepoint, however giving the application access rights to all files isn't entirely safe. If in the event the website gets compromised, they may be able to access files from an organisations Sharepoint.
If this cannot be achieved at all, are there any possible workarounds?
There's no good way to achieve this (yet) using application permissions (i.e. app-only permissions).
If the application is using delegated permissions (in other words, if the application is acting on behalf of the signed-in user), then the application can only access files which the user is also authorized to access. This is still not perfect, but it does reduce the scope of access when compared to every file in every site collection.
For various reasons (including security and stability), it is not recommended that applications use a non-interactive username/password flow to access Microsoft Graph (or any API).
That said, as a last resort you might consider using this flow with a dedicated non-privileged user account that only has access to the site collection/files in question. Only explore this option if you are completely confident that the application in question is highly trusted and secure (sicne the application will technically be able to do anything that user account can do, which in many cases includes a bunch of things you don't want anything other than very highly trusted apps to be able to do).