Probably I could make an application domain and hook to DomainUnload event, but I'd like to do that on primary app domain.
There is no analogue in .net :(
You've got the right idea, you'll need to explicitly delete it.
// Delete the previously created file.
File.Delete(filepath);