Currently I create a GUID-key in C#, to place it on a USB flash drive. I use this key for user identification. The key itself is inside a .txt-file, which is easy to manipulate. Is there a way to make the .txt-file undeleteable and invisible but still readable?
No.
Imagine what would happen if this were possible. Yeah.
Make the file hidden - maybe hidden+system. That's the best you can do. If the user deletes this anyway, well, that's his problem. You did the contractual "hiding", and you can't be responsible for the actions of a user who knowingly overcomes those safety precautions.