I am writing a C# application to send emails with custom headers. If I send an email to someone with a custom header and they reply back, will it retain that custom header in the reply back?
Probably not.
The format or content of reply messages (as a special case of e-mail messages in general) is not standardized in any way; therefore, mail clients are free to include (or not) any part considered relevant. Headers are, in most cases, not preserved.
So: the header could be present in the reply, but it would be more of a coincidence than a certainty.
What you could do: embed some sort of identifier into the mail - e.g. into the Subject, the From address, or perhaps Reply-To, and save the specific headers you've sent on your side. Then, if a reply arrives, you'd be able to look up what they were for that particular message.