Say for example I wish to stream something over TCP, for instance a file share (bytes of a read file) or the output of a process (stdin, stderr, stdout) via a pipe using CreateProcess or _popen. I wouldn't be able to calculate the size in bytes of the streamed output. If I were to send a specific set of special characters to indicate the stream is over to be parsed by the server/client, would this be practical and secure?
If I were to send a specific set of special characters to indicate the stream is over to be parsed by the server/client, would this be practical and secure?
If the contents of the file can be arbitrary binary data, then, no, this is not by itself a “safe” way of doing it, because the “specific set of special characters” you use could appear somewhere in the file, so they would be sent over the connection, and the receiver would think the file has ended when it has not.
(I say this is not “safe,” meaning it is not a guaranteed way to communicate the file correctly, because it is not clear what you mean by “practical” or “secure.” Secure from what? Is there some attacker giving you files to transmit, so they can craft them in malicious ways? If all your files are known to contain only plain text ASCII characters, you could “safely” use a non-ASCII character as a sentinel.)
There are myriad ways of making a protocol to send arbitrary data. You could send the length first (although then you need to decide how many bits to use for the length, or design some protocol to send arbitrarily many bits for the length). You could send the data in blocks of, say, 512 bytes, and send each block with its length first. As long as that length is 512 bytes, it would mean another block for the same data is coming. Once the length is less than 512 bytes (including 0), it means this is the last block for the data. You could send the data as a quoted string, starting with ", then followed by the bytes of the data except that each " in the data is actually sent as \ then " and each \ in the data is sent as \ and \, and then the end of the data is indicated by a " (not preceded by a \ that escapes it).