How to export CMD output to a text file in C# after all the text printing on the CMD window is done?

Question

There is a lot of text printed on the terminal window from my C# code, some text is printed by an other app that I ran through C# System.Diagnostics (without RedirectStandardOutput neither I wanna use that async thing) and it printed it's own text and some text is printed by C# Console.WriteLine function.

I want to save all of that text from top to bottom to a text file. I don't want to execute any file and store it's text since all the execution is already done and all the text is printed already. I just want to save all of that text to a file at the end of the program.

NOTE: The following is not my real code (obviously) but it looks kind of like this.

from rich.progress import track
import time, os

print("Python Test")

for i in track(range(20), description="Processing..."):
    time.sleep(0.1)  # Simulate work being done

os.system("color 08")

using System.Diagnostics;


Console.WriteLine("Test");

// Create a new process instance
Process process = new Process();

// Configure the process using StartInfo
process.StartInfo.FileName = "cmd.exe";
process.StartInfo.Arguments = $"/c echo Hello world! & timeout /t 2 & python test.py";

// Start the process
process.Start();

// Wait for CMD to finish
process.WaitForExit();
Console.WriteLine("Test1");


/*
---- Save all of that above text that was printed here at the end of the code. ----
*/

I don't want to use RedirectStandardOutput because as far as I know if I do that and print the things from the process using those async things, first, it will not be able to print live updates such as in that timeout /t 2 part, it's also won't be able to account for change in color of the window color 08.

I want such a system that will let the code execute normally but when the code reaches the end it exports the text to a file much like how the now Windows Terminal's Export Text feature works.

This is what I want to achieve:

The terminal should work normally no change in how it works.

After the all the things are done at the end of my C# code I want to save all of that text to a text file.

Answer 1

Definition of STDIN, STDOUT, and STDERR

The OS' kernel of all operating systems use these 3 main I/O (Input/Output) streams, and these are STDIN, STDOUT, and STDERR. STDIN is the I/O stream that is processing information related to input, STDOUT is the I/O stream that is processing information related to output, and STDERR is the I/O stream that is processing information related to errors. All services, applications, and components of the OS' kernel use these streams to manipulate I/O information on any OS.

Solution using STDOUT redirection

In the code provided by you, the issue defined is that you want to process the information provided by the operation of a Batch script without redirecting the STDOUT stream. The ways in which the C# application can read the output of the Batch script is either by using files (e.g. JSON configuration files), Sockets, Pipes, and the STDOUT stream. The aforementioned methods are Inter-Process communication methods ( You can check this link for more information: https://stackoverflow.com/a/76196178/16587692 ). These methods must be used because there is no direct communication channel between the C# application and the Batch script. In this situation the most viable method is to use the STDOUT stream. You can preserve the console colors by changing them through the C# application rather than changing them by passing arguments to the Batch script. You can also write the output in real time to the desired log file, but as a word of caution, do not read from the log file while the C# application is writing to that file because it will trigger a Race Condition and this may corrupt your HDD/SSD cells where the file is located.

using System.Text;
using System.Diagnostics;


namespace Test
{
    class Program
    {

        static void Main(string[] args)
        {
            Operation().Wait();
            Console.ReadLine();
        }


        private static void SetPermissions(string file_name)
        {
            #pragma warning disable CA1416 // Validate platform compatibility

            // CHECK IF THE CURRENT OS IS WINDOWS
            if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Windows) == true)
            {
                // GET THE SPECIFIED FILE INFORMATION OF THE SELECTED FILE
                FileInfo settings_file_info = new FileInfo(file_name);

                // GET THE ACCESS CONTROL INFORMATION OF THE SELECTED FILE AND STORE THEM IN A 'FileSecurity' OBJECT
                System.Security.AccessControl.FileSecurity settings_file_security = settings_file_info.GetAccessControl();

                // ADD THE ACCESS RULES THAT ALLOW READ, WRITE, AND DELETE PERMISSIONS ON THE SELECTED FILE FOR THE CURRENT USER
                settings_file_security.AddAccessRule(new System.Security.AccessControl.FileSystemAccessRule(System.Security.Principal.WindowsIdentity.GetCurrent().Name, System.Security.AccessControl.FileSystemRights.Write, System.Security.AccessControl.AccessControlType.Allow));
                settings_file_security.AddAccessRule(new System.Security.AccessControl.FileSystemAccessRule(System.Security.Principal.WindowsIdentity.GetCurrent().Name, System.Security.AccessControl.FileSystemRights.Read, System.Security.AccessControl.AccessControlType.Allow));
                settings_file_security.AddAccessRule(new System.Security.AccessControl.FileSystemAccessRule(System.Security.Principal.WindowsIdentity.GetCurrent().Name, System.Security.AccessControl.FileSystemRights.Delete, System.Security.AccessControl.AccessControlType.Allow));
                
                // UPDATE THE ACCESS CONTROL SETTINGS OF THE FILE BY SETTING THE 
                // MODIFIED ACCESS CONTROL SETTINGS AS THE CURRENT SETTINGS
                settings_file_info.SetAccessControl(settings_file_security);
            }
            else
            {
                // IF THE OS IS A UNIX BASED OS, SET THE FILE PERMISSIONS FOR READ AND WRITE OPERATIONS
                // WITH THE 'UnixFileMode.UserRead | UnixFileMode.UserWrite' BITWISE 'OR' OPERATION
                File.SetUnixFileMode(file_name, UnixFileMode.UserRead | UnixFileMode.UserWrite);
                
            }
            #pragma warning restore CA1416 // Validate platform compatibility
        }



        private static async Task<bool> Operation()
        {
            // Process object
            System.Diagnostics.Process proc = new System.Diagnostics.Process();

            string file_name = String.Empty;
            string arguments = String.Empty;

            // CHECK IF THE CURRENT OS IS WINDOWS AND SET THE FILE PATH AND ARGUMETS ACCORDINGLY
            if(System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Windows) == true)
            {
                file_name = @"C:\Users\teodo\PycharmProjects\Test\.venv\Scripts\python.exe";
                arguments = @"C:\Users\teodo\PycharmProjects\Test\main.py";
            }
            else
            {
                file_name = @"python3";
                arguments = @"/mnt/c/Users/teodo/PycharmProjects/Test/main.py";
            }

            // Path where the python executable is located
            proc.StartInfo.FileName = file_name;

            // Path where python executable is located
            proc.StartInfo.Arguments = arguments;

            // Start the process
            proc.Start();


            // Named pipe server object with an "In" direction. This means that this pipe can only read messages. On Windows it creates a pipe at the 
            // '\\.\pipe\pipe-sub-directory\pipe-name' virtual directory, on Linux it creates a Unix Named Socket in the '/tmp' directory 
            System.IO.Pipes.NamedPipeServerStream fifo_pipe_connection = new System.IO.Pipes.NamedPipeServerStream("/tmp/fifo_pipe", System.IO.Pipes.PipeDirection.In);

            // Create a backlog text file if none is existent, set its permissions as Read/Write,
            // and create a stream that allows direct read and write operations to the file
            FileStream fs = File.Open("backlog.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite);
            SetPermissions("backlog.txt");

            try
            {
                // Wait for a client to connect synchronously
                fifo_pipe_connection.WaitForConnection();

                while (true)
                {
                    // Initiate a binary buffer byte array with a size of 1 Kb 
                    byte[] buffer = new byte[1024];

                    // Read the received bytes into the buffer byte array and also 
                    // store the number of bytes read into an integer named 'read'
                    int read = await fifo_pipe_connection.ReadAsync(buffer, 0, buffer.Length);

                    // Write the received bytes into the 'backlog.txt' file
                    await fs.WriteAsync(buffer, 0, read);

                    // Flush the bytes within the stream's buffer into the file
                    await fs.FlushAsync();

                    // If the number of bytes read is equal to '0' there are no bytes
                    // left to read on the Pipe's stream and the read loop is closed
                    if (read == 0)
                    {
                        break;
                    }
                }

            }
            catch
            {
            }
            finally
            {
                fs?.DisposeAsync();
                fifo_pipe_connection?.DisposeAsync();
            }
            return true;
        }
    }
}

STDOUT redirection method output

Definition of FIFO Pipes

FIFO Pipes, also known as Named Pipes, are a type of socket that use the operating system's file system in order to facilitate information exchange between applications. FIFO Pipes are categorised into 3 categories and these are, Inward Pipes, Outward Pipes, and Two-way Pipes. Inward Pipes are pipes on which the Pipe Server can only receive information from the Pipe Clients, Outward Pipes are pipes on which the Pipe Server can only send information to the Pipe Clients, and Two-way Pipes are pipes on which the Pipe Server can both send and receive information to and from the Pipe Clients.

Cross-platform solution using FIFO Pipes

If the integrity of the output must be preserved, FIFO pipes are the best option for an inter-process communication method. In this scenario an Inward Pipe will be the best solution due to the fact that the C# application must receive information from the Python application. In order to have cross-platform capabilities, both the C# and Python application use conditional statements to verify which OS platform the applications are running on.

C# Application

using System.Text;
using System.Diagnostics;


namespace Test
{
    class Program
    {

        static void Main(string[] args)
        {
            Operation().Wait();
            Console.ReadLine();
        }


        private static void SetPermissions(string file_name)
        {
            #pragma warning disable CA1416 // Validate platform compatibility

            // CHECK IF THE CURRENT OS IS WINDOWS
            if (System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Windows) == true)
            {
                // GET THE SPECIFIED FILE INFORMATION OF THE SELECTED FILE
                FileInfo settings_file_info = new FileInfo(file_name);

                // GET THE ACCESS CONTROL INFORMATION OF THE SELECTED FILE AND STORE THEM IN A 'FileSecurity' OBJECT
                System.Security.AccessControl.FileSecurity settings_file_security = settings_file_info.GetAccessControl();

                // ADD THE ACCESS RULES THAT ALLOW READ, WRITE, AND DELETE PERMISSIONS ON THE SELECTED FILE FOR THE CURRENT USER
                settings_file_security.AddAccessRule(new System.Security.AccessControl.FileSystemAccessRule(System.Security.Principal.WindowsIdentity.GetCurrent().Name, System.Security.AccessControl.FileSystemRights.Write, System.Security.AccessControl.AccessControlType.Allow));
                settings_file_security.AddAccessRule(new System.Security.AccessControl.FileSystemAccessRule(System.Security.Principal.WindowsIdentity.GetCurrent().Name, System.Security.AccessControl.FileSystemRights.Read, System.Security.AccessControl.AccessControlType.Allow));
                settings_file_security.AddAccessRule(new System.Security.AccessControl.FileSystemAccessRule(System.Security.Principal.WindowsIdentity.GetCurrent().Name, System.Security.AccessControl.FileSystemRights.Delete, System.Security.AccessControl.AccessControlType.Allow));
                
                // UPDATE THE ACCESS CONTROL SETTINGS OF THE FILE BY SETTING THE 
                // MODIFIED ACCESS CONTROL SETTINGS AS THE CURRENT SETTINGS
                settings_file_info.SetAccessControl(settings_file_security);
            }
            else
            {
                // IF THE OS IS A UNIX BASED OS, SET THE FILE PERMISSIONS FOR READ AND WRITE OPERATIONS
                // WITH THE 'UnixFileMode.UserRead | UnixFileMode.UserWrite' BITWISE 'OR' OPERATION
                File.SetUnixFileMode(file_name, UnixFileMode.UserRead | UnixFileMode.UserWrite);
                
            }
            #pragma warning restore CA1416 // Validate platform compatibility
        }



        private static async Task<bool> Operation()
        {
            // Process object
            System.Diagnostics.Process proc = new System.Diagnostics.Process();

            string file_name = String.Empty;
            string arguments = String.Empty;

            // CHECK IF THE CURRENT OS IS WINDOWS AND SET THE FILE PATH AND ARGUMETS ACCORDINGLY
            if(System.Runtime.InteropServices.RuntimeInformation.IsOSPlatform(System.Runtime.InteropServices.OSPlatform.Windows) == true)
            {
                file_name = @"C:\Users\teodo\PycharmProjects\Test\.venv\Scripts\python.exe";
                arguments = @"C:\Users\teodo\PycharmProjects\Test\main.py";
            }
            else
            {
                file_name = @"python3";
                arguments = @"/mnt/c/Users/teodo/PycharmProjects/Test/main.py";
            }

            // Path where the python executable is located
            proc.StartInfo.FileName = file_name;

            // Path where python executable is located
            proc.StartInfo.Arguments = arguments;

            // Start the process
            proc.Start();


            // Named pipe server object with an "In" direction. This means that this pipe can only read messages. On Windows it creates a pipe at the 
            // '\\.\pipe\pipe-sub-directory\pipe-name' virtual directory, on Linux it creates a Unix Named Socket in the '/tmp' directory 
            System.IO.Pipes.NamedPipeServerStream fifo_pipe_connection = new System.IO.Pipes.NamedPipeServerStream("/tmp/fifo_pipe", System.IO.Pipes.PipeDirection.In);

            // Create a backlog text file if none is existent, set its permissions as Read/Write,
            // and create a stream that allows direct read and write operations to the file
            FileStream fs = File.Open("backlog.txt", FileMode.OpenOrCreate, FileAccess.ReadWrite);
            SetPermissions("backlog.txt");

            try
            {
                // Wait for a client to connect synchronously
                fifo_pipe_connection.WaitForConnection();

                while (true)
                {
                    // Initiate a binary buffer byte array with a size of 1 Kb 
                    byte[] buffer = new byte[1024];

                    // Read the received bytes into the buffer byte array and also 
                    // store the number of bytes read into an integer named 'read'
                    int read = await fifo_pipe_connection.ReadAsync(buffer, 0, buffer.Length);


                    // Print the bytes sent by the Python application on the pipe
                    Console.WriteLine(Encoding.UTF8.GetString(buffer));

                    // Write the received bytes into the 'backlog.txt' file
                    await fs.WriteAsync(buffer, 0, read);

                    // Flush the bytes within the stream's buffer into the file
                    await fs.FlushAsync();

                    // If the number of bytes read is equal to '0' there are no bytes
                    // left to read on the Pipe's stream and the read loop is closed
                    if (read == 0)
                    {
                        break;
                    }
                }

            }
            catch
            {
            }
            finally
            {
                fs?.DisposeAsync();
                fifo_pipe_connection?.DisposeAsync();
            }
            return true;
        }
    }
}

Python Application

import os
import sys
import time
from rich.progress import track
import platform
import socket

fifo_write = None
unix_named_pipe = None


def operation():
    try:
        write("!!! Python Test !!!\n\n")

        # Simulate work being done
        set_range = range(20)
        for i in track(set_range, description="Processing..."):
            time.sleep(0.1)

            # SEND THE CURRENT PROGRESS AS A PERCENTAGE OVER THE PIPE
            write("Processing..." + str((100 / set_range.stop) * (i + 1)) + "%\n")

        # CHANGE THE TERMINAL COLOR TO GREY
        if platform.system() == "Windows":
            os.system("color 08")
        else:
            print("\n\n")
            os.system(r"echo '\e[91m!!! COLOR !!!'")

        # PAUSE THE CURRENT THREAD FOR 2 SECONDS
        time.sleep(2)

        # CHANGE THE TERMINAL COLOR TO WHITE
        if platform.system() == "Windows":
            os.system("color F")
        else:
            os.system(r"echo '\e[00m!!! COLOR !!!'")
            print("\n\n")
        write("[ Finished ]")
    except KeyboardInterrupt:
        sys.exit(0)


def write(msg):
    if platform.system() == "Windows":
        fifo_pipe_write(msg)
    else:
        unix_named_socket_write(msg)


def fifo_pipe_write(msg):
    try:
        global fifo_write
        if fifo_write is not None:
            # WRITE THE STRING PASSED TO THE FUNCTION'S AS AN ARGUMENT
            # IN THE FIFO PIPE FILE USING THE GLOBAL STREAM
            fifo_write.write(msg)
    except KeyboardInterrupt:
        sys.exit(0)


def unix_named_socket_write(msg):
    try:
        global unix_named_pipe
        if unix_named_pipe is not None:
            unix_named_pipe.send(str(msg).encode(encoding="utf-8"))
    except KeyboardInterrupt:
        pass


def stream_finder() -> bool:
    is_found = False

    try:
        # INITIATE A PIPE SEARCH SEQUENCE FOR 10 SECONDS
        for t in range(0, 10):
            try:
                try:
                    # IF THE OS IS WINDOWS SEARCH FOR A PIPE FILE
                    if platform.system() == "Windows":
                        # IF PIPE IS FOUND RETURN TRUE AND STORE THE OPENED PIPE FILE STREAM GLOBALLY
                        global fifo_write
                        fifo_write = open(r"\\.\pipe\tmp\fifo_pipe", "w")
                        is_found = True
                        break
                    # ELSE, SEARCH FOR A NAMED UNIX SOCKET
                    else:
                        # IF SOCKET IS FOUND RETURN TRUE AND STORE THE OPENED SOCKET GLOBALLY
                        global unix_named_pipe
                        unix_named_pipe = socket.socket(socket.AF_UNIX, socket.SOCK_STREAM)
                        unix_named_pipe.connect("/tmp/fifo_pipe")
                        is_found = True
                        break
                except FileNotFoundError:
                    # IF PIPE IS NOT FOUND
                    print("\n\n[ Searching pipe ]")
            except OSError:
                # IF PIPE IS NOT FOUND
                print("\n\n[ Searching pipe ]")

            # MAKE THE LOOP WAIT 1 SECOND FOR EACH ITERATION
            time.sleep(1)
    except KeyboardInterrupt:
        sys.exit(0)

    return is_found


if __name__ == "__main__":
    try:
        # INITIATE THE PIPE SEARCHING OPERATION
        found = stream_finder()
        # IF PIPE SEARCHING OPERATION IS SUCCESSFUL
        if found is True:
            print("\n\n[ Pipe found ]\n\n")
            # INITIATE THE MAIN OPERATION
            operation()
    except KeyboardInterrupt:
        sys.exit(0)

FIFO Pipes output Windows

FIFO Pipes output Linux

Linux OS Details

Linux OS Output