Search code examples
c#winapipipe32bit-64bit

Problem with StdIN/StdOUT pipes when parent process is 64bits


I made a class to start child process that inherit new pipes for standard input/output/error. All is working fine in 32bits: I can write in child StdIn and read child StdOut/Err without problem (the child process can also read the new StdIn pipe and write in the new stdOut/Err pipes).

But, if I compile my parent process in 64bits, the child process (32 and 64bits) cannot read the new pipes.

Parent | Child | RedirectPipes | Result (In Child process)
32bits | 32/64 | In+Out        | GOOD
64bits | 32/64 | In+Out        | Access Denied for StdIn (Console.ReadLine)
64bits | 32/64 | In            | *** No error but no data for StdIn.

*** When I don't redirect the Output pipe, I can manually write (with my keyboard) in the new window and the child receives that data. So the stdIn isn't redirect.

In all case, no error in parent process

I tried to adjust the SecurityDescriptor of the SECURITY_ATTRIBUTES but without success. I know that the SECURITY_ATTRIBUTES structure has a different size in 64bits but I'm not sure if it can be a problem and how to manage that.

Do you have any suggestions? Questions?

Thank you

If you want to test, I made a smaller project with only the minimal.

Parent code :

using System;
using System.IO;
using System.Runtime.ConstrainedExecution;
using System.Runtime.InteropServices;
using System.Security;
using System.Security.Permissions;
using Microsoft.Win32.SafeHandles;

namespace Shell.TestShell
{
    class TestShell
    {
        static void Main()
        {
            var OneShell = new Shell2();
        }
    }

    class Shell2
    {
        public const Int32 STARTF_USESTDHANDLES = 0x100;
        public const Int32 STARTF_USESHOWWINDOW = 1;

        public const UInt16 SW_SHOW = 5;
        public const UInt16 SW_HIDE = 0;

        [Flags()]
        public enum CreateProcessFlags
        {
            CREATE_SUSPENDED = 0x4,
            DETACHED_PROCESS = 0x8,
            CREATE_DEFAULT_ERROR_MODE = 0x4000000,

            CREATE_NEW_CONSOLE = 0x10,
            CREATE_NEW_PROCESS_GROUP = 0x200,
            CREATE_NO_WINDOW = 0x8000000,
            CREATE_SEPARATE_WOW_VDM = 0x800,
            CREATE_UNICODE_ENVIRONMENT = 0x400,
            
            IDLE_PRIORITY_CLASS = 0x40,
            BELOW_NORMAL_PRIORITY_CLASS = 0x4000,
            ABOVE_NORMAL_PRIORITY_CLASS = 0x8000,
            NORMAL_PRIORITY_CLASS = 0x20,
            HIGH_PRIORITY_CLASS = 0x80,
            REALTIME_PRIORITY_CLASS = 0x100
        }
        [StructLayout(LayoutKind.Sequential, CharSet = CharSet.Auto)]
        public struct StartupInfo
        {
            public int cb;
            public String reserved;
            public String desktop;
            public String title;
            public int x;
            public int y;
            public int xSize;
            public int ySize;
            public int xCountChars;
            public int yCountChars;
            public int fillAttribute;
            public int flags;
            public UInt16 showWindow;
            public UInt16 reserved2;
            public byte reserved3;
            public SafeFileHandle hStdInput;
            public SafeFileHandle hStdOutput;
            public SafeFileHandle hStdError;
        }


        public struct ProcessInformation
        {
            public IntPtr process;
            public IntPtr thread;
            public int processId;
            public int threadId;
        }

        [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        public static extern bool CreateProcess(string lpApplicationName,
                                                string lpCommandLine,
                                                IntPtr lpProcessAttributes,
                                                IntPtr lpThreadAttributes,
                                                bool bInheritHandles,
                                                CreateProcessFlags dwCreationFlags,
                                                IntPtr lpEnvironment,
                                                string lpCurrentDirectory,
                                                ref StartupInfo lpStartupInfo,
                                                out ProcessInformation lpProcessInformation);

        [StructLayout(LayoutKind.Sequential)]
        public struct SECURITY_ATTRIBUTES
        {
            public int Length;
            public IntPtr SecurityDescriptor;
            public bool InheritHandle;
        }

        [DllImport("kernel32.dll", CharSet = CharSet.Auto, SetLastError = true)]
        public static extern bool CreatePipe(out SafeFileHandle hReadPipe, out SafeFileHandle hWritePipe, ref SECURITY_ATTRIBUTES lpPipeAttributes, uint nSize);


        public Shell2()
        {
            StartupInfo Shell2StartupInfo = new StartupInfo();
            Shell2StartupInfo.flags       = STARTF_USESTDHANDLES | STARTF_USESHOWWINDOW;
            Shell2StartupInfo.showWindow  = SW_SHOW; // SW_SHOW for testing only
            Shell2StartupInfo.reserved    = null;
            Shell2StartupInfo.cb          = Marshal.SizeOf(Shell2StartupInfo);


            SECURITY_ATTRIBUTES lpPipeAttributesInput = new SECURITY_ATTRIBUTES();
            lpPipeAttributesInput.InheritHandle       = true;
            lpPipeAttributesInput.Length              = Marshal.SizeOf(lpPipeAttributesInput);
            lpPipeAttributesInput.SecurityDescriptor  = IntPtr.Zero;


            // Parent pipes
            SafeFileHandle StandardInputWriteHandle;
            SafeFileHandle StandardOutputReadHandle;

            // Child pipes
            SafeFileHandle StandardInputReadHandle;
            SafeFileHandle StandardOutputWriteHandle;

            // New pipes for StdIN
            if (!CreatePipe(out StandardInputReadHandle, out StandardInputWriteHandle, ref lpPipeAttributesInput, 0))
                throw new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error());
            // New pipes for StdOUT
            if (!CreatePipe(out StandardOutputReadHandle, out StandardOutputWriteHandle, ref lpPipeAttributesInput, 0))
                throw new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error());


            //  Redirect child pipes
            Shell2StartupInfo.hStdInput  = StandardInputReadHandle;
            Shell2StartupInfo.hStdOutput = StandardOutputWriteHandle;
            //Shell2StartupInfo.hStdOutput = new SafeFileHandle(IntPtr.Zero, false);
            Shell2StartupInfo.hStdError  = new SafeFileHandle(IntPtr.Zero, false);


            String PathProgram;

            // My testing child .Net Application
            //PathProgram = @"C:\Temp\ConsoleEcho32.exe";
            //PathProgram = @"C:\Temp\ConsoleEcho64.exe";

            // cmd.exe same platform (32/64bits) as parent process  
            PathProgram = @"C:\Windows\System32\cmd.exe";

            // Force 32 bits cmd.exe child from 64bits parent
            //PathProgram = @"C:\Windows\SysWOW64\cmd.exe";

            // Force 64 bits cmd.exe child from 32bits parent
            //PathProgram = @"C:\Windows\sysnative\cmd.exe";


            FileStream fsOUT = new FileStream(StandardOutputReadHandle, FileAccess.Read, 4096, false);
            StreamReader SR = new StreamReader(fsOUT, Console.OutputEncoding);

            FileStream fsIN = new FileStream(StandardInputWriteHandle, FileAccess.Write, 4096, false);
            StreamWriter SW = new StreamWriter(fsIN, Console.InputEncoding);

            ProcessInformation ProcessInfo;
            if (!CreateProcess(PathProgram, @"",
                                IntPtr.Zero,
                                IntPtr.Zero,
                                true,
                                CreateProcessFlags.CREATE_NEW_CONSOLE, // CREATE_NEW_CONSOLE for testing only
                                IntPtr.Zero, @"C:\temp",
                                ref Shell2StartupInfo,
                                out ProcessInfo))
            {
                throw new System.ComponentModel.Win32Exception(Marshal.GetLastWin32Error());
            }

            Console.WriteLine("Child started");
           
            SW.WriteLine("echo Result should be in child process StdOUT");
            //SW.WriteLine(@"echo b > c:\temp\ttt.txt"); // test StdIN without StdOutput
            SW.Flush();
            for (int i = 1; i <= 5; i++)
            {
                Console.WriteLine("Child output " + i + " : "+ SR.ReadLine());
            }
            
            SW.Close();

            Console.WriteLine("END");
            Console.ReadLine();
        }
    }
}

You can use cmd.exe as child process or, if you prefer, my ConsoleEcho code

using System;
using System.IO;
using System.Runtime.InteropServices;

namespace ConsoleEcho
{
    class Program
    {
        static void Main(string[] args)
        {
            String NewLine = Environment.NewLine;
            String PathLog = @"c:\temp\logConsoleEcho.txt";

            try
            {
                File.Delete(PathLog);
                
                File.AppendAllText(PathLog, DateTime.Now.ToString() + " ConsoleEcho begin" + NewLine);
                Console.WriteLine(DateTime.Now.ToString() + " ConsoleEcho begin");
                File.AppendAllText(PathLog, "After first WriteLine" + NewLine); // Useful if the process crash of freeze while writing to StdOut

                String Input;
                do
                {
                    Input = Console.ReadLine();
                    File.AppendAllText(PathLog, Input + NewLine);
                    Console.WriteLine(Input);
                } while (Input != null);

            }
            catch (Exception Ex)
            {
                int error = Marshal.GetLastWin32Error();

                File.AppendAllText(PathLog, "The last Win32 Error was: " + error + NewLine);
                Console.WriteLine("The last Win32 Error was: " + error);

                File.AppendAllText(PathLog, Ex.ToString() + NewLine);
                Console.WriteLine(Ex.ToString());

                File.AppendAllText(PathLog, Ex.HResult.ToString() + NewLine);
                Console.WriteLine(Ex.HResult.ToString());

                System.Threading.Thread.Sleep(30000);
            }
            
        }
    }
}

Using cmd.exe, with a 32bits parent, you should get:

Parent output

Child started
Child output 1 : Microsoft Windows [Version 10.0.18363.1198]
Child output 2 : (c) 2019 Microsoft Corporation. All rights reserved.
Child output 3 :
Child output 4 : C:\temp>echo Result should be in child process StdOUT
Child output 5 : Result should be in child process StdOUT
END

Child Output : Nothing

Using cmd.exe, with a 64bits parent, you should get:

Parent output

Child started

Child Output : Nothing and the cmd.exe will close

Using ConsoleEcho.exe, with a 32bits parent, you should get:

Parent output

Child started
Child output 1 : 2021-01-22 13:52:24 ConsoleEcho begin
Child output 2 : echo Result should be in child process StdOUT

Child Output : Nothing

Using ConsoleEcho.exe, with a 64bits parent, you should get:

Parent output

Child started

Child Output

2021-01-22 13:55:15 ConsoleEcho begin
The last Win32 Error was: 5
System.UnauthorizedAccessException: Access to the path is denied.
   at System.IO.__Error.WinIOError(Int32 errorCode, String maybeFullPath)
   at System.IO.__ConsoleStream.Read(Byte[] buffer, Int32 offset, Int32 count)
   at System.IO.StreamReader.ReadBuffer()
   at System.IO.StreamReader.ReadLine()
   at System.IO.TextReader.SyncTextReader.ReadLine()
   at System.Console.ReadLine()
   at ConsoleEcho.Program.Main(String[] args) in \\..\ConsoleEcho\Program.cs:line 25
-2147024891

. . .

Xanatos found the solution.

Here is my approach/mistake if it can help someone :

It was the first time for me with createProcess+CreatePipe and I started with pInvoke definition. But, I had some problem with the redirection of pipes. So, I take a "working" example on Internet (that had this byte/IntPtr definition error). This code was working in 32bits and come with a lot of struct/API definitions. I didn't compared/challenged these definitions against pinvoke.

When I rebuild my project in 64bits, I got the access denied. I searched for hints/solutions and found the "same" error (access denied) with CreateProcess/CreatePipe in 64bits only. According to what I found, their problems was related to the struct SECURITY_ATTRIBUTES that is different in 32/64bits.

The code I found on Internet also had this bad definition for SECURITY_ATTRIBUTES. But, even after correcting it, the error was still there. I spent several hours trying to fix SECURITY_ATTRIBUTES or anything relate to it.

So, I partially had the answer (wrong definition) but I wasn't looking at the right place. I should have take a step back.

A bad definition (32/64bits) of SECURITY_ATTRIBUTES may result in a access denied with pipe. But, an access denied with pipe when changing platform (32/64bits) may not be relate to a bad SECURITY_ATTRIBUTES definition...


Solution

  • StartupInfo.reserved3 must be an IntPtr. It is a LPBYTE lpReserved2 in the MSDN.

    The other pinvokes seems to be correct