Using and Overlapped Named Pipe in Python 3.7

Question

I am using Windows with Python 3.7 and I am trying to asynchronly share data, ony strings, between to python processes. One of the is running indefinitely (reciever), the other one may start at any point sends some data and then ends (sender). I am trying to use a named pipe for that.

I managed to get this when they run synchronously (the reciever waiting at an blocked pipe until he gets data), however the reciever has other stuff to do so ideally he shouldn't wait all the time. Also there might be a second sender at some point so a blocked pipe isin't great.

The code for the receiver is:

import os
import time
import sys
import win32pipe, win32file, pywintypes

pipe_name = r'\\.\pipe\mypipe' 

pipe = win32pipe.CreateNamedPipe(
        pipe_name,
        win32pipe.PIPE_ACCESS_DUPLEX | win32file.FILE_FLAG_OVERLAPPED,  # open mode 
        win32pipe.PIPE_TYPE_MESSAGE | win32pipe.PIPE_READMODE_MESSAGE | win32pipe.PIPE_WAIT, # pipe mode
        1, 65536, 65536, # max instances, out buffer size,  in buffer size
        0, # timeout
    None)


while 1:
    print("doing my other stuff")
    try:
        win32pipe.ConnectNamedPipe(pipe, pywintypes.OVERLAPPED())
    except pywintypes.error as e:
        if e.winerror == 232:   #disconnected pipe
            win32pipe.DisconnectNamedPipe(pipe)
            print("Disconnecting pipe")
        else:
            print(e)
    try:
        retval, msg = win32file.ReadFile(pipe, 0, pywintypes.OVERLAPPED())
        print(msg)
    except pywintypes.error as e:
        if e.winerror == 536: #Wating for connection
            print("waiting")
        elif e.winerror == 233: #no process on other side
            continue
    time.sleep(1)

The code for the sender is:

import os
import time
import sys
import win32pipe, win32file, pywintypes


pipe_name = r'\\.\pipe\mypipe' 


for x in range(5):

    handle = win32file.CreateFile(
            pipe_name,
            win32file.GENERIC_READ | win32file.GENERIC_WRITE,
            0,
            None,
            win32file.OPEN_EXISTING,
            win32file.FILE_FLAG_OVERLAPPED,
            None
            )
    res = win32pipe.SetNamedPipeHandleState(handle, win32pipe.PIPE_READMODE_MESSAGE, None, None)
    print(f"sending {x}")
    win32file.WriteFile(handle, str.encode(f"hello world {x}"))
    win32file.CloseHandle(handle)
    time.sleep(2)

Right now both can run and have some connection but i cant really get the data. The reciever can do oter stuff and disconnects and reopens the pipe if something is send, but msg ends up being empty. If i stop it in the debugger and send something the value of msg gets "memory at 0x0......." which I would interpret as some sort of pointer, yet as you probalby already noticed my understaing of pipes is limited.

Here I found a good example for a synchronos pipe which worked. I changed the the creation of the pipe to the reciever but that wasn't to hard. I found some examples for asynchronous (overlapped) pipes here, which were also great but left me with the issue i am facing now.

Ist reading from an overlapped pipe still a task for win32file.ReadFileor is their something else I am missing?

Thank you very much!

Answer 1

I found the solution and want to share it, just in case anyone else is stumbeling upon this issue. As it turns out msg gets "memory at 0x0......." is a memoryview object and its data can be exposed via bytes(msg).

also there was an issue with my ReadFile command as the buffer has to >0 for it to achieve anything. Now it reads every byte individually and adds them up to a string. This is proabably not great performancewise, but it works for me and it solves the issue of having to cut the end of if the message is shorter than the buffer length.

msg = ''
rtnvalue, data = win32file.ReadFile(pipe, 1, pywintypes.OVERLAPPED())
while rtnvalue == 234:
    msg = msg + bytes(data).decode('ASCII') 
    rtnvalue, data = win32file.ReadFile(pipe, 1, pywintypes.OVERLAPPED())
if rtnvalue == 0: #end of stream is reached
    msg = msg + bytes(data).decode('ASCII')
return msg