Python: Popen stdout=PIPE can't read progress bar output

Question

I'm trying to route the progress bar from a console application over a UDP socket to remotely show the status of completion of a task. For some odd reason all data from output from subprocess stdout.PIPE could be read except the progress bar which appeared blank if I tried to write to a file or transmit over a UDP socket. I suspect the progress bar isn't being channeled to stdout.PIPE at all due to not ending with \n since if I remove the print statement for stdout.PIPE the progress bar still prints in the console. I've tried to reproduce the problem with tqdm library:

sample_process.py

from time import sleep
from tqdm import tqdm
for i in range(10):
    print(i)
    
for i in tqdm(range(10)):
    sleep(0.5)

client.py

import socket
from subprocess import Popen, PIPE, CalledProcessError

serverAddressPort   = ("127.0.0.1", 20001)
bufferSize          = 1024

UDPClientSocket = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)
cmd = ['python', 'sample_process.py']

with Popen(cmd, stdout=PIPE, bufsize=0, universal_newlines=True) as p:
    for line in p.stdout:
        print(line, end='')
        UDPClientSocket.sendto(line.encode(), serverAddressPort)

if p.returncode != 0:
    raise CalledProcessError(p.returncode, p.args)

server.py

import socket

UDPServerSocket = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)
UDPServerSocket.bind(("127.0.0.1", 20001))
while(True):
    bytesAddressPair = UDPServerSocket.recvfrom(1024)
    message = bytesAddressPair[0]
    print(message.decode())

If I do print(p.stdout.read()), then I don't get any output till end out process. Is there anyway to read the progress bar in real-time? I'm using Python 3.9.13 on Windows 11.

Answer 1

It seems my speculation was wrong, the progress bar output was being channeled to std.err which is why it was not being captured. Changing the code in client.py to following fixes the problem:

import socket
from subprocess import Popen, PIPE, CalledProcessError, STDOUT

serverAddressPort   = ("127.0.0.1", 20001)
bufferSize          = 1024

UDPClientSocket = socket.socket(family=socket.AF_INET, type=socket.SOCK_DGRAM)
cmd = ['python', 'sample_process.py']

with Popen(cmd, stdout=PIPE, stderr=STDOUT, bufsize=1, universal_newlines=True) as p:
        for line in p.stdout:
            print(line, end='')
            UDPClientSocket.sendto(line.encode(), serverAddressPort)
        

if p.returncode != 0:
    raise CalledProcessError(p.returncode, p.args)

Credits to @TurePålsson and @simon for rightfully pointing that out in the comments.