python 3 using ffmpeg in a subprocess getting stderr decoding error

I am running ffmpeg as a subprocess and using the stderr to get various bits of data like the subtitles stream Id's. It works fine for most videos, but one with japanese subtitles results in an error:

'charmap' codec can't decode byte in position xxx : character maps to

Much googling suggests the problem is that the japanese requires unicode, whereas English does not. Solutions offered refer to problems with files, and I cannot find a way of doing the same with the stderr. Relevent Code is below:

command = [ffmpeg,"-y","-i",fileSelected,"-acodec","pcm_s16le",
                  "-vn","-t","3", "-f",            "null","-"]
print(command)   
proc = subprocess.Popen(command,stderr=PIPE,Jstdin=subprocess.PIPE,
                            universal_newlines=True,startupinfo=startupinfo)
  
stream = ""    
for line in proc.stderr:
    try:
        print("line",line)
    except exception as error:
        print("print",error)
        line = line[:-1]
        if "Stream #" in line:
            estream = line.split("#",1)[1]
            estream =estream.split(" (",1)[0]
            print("estream",estream)
            stream = stream + estream +"\n"  #.split("(",1)[0] 
            stream = stream + estream +"\n"

Solution

you can stipulate encoding as:

proc = subprocess.Popen(command, stderr=PIPE, stdin=subprocess.PIPE, universal_newlines=True, startupinfo=startupinfo, encoding='utf-8')

It may also be a good idea for you to not use universal_newlines as that can sometimes cause issues with streams.

I would also move your stream extraction into your try: except: block in order to provide for sane crashes.

Hope this helps