Inputting Audio Stream to FFMPEG

Question

I’m building a real time chat application with C# and ffmpeg.exe. My requirement is to get a memory stream from Microsoft Speech API and feed it to ffmpeg process in real time. I can take a memory stream from Microsoft Speech API. I’m using following code to create the memory stream.

        using (MemoryStream stream = new MemoryStream())
        {
            MemoryStream streamAudio = new MemoryStream();
            System.Media.SoundPlayer m_SoundPlayer = new System.Media.SoundPlayer();
            _speechSynthesizerVisemesSender.SetOutputToWaveStream(streamAudio);
            _speechSynthesizerVisemesSender.SetOutputToNull();
            stream.WriteTo(proc.StandardInput.BaseStream);
        }   

I’m already using another datapipe with another command to feed video content to ffmpeg. But I couldn't find a stable solution to feed audio through a datapipe. This article briefly explains about audio datapipe. I’m using following command to stream audio.

"ffmpeg -re -f s16le -i pipe:wav -f mpegts udp://127.0.0.1:1234"

But it is not working with the datapipe. If I try the command with mp3 or wav file, it works.

Answer 1

Since Microsoft Speech API is designed for desktop applications and does not support for web applications, you can't get a continues audio stream from speech API. So only option you have is to create a wav file using SAPI and combine audio file with the stream.

When you have an audio file you can use audio input option in ffmpeg.

ffmpeg -re -f s16le -i pipe:wav -i audio.mp3 -r 10 -vcodec mpeg4 -f mpegts udp://127.0.0.1:1234

But problem is you won't be able to map your audio with video stream. You can use above command, if you don't have to sync audio with video.