Golang executed command with Run() is not returning

Question

I am attempting to create a music streaming service using youtube-dl and ffmpeg. When a user sends a POST request with a video URL, my handler code works as shown below:

 router.POST("/submit", func(c *gin.Context) {
        body := Body{}
        if err := c.BindJSON(&body); err != nil {
            c.AbortWithError(http.StatusBadRequest, err)
            return
        }
        w := c.Writer
        header := w.Header()
        header.Set("Content-Type", "audio/mp3")
        w.WriteHeader(http.StatusOK)
        fetchMusic(c, body.Data)
    })
func fetchMusic(c *gin.Context, data string) {

    r, w := io.Pipe()
    defer r.Close()

    ydl := exec.Command("youtube-dl", data, "-o-")
    ffmpeg := exec.Command("ffmpeg", "-i", "/dev/stdin", "-vn", "-f", "mp3", "-")

    ydl.Stdout = w
    ydl.Stderr = os.Stderr

    ffmpeg.Stdin = r
    ffmpeg.Stdout = c.Writer
    ffmpeg.Stderr = os.Stderr
    fmt.Println("Starting-----------------------")
    go func() {
        if err := ydl.Run(); err != nil {
            panic(err)
        }
    }()
    
    if err := ffmpeg.Run(); err != nil {
        panic(err)
    }

    
    fmt.Println("Done-----------------------")
}

I created a pipe with ffmpeg and youtube-dl. In the testing stage, I sent a POST request, but the request is not completing. If I look at the logs, I couldn't see "Done----------------". The process seems to be stuck, I think. Do you have any idea? Is my usage true?

Answer 1

This is very close — we need to close the the *io.PipeWriter as well.

Otherwise ffmpeg thinks more data could come through. This will show up to the program as an EOF.

Here is a runnable example that I threw together, using echo and cat instead. You should be able to reproduce the issue by commenting out the new defer w.Close():

package main

import (
    "fmt"
    "io"
    "os"
    "os/exec"
)

func main() {
    r, w := io.Pipe()
    defer r.Close()

    echo := exec.Command("echo", "hello")
    cat := exec.Command("cat")

    echo.Stdout = w
    echo.Stderr = os.Stderr

    cat.Stdin = r
    cat.Stdout = os.Stdout
    cat.Stderr = os.Stderr
    fmt.Println("Starting-----------------------")
    go func() {
        // -- I added this --
        defer w.Close()
        if err := echo.Run(); err != nil {
            panic(err)
        }
    }()

    if err := cat.Run(); err != nil {
        panic(err)
    }

    fmt.Println("Done-----------------------")
}