Is it possible to collect audio data then publish audio data every 0.5 second?

Question

Is it possible to collect audio data then publish audio data every 0.5 second?

I am using Go RTMP Server with https://github.com/yutopp/go-rtmp this package and it works well.

inside of function OnAudio(): this is function that processing of audio data, it receives data from OBS and DecodeAudioData and Publish to vlc player

this is original code of OnAudio()

func (h *Handler) OnAudio(timestamp uint32, payload io.Reader) error {
    var audio flvtag.AudioData
    if err := flvtag.DecodeAudioData(payload, &audio); err != nil {
        return err
    }

    flvBody := new(bytes.Buffer)
    if _, err := io.Copy(flvBody, audio.Data); err != nil {
        return err
    }
    audio.Data = flvBody
    _ = h.pub.Publish(&flvtag.FlvTag{
        TagType:   flvtag.TagTypeAudio,
        Timestamp: timestamp,
        Data:      &audio,
    })

    return nil
}

this code receives audio data every 0.02 second and publish it. I want to chage the code that colelct audio data until 0.5 second and publish audio data so trying like this ...

func (h *Handler) OnAudio(timestamp uint32, payload io.Reader) error {

    var audio flvtag.AudioData
    if err := flvtag.DecodeAudioData(payload, &audio); err != nil {
        return err
    }
       // if flag == true, startTimeStamp Control
    if h.Flag == true {
        h.startTimeStamp = timestamp
        h.Buf.Reset()
    }
    h.Flag = false

    buf := new(bytes.Buffer)
    if _, err := io.Copy(buf, audio.Data); err != nil {
        return err
    }

    if audio.AACPacketType == 0 {
        audio.Data = buf
        tag := &flvtag.FlvTag{
            TagType:   flvtag.TagTypeAudio,
            Timestamp: h.startTimeStamp,
            Data:      &audio,
        }
        timestamp, (timestamp - h.startTimeStamp), buf.Len())
        _ = h.pub.Publish(tag)
        h.Flag = true
        return nil
    }

    h.Buf.Write(buf.Bytes())

        // collect audio data into h.Buf and then publish every 0.5 seconds
    if timestamp-h.startTimeStamp >= 500 {
        h.lastTimeStamp = timestamp
        audio.Data = h.Buf
        tag := &flvtag.FlvTag{
            TagType:   flvtag.TagTypeAudio,
            Timestamp: h.startTimeStamp,
            Data:      &audio,
        }

        _ = h.pub.Publish(tag)
        h.Flag = true
    }

    return nil
}

when I check the audio data size There is no Data loss but audio datas are breaking out. Is there any problem with my code?

Answer 1

I applied your change to the official server_relay_demo (see this commit). Then built and run this demo:

go build -v -o server_relay_demo .
./server_relay_demo

Then I downloaded http://commondatastorage.googleapis.com/gtv-videos-bucket/sample/ForBiggerBlazes.mp4, and publish it with ffmpeg:

ffmpeg -re -stream_loop -1 -i ForBiggerBlazes.mp4 -acodec copy -vcodec copy -f flv rtmp://localhost/appname/stream

And finally play it with vlc:

$ vlc rtmp://localhost/appname/stream
VLC media player 3.0.18 Vetinari (revision 3.0.18-0-ge9eceaed4d)
[000055c1cc8f3b10] main libvlc: Running vlc with the default interface. Use 'cvlc' to use vlc without interface.
Qt: Session management error: Could not open network socket
[tcp @ 0x7f84280045c0] RTMP packet size mismatch 14272640 != 5642

It failed to play with this error: RTMP packet size mismatch 14272640 != 5642.

Then I replaced h.startTimeStamp with timestamp (see this commit):

tag := &flvtag.FlvTag{
    TagType:   flvtag.TagTypeAudio,
    Timestamp: h.startTimeStamp,
    Data:      &audio,
}

It works with this change. Though the audio and the video are out of sync.

Please try this solution. If it still does not work, please provide a demo and a detail step by step guide to reproduce the issue.