FFmpeg C demo generates "Could not update timestamps for skipped samples" warning

Question

When I run my demo code I get these warnings when testing it on a webm video:

[opus @ 0x5ec0fc1b4580] Could not update timestamps for skipped samples.
[opus @ 0x5ec0fc1b4580] Could not update timestamps for discarded samples.

But it's not limited to webm, I also get this warning when running with a mp4:

[aac @ 0x61326fb83700] Could not update timestamps for skipped samples.

I know I'm getting warnings because ffmpeg must be skipping packets, but I have no idea why. Why are we skipping packets (and if that's not the problem, what is) and how can we fix the problem?

Here's my code for context:

#include <libavcodec/avcodec.h>
#include <libavformat/avformat.h>

int main()
{
    int ret = 0;

    const AVCodec* codec;
    AVFormatContext* fmt_ctx = avformat_alloc_context();

    const char* file2 = "/home/aabiji/Videos/sync-test.webm";
    if ((ret = avformat_open_input(&fmt_ctx, file2, NULL, NULL)) < 0) {
        av_log(NULL, AV_LOG_ERROR, "Couldn't open input file\n");
        return -1;
    }

    ret = av_find_best_stream(fmt_ctx, AVMEDIA_TYPE_AUDIO, -1, -1, &codec, 0);
    if (ret < 0) {
        av_log(NULL, AV_LOG_ERROR, "Couldn't find a media stream\n");
        return -1;
    }

    int stream_index = ret;
    AVStream* media = fmt_ctx->streams[stream_index];

    AVCodecContext* codec_context = avcodec_alloc_context3(codec);
    if (avcodec_parameters_to_context(codec_context, media->codecpar) < 0) {
        return -1;
    }

    if ((ret = avcodec_open2(codec_context, codec, NULL)) < 0) {
        av_log(NULL, AV_LOG_ERROR, "Couldn't open media decoder\n");
        return -1;
    }

    AVPacket* packet = av_packet_alloc();
    AVFrame* frame = av_frame_alloc();

    while ((ret = av_read_frame(fmt_ctx, packet)) >= 0) {
        if (packet->stream_index != stream_index) {
            continue;
        }

        ret = avcodec_send_packet(codec_context, packet);
        if (ret < 0) {
            break; // Error
        }

        while (ret >= 0) {
            ret = avcodec_receive_frame(codec_context, frame);
            if (ret == AVERROR_EOF || ret == AVERROR(EAGAIN)) {
                break;
            } else if (ret < 0) {
                fprintf(stderr, "Error during decoding\n");
                break;
            }
            av_frame_unref(frame);
        }

        av_packet_unref(packet);
    }

    avcodec_flush_buffers(codec_context);

    av_packet_unref(packet);
    av_frame_free(&frame);
    av_packet_free(&packet);
    avcodec_free_context(&codec_context);
    avformat_close_input(&fmt_ctx);
    return 0;
}

Answer 1

ffmpeg is not skipping packets, at least not in the way you think. Packets from an audio encoder for codecs such as MP3/AAC/Opus are interdependent on neighbouring packets. For the head and tail of a stream, the encoder inserts blank samples for that purpose, which are to be skipped (at head) and discarded (at tail) by the decoder.

The avcodec framework needs two parameters to be populated to update timestamps for these samples - the packet timebase and the audio sample rate. The latter should be set by the decoder,

For the former, set

codec_context->pkt_timebase = media->time_base;

before the call to avcodec_open2