Java FFmpeg decoding AVPacket result of avcodec_decode_video2 negative

Question

I'm new to ffmpeg. I'm using FFmpegFrameGrabber and JavaCPP (version 1.3.3) to grab packets of an mp4-video-file. I would like to save the packet's bytestream in a database in order to decode the packets at the time the data is requested and to process the image.

public static void saveVideoToDB(String pathToVideo){
FFmpegFrameGrabber grabber = new FFmpegFrameGrabber(pathToVideo);
grabber.start();
AVPacket pkt;
while ((pkt = grabber.grabPacket()) != null) {
    BytePointer data = pkt.data();
    data.capacity(pkt.size());
    byte[] arr = data.getStringBytes();  
    //arr = [0, 0, 0, 62, 39, 100, 0, 40, -83, -124.....]
     if (pkt.stream_index() == AVMEDIA_TYPE_VIDEO) {
        //ToDo: save arr to database
         testDecode(arr);
     }


}
grabber.close();
logger.info("Import video finished.");
}   

In my code I first tried to decode a packet's data for a proof-of-concept, but I'm not sure if it's working like this:

 public static void testDecode(byte[] data){
    AVCodec avCodec = avcodec_find_decoder(AV_CODEC_ID_H264);
    AVCodecContext avCodecContext = avcodec_alloc_context3(avCodec);
    AVDictionary opts = new AVDictionary();
    avcodec_open2(avCodecContext, avCodec, opts);
    av_dict_free(opts);
    AVFrame avFrame = av_frame_alloc();
    AVPacket avPacket = new AVPacket();
    av_init_packet(avPacket);
    Frame frame = new Frame();
    avPacket.pts(AV_NOPTS_VALUE);
    avPacket.dts(AV_NOPTS_VALUE);
    BytePointer bp = new BytePointer(data);
    bp.capacity(data.length);
    avPacket.data(bp);
    avPacket.size(data.length);
    avPacket.pos(-1);

    IntBuffer gotPicture = IntBuffer.allocate(1);
    boolean doVideo = true;
    boolean keyFrames = false;
    boolean processImage = true;
    AVPacket pkt = avPacket; 
    if (doVideo) {
      int len = avcodec_decode_video2(avCodecContext, avFrame, gotPicture, avPacket);
      if (len >= 0 && gotPicture.get(0) != 0
          && (!keyFrames || avFrame.pict_type() == AV_PICTURE_TYPE_I)) {
        //ToDo: process image
        logger.info("decode success");
      }else{
        logger.info("decode failed");
      }
    }
  }

The result of avcodec_decode_video2 is always negative (-1094995529 => Invalid data found when processing input) and I receive following errors:

[h264 @ 00000000214d11a0] No start code is found.

[h264 @ 00000000214d11a0] Error splitting the input into NAL units.

Here are some metadata from the input:

Input #0, mov,mp4,m4a,3gp,3g2,mj2, from 'C:\Users\user01\Documents\fullstream.mp4':
  Metadata:
    major_brand     : mp42
    minor_version   : 0
    compatible_brands: mp42mp41isomiso2
    creation_time   : 2017-07-27T11:17:19.000000Z
  Duration: 00:55:55.48, start: 0.000000, bitrate: 5126 kb/s
    Stream #0:0(eng): Video: h264 (High) (avc1 / 0x31637661), yuv420p, 1920x1080, 4996 kb/s, 25 fps, 25 tbr, 2500 tbn, 5k tbc (default)
    Metadata:
      creation_time   : 2017-07-27T11:17:19.000000Z
      handler_name    : VideoHandler
    Stream #0:1(eng): Audio: mp3 (mp4a / 0x6134706D), 24000 Hz, mono, s16p, 125 kb/s (default)
    Metadata:
      creation_time   : 2017-07-27T11:17:19.000000Z
      handler_name    : SoundHandler

Answer 1

Sorry, i only know ffmpeg in c++ but not your java stuff. Your problem is that you are not clear about what you actually feed ito the mpeg4 decoder. You can only feed raw mpeg4 streams into it.

My recommendation is to get out whats the difference between typical C type usage of libavcodec and your java.

in C we do something like this in order to be sure to just feed the video binary data into the decoder:

while(av_read_frame(pFormatCtx, &packet)>=0) {
  // Is this a packet from the video stream?
  if(packet.stream_index==videoStream) {
    // Decode video frame
    avcodec_decode_video2(pCodecCtx, pFrame, &frameFinished, &packet);
    ...

Looking at your code, i dont know how FFmpegFrameGrabber maps to av_read_frame and please take a very close look to your function call to

byte[] arr = data.getStringBytes();  

The naked fact that the term "String" is contained here is frightening for me because most string functions tend to do some byte conversions and on video stuff we are dealing with binary data.

Please let me know any question or your solution, i am somehow interested in this one...

P.S. you might maybe just need to read more data than just one single chunk to get out some video frame of avcodec_decode_video2. Dont expect the first frame in your video to be an Iframe. Try to make your program to read from the start to the end of the file before you concentrate on what i wrote above

Edit 2: try to format your input into a raw h264 stream for tesing your program, e.g. use ffmpeg -i test.mp4 -vcodec copy -an c:\temp\test.h264