I'm trying to make a video from a list of BufferedImages, using xuggler. I followed this tutorial, and the code works perfectly.
However, when I incorporate the working code from the example to my code, I get a RuntimeException:
java.lang.RuntimeException: failed to write packet: com.xuggle.xuggler.IPacket@46575264[complete:true;dts:67915;pts:67915;size:24201;key:false;flags:0;stream index:0;duration:1;position:-1;time base:1/65535;] at com.xuggle.mediatool.MediaWriter.writePacket(MediaWriter.java:1215) at com.xuggle.mediatool.MediaWriter.encodeVideo(MediaWriter.java:767) at com.xuggle.mediatool.MediaWriter.encodeVideo(MediaWriter.java:810) at s3.S3CompositeUtil.generateVideo(S3CompositeUtil.java:247) at s3.S3CompositeUtil.main(S3CompositeUtil.java:349)
The Xuggler documentation mentions that : Callers must ensure that IMediaData.getTimeStamp(), if specified is always monotonically increasing or an RuntimeException will be raised.
But I don't think the times I'm working with in generating the video decrease at any point.
private void generateVideo() {
System.out.println("Creating video...");
String outputMoviePath = "";
IMediaWriter writer = null;
if (allLabels) {
outputMoviePath = TEMP_DIR_PATH + "/" + clipName + "/" + clipName + "_" + "all_labels" + ".avi";
}
else {
outputMoviePath = TEMP_DIR_PATH + "/" + clipName + "/" + clipName + "_" + labels[desiredLabelIdx] + ".avi";
}
writer = ToolFactory.makeWriter(outputMoviePath);
// add one video stream, with position 0 and ID 0, and a frame rate of OUTPUT_MOVIE_FPS
writer.addVideoStream(0, 0, ICodec.ID.CODEC_ID_MPEG4, vidWidth, vidHeight);
long startTime = System.nanoTime();
for (int idxF = 0; idxF < numFrames; idxF++) {
BufferedImage currFrameImg = convertToType(videoBIHolder.get(idxF), BufferedImage.TYPE_3BYTE_BGR);
// BufferedImage currFrameImg = videoBIHolder.get(idxF);
System.out.println("currFrameImg type is " + currFrameImg.getType()); // debug
// encode video to stream at 0
long frameTimeInVideo = System.nanoTime() - startTime;
System.out.println("Time: " + frameTimeInVideo);
writer.encodeVideo(0, currFrameImg, frameTimeInVideo, TimeUnit.NANOSECONDS);
// sleep for millisec. amount of time of the frame rate
try {
Thread.sleep((long) (1000/OUTPUT_MOVIE_FPS)); //OUTPUT_MOVIE_FPS
}
catch (InterruptedException e) {
System.err.println();
}
}
(S3CompositeUtil.java:247) is the line
writer.encodeVideo(0, currFrameImg, frameTimeInVideo, TimeUnit.NANOSECONDS);
I need the OUTPUT_MOVIE_FPS to be 1, but the strange thing is that when it's 5, the video is created with no exceptions, when it's 2, the "video" is only the first frame (no other frames), and when it's 1, I get this exception.
Any thoughts? I would really appreciate all the help I could get!
I am teaching Xuggler in one of my multimedia classes and some of my students reported similar problems when creating stop motion animations or videos that create picture slideshows.
We resorted to at least create 2 or even 10 FPS. The fun thing is, that you will most likely not use very much more memory to store the video. The MPG4 encoder is clever enough to insert only predictive frames into the video file which are basically empty since there are no differences in the frames until a completely new frame is inserted.
The only downside to this approach is that you will cause a bit more encoding overhead. Obviously inserting any frame into the video stream will cause CPU load even if in the end the decoder decides that only an empty predictive frame can be inserted.
edit: It turns out that we actually resorted to insert frames into the video at a rate of 25 fps. With all the downsides described above. However, out of curiosity I did a tests with frame rates between 10 fps and 25 fps now and it turns out that there are no significant changes in the size of the files. Funnily the 10 fps test file is the largest one.
A 5 second full HD video of a still image is in the area of 450kb to 580kb. So I suggest you just experiment with frame rates and just choose the lowest one ffmpeg / Xuggler can handle for H264 (actually this should be on documentation somewhere but I cannot find it right now).
For testing I used the following code which inserts the same BufferedImage all over again.
final IMediaWriter writer = ToolFactory.makeWriter(outputFile);
writer.addVideoStream(0, 0, ICodec.ID.CODEC_ID_H264, img.getSize().width, screenshot.getSize().height);
long startTime = System.nanoTime();
for (int i=0; i<seconds*25; i++) {
BufferedImage bgrScreen = Tools.convertToType(img, BufferedImage.TYPE_3BYTE_BGR);
writer.encodeVideo(0, bgrScreen, System.nanoTime() - startTime, TimeUnit.NANOSECONDS);
try {
Thread.sleep((long) (1000 / 25));
} catch (InterruptedException e) {
// ignore
}
}
writer.close();
Pro-Tip: In real world applications we use one thread to provide the image frames and one thread to perform the encoding (writer.encodeVideo) such that we can make sure that we hit an exact frame-rate of 25 fps. In your example (and also in my test code) the encoding time of Xuggler / ffmpeg is added.