in my system, I have a camera, and I have got an image sequence from the camera, but the time interval between each image is not the same. See below:
┌─────┐ ┌─────┐ ┌─────┐ ┌─────┐
│ │ │ │ │ │ │ │
│image├─►│image├─►│image├─►│image├─►
│ │ │ │ │ │ │ │
└─────┘ └─────┘ └─────┘ └─────┘
t0 t1 t2 t3
Basically, t1-t0 != t2-t1, and t3-t2 != t2-t1.
A simple method could same those images as separate files, such as 09-39-001.png, 09-39-058.png, the file name contains the minute-second-millisecond timestamp of the image. With this method, I got a lot of files, and it runs slowly if I got a lot of images. The reason I guess is that there are two many file creation option to the disk.
I know there are a lot of method to store them in a single file, for example, save them to a mp4 file, e.g. the OpenCV library has a way to write to a mp4 file by cv::VideoWriter object. But with this method, the mp4 file stores images in a const rate(such as 30fps), and my time stamp information is lost. When I try to replay the mp4 file, each image(frame) is played in a const time interval way. The only good thing is the mp4 is compressed thus its file size is smaller than the total amount of separate image files.
I heard that the ROS system can capture and save images in a bag or bag2 file, and I think using ROS is a bit complex for my simple project. In my simple project, I need to capture camera images and one or more sensor data from other sources, so I do need a way to save those data so that I can reply them later precisely in time.
As a conclusion, I need a method to same data with their time stamp in a simple file, and later playback again. Hopefully the file could be compressed like the mp4 file, but I think file size is the secondary factor.
Any suggestions? Thanks.
I think I can answer this question myself, there are several ways to handle this:
Method 1, using the FFmpeg library(which supports variable frame rate recording)
This is done in my answer to my own question here: save a serial of images(cv::Mat) to a mp4 file in Variable Frame Rate mode by using ffmpeg library, how to set the pts?
In this answer(with source code), I can correctly set the PTS value of each frame, and the result mp4 file will respect each time stamp of the image.
source code:
save to mp4 file: opencv-image-to-mp4.cpp
read mp4 file and show them by opencv: read_image_from_mp4_imshow.cpp
Method 2, using the original OpenCV method cv::VideoWriter, which only support constant frame rate recording, but time stamp can be saved in another file
With this method, I have to add a separate file, such as "timestamp.txt" file along with the mp4 file, this file should contains the timestamp of each saved image in the video file, as in the discussion suggest by @fana, I have a test code in github gist: save image sequence to mp4 file, and read them back