Search code examples
c++cffmpegvideo-processingswscale

Using swscale for image composing

I have an input image A and a resulting image B with the size 800x600 stored in YUV420 format and I need to scale image A into 100x100 size and place it into resulting image B at some point (x=100, y=100). To decrease memory and CPU usage I put swscale result right into final B image.

Here is a code snippets (pretty straightforward):

//here we a creating sws context for scaling into 100x100
sws_ctx = sws_getCachedContext(sws_ctx, frame.hdr.width, frame.hdr.height, AV_PIX_FMT_YUV420P,
                               100, 100, AV_PIX_FMT_YUV420P, SWS_BILINEAR, nullptr, nullptr, nullptr);

Next we create corresponding slice and strides describing image A

    int src_y_plane_sz = frame.hdr.width * frame.hdr.height;
    int src_uv_plane_sz = src_y_plane_sz / 2;
    std::int32_t src_stride[] = {
        frame.hdr.width,
        frame.hdr.width / 2,
        frame.hdr.width / 2,
        0};

    const uint8_t* const src_slice[] = {
        &frame.raw_frame[0],
        &frame.raw_frame[0] + src_y_plane_sz,
        &frame.raw_frame[0] + src_y_plane_sz + src_uv_plane_sz,
        nullptr};

Now doing the same for a destination B image

    std::int32_t dst_stride[] = {
        current_frame.hdr.width,
        current_frame.hdr.width /2,
        current_frame.hdr.width /2,
        0
    };

    std::int32_t y_plane_sz = current_frame.hdr.width * current_frame.hdr.height;
    std::int32_t uv_plane_sz = y_plane_sz / 2;

    //calculate offset in slices for x=100, y=100 position
    std::int32_t y_offset = current_frame.hdr.width * 100 + 100;

    uint8_t* const dst_slice[] = {
        &current_frame.raw_frame[0] + y_offset,
        &current_frame.raw_frame[0] + y_plane_sz + y_offset / 2,
        &current_frame.raw_frame[0] + y_plane_sz + uv_plane_sz + y_offset / 2,
        nullptr};

After all - calling swscale 

    int ret = sws_scale(sws_ctx, src_slice, src_stride, 0, frame.hdr.height,
                        dst_slice, dst_stride);

After using a testing sequence I having some invalid result with the following problems:

  1. Y component got some padding line
  2. UV components got misplaced - they are a bit lower then original Y components.

Artefacts

Does anyone have had the same problems with swscale function? I am pretty new to this FFmpeg library collection so I am open to any opinions how to perform this task correctly.

FFmpeg version used 3.3

Solution

  • Thanks to @VTT for pointing out the possible problem - I have fixed destination slice pointers calculation to the following:

        int dest_x = 200, dest_y = 70;

    //into 100x100 position
    std::int32_t y_offset = current_frame.hdr.width * dest_y + dest_x;
    std::int32_t u_offset = ( current_frame.hdr.width * dest_y )  / 4 + dest_x /2;
    std::int32_t v_offset = u_offset + y_plane_sz / 4;

    uint8_t* const dst_slice[] = {
        &current_frame.raw_frame[0] + y_offset,
        &current_frame.raw_frame[0] + y_plane_sz + u_offset,
        &current_frame.raw_frame[0] + y_plane_sz + v_offset,
        nullptr};

    And the second problem with the "line artefact" is solved by using scaled dimensions sizes factor by 8.

    One more addition for proper position calculations for destination slice pointers - that y coordinates must be re-adjust according to a current Y plane pointing because per each two Y strides there is only one U or V stride. For example (see adjusted_uv_y variable):

    std::int32_t adjusted_uv_y = dest_y % 2 == 0 ? dest_y : dest_y - 1;
std::int32_t y_offset = current_frame.hdr.width * dest_y + dest_x;
std::int32_t u_offset = ( current_frame.hdr.width * adjusted_uv_y )  / 4 + dest_x /2;
std::int32_t v_offset = u_offset + y_plane_sz / 4;