Search code examples
halide

Showing result of template matching

I ran into another problem while making template matching in Halide (original link with resolved issue: output shifted in template matching)

Now I'm trying to draw a rectangle at the position with the lowest score (which indicates the best match).

template matching part:

Image<float> source = load_image("C:\\Users\\Admin\\Desktop\\templateMatchingOpenCV\\clip\\clip2.png");
Image<float> templ = load_image("C:\\Users\\Admin\\Desktop\\templateMatchingOpenCV\\clip\\object3.png");
Var x, y, xt, yt, x_outer, y_outer, x_inner, y_inner, tile_index;

RDom r(0, templ.width(), 0, templ.height());

Func limit, comparesqdiff, comparesqdiffnorm, compareccorr;
limit = BoundaryConditions::constant_exterior(source, 1.0f);

Expr function = sum(Halide::pow(templ(r.x, r.y) - limit(x + r.x, y + r.y), 2)) / (templ.width()*templ.height());
comparesqdiff(x, y) = sum(Halide::pow(templ(r.x, r.y) - limit(x + r.x, y + r.y), 2)) / (templ.width()*templ.height());
Image<float> outputsqdiff;

comparesqdiff.tile(x, y, x_outer, y_outer, x_inner, y_inner, 64,64).fuse(x_outer, y_outer, tile_index).vectorize(x_inner).unroll(y_inner).parallel(tile_index);

comparesqdiff.compile_jit();

Now it's clear to me that the function I should be using to find the position of the lowest score is argmin but i don't quite understand how it's used. Also I am aware that the drawing method would cover everything from that's below and right of the pixel but I didn't get to that part yet.

Drawing the rectangle part: 

RDom wholeImage(0, source.width() - templ.width(), 0, source.height() - templ.height());

Tuple coords = argmin(r, function, "argmin");

Func show;

show(x, y) = select(x >= coords[0] && y >= coords[1] && x <= coords[0] + templ.width() && y <= coords[1] + templ.height(), 0, limit(x, y));

Image<float> test(source.width(), source.height());

test = show.realize(source.width(), source.height());

Thank you in advance.

Solution

  • The argmin reduction iterates over your RDom, compares the values and keeps the location of the lowest value and the value.

    Halide::RDom matchDom
    ( 0, templ.width ()
    , 0, templ.height()
    );
Halide::RDom searchDom
    ( 0, source.width () - templ.width ()
    , 0, source.height() - templ.height()
    );
Halide::Expr score = Halide::sum
    ( matchDom 
    , Halide::pow
        ( templ( matchDom.x, matchDom.y )
        - limit( searchDom.x + matchDom.x
               , searchDom.y + matchDom.y)
        , 2
        )
    )
    / ( templ.width() * templ.height() );
Halide::Tuple searchBest = Halide::argmin( searchDom, score );
Halide::Func best;
best(_) = searchBest(_);

    Then you can call best.realize() to get a Halide::Realization. That realization will contain 3 buffers, each of a single value: the x coordinate of the lowest value, the y coordinate of the lowest value and the lowest value.

    Halide is not the best tool for drawing geometric shapes. For my money, it would just be easier to write pixels into the image with a for loop.

    ASKER'S EDIT: added marking the best result which uses the answer's method for finding best score

    Realization re = best.realize();

Func draw("draw");

Image<int> x_coordinate(re[0]);
Image<int> y_coordinate(re[1]);
Image<float> s(re[2]);

draw(x,y) = select(x == x_coordinate(0, 0) || y == y_coordinate(0,0) || x == (x_coordinate(0,0) + templ.width()) || y ==  (y_coordinate(0,0) + templ.height()), 0.0f, source(x,y));

Image<float> drawTest;

drawTest = draw.realize(source.width(), source.height());

save_image(drawTest, path);

    Example of the drawing:

    Source:

    Source picture

    Template:

    Template picture

    Result:

    Result

    ASKER'S EDIT2:

    made it so it draws only the rectangle around the match

    draw(x, y) = select((x == x_coordinate(0, 0) && (y >= y_coordinate(0, 0) && y <= y_coordinate(0, 0) + templ.height())) ||
    ((x == x_coordinate(0, 0) + templ.width()) && (y >= y_coordinate(0, 0) && y <= y_coordinate(0, 0) + templ.height())) ||
    (y == y_coordinate(0, 0) && (x >= x_coordinate(0, 0) && x <= x_coordinate(0, 0) + templ.width())) ||
    ((y == y_coordinate(0, 0) + templ.height()) && (x >= x_coordinate(0, 0) && x <= x_coordinate(0, 0) + templ.width())), 0.0f, limit(x, y));

draw.tile(x, y, x_outer, y_outer, x_inner, y_inner, 64, 64).fuse(x_outer, y_outer, tile_index).vectorize(x_inner).unroll(y_inner).parallel(tile_index);

    Result:

    Result2