This is just some handler code copied from the Vix examples
defmodule VixExt do
alias Vix.Vips.Image
alias Vix.Vips.Operation
@max_height 500
def show(%Image{} = image) do
height = Image.height(image)
# scale down if image height is larger than 500px
image =
if height > @max_height do
Operation.resize!(image, @max_height / height)
else
image
end
# write vips-image as png image to memory
{:ok, image_bin} = Image.write_to_buffer(image, ".png")
Kino.render(Kino.Image.new(image_bin, "image/png"))
:ok
end
end
The code that does stuff:
import VixExt
{:ok, fore} = Image.open("/home/user/Downloads/greenscreen.jpg")
{:ok, back} = Image.open("/home/user/Downloads/background.jpg")
# Lower bound green
{:ok, l_green} = Image.Math.greater_than(fore, [0.0, 100.0, 0.0])
# Upper bound green
{:ok, u_green} = Image.Math.less_than(fore, [100.0, 255.0, 95.0])
{:ok, color_fore_mask} = Image.Math.boolean_and(l_green, u_green)
{:ok, fore_mask} = Vix.Vips.Operation.bandbool(color_fore_mask, :VIPS_OPERATION_BOOLEAN_AND)
{:ok, masked} = Image.Math.subtract(fore, fore_mask)
{:ok, inverted_fore_mask} = Vix.Vips.Operation.invert(fore_mask)
{:ok, masked_back} = Image.Math.subtract(back, inverted_fore_mask)
{:ok, masked_bin} = Vix.Vips.Image.write_to_buffer(masked, ".jpg")
{:ok, masked_clone} = Vix.Vips.Image.new_from_buffer(masked_bin)
{:ok, masked_back_bin} = Vix.Vips.Image.write_to_buffer(masked_back, ".jpg")
{:ok, masked_back_clone} = Image.from_binary(masked_back_bin)
{:ok, composite} = Vix.Vips.Operation.add(masked_back, masked_clone)
show(composite)
The problem that I encountered at first is that the clipped images don't get added together properly. Elixir is immutable so I was kind of expecting that behaviour. However, they're all pointed to the same image so when I tried to add them together it was kind of a mashup of the originals.
I can force it to the right behaviour by loading preclipped images which then behaves as I expect but I don't want to have to save to disk for obvious reasons. So now I'm trying to force a copy so that I get the clipped version not the original.
The image shown is the result I want. The person starts out on a green background which gets removed and composited onto the scene.
The current error I'm getting is
** (MatchError) no match of right hand side value: {:error, "Failed to write VipsImage to memory"}
which happens anytime I try to write to the buffer twice.
I would have thought that green screen removal was a rather routine use case which would be easier to do. I'm perfectly happy to take a better approach.
The Image Elixir library now provides Image.chroma_key/2 in response to this challenge from @GeneticJam. Used like this which is basically:
Image.chroma_key!(foreground)
|> then(&Image.compose!(background, &1))
The default chroma covers the likely gamut of chroma green.