How to using mask image to remove pixels from original image?

[sergiotapia]

Sorry for the "guide" type question, but having a hard time finding the right docs here. I have an original image, and using SAM2 I created a mask of the image with the pixels I am interested in keeping.

---

In python here's what I did with the Image library, and it works, results in the image I'm looking for.

from PIL import Image

# Load the original image and the mask
original = Image.open("original.png")
mask = Image.open("mask.png").convert("L")  # Convert mask to grayscale

# Ensure both images are the same size
if original.size != mask.size:
    raise ValueError("Original and mask images must have the same dimensions")

# Create a new transparent image and paste the original using the mask
result = Image.new("RGBA", original.size)
result.paste(original, (0, 0), mask)

# Save the result
result.save("result.png")

How can I do similar with the Image library in Elixir?

[kipcole9]

Very reasonable question. And for sure there needs to be much better tutorial material for this library.

A mask is nothing more than an alpha band added to an image. So you can do something like:

{:ok, original} = Image.open("original.png")

# libvips calls it "bw" but its really greyscale - although
# if you really want black and white only you can threshold
# the image.
{:ok, mask} =  Image.open!("mask.png") |> Image.to_colorspace(:bw)
 
# Add the first band of the mask image as an alpha band to
# the original image.
{:oh, masked_image} = Image.add_alpha(original, mask[0])

Does that produce the result you're expecting?

[kipcole9]

Note that this solution expects that the original image does not have an alpha band. If it does have an alpha band then the alpha band and the mask need to be blended first (using Image.compose/3). Let me know if you need an example of that.

I may add Image.apply_mask (or similar) as a function to make this clearer and to build in alpha blending as required. Any thoughts on that?

[kipcole9]

@sergiotapia anything more you need help on for this?

[sergiotapia]

Hi @kipcole9 thank you for your help! I'm going to tackle this feature tomorrow and report back with the functioning code to help would-be-searchers. 😄

[kipcole9]

Cool, thanks for letting me know. You might find this livebook useful too - it also deals with masking. (I just added it yesterday)

[sergiotapia]

Finally had time today to finish this feature and it works really well!

Here's the code I used to apply the mask to remove those pixels and be left with what I needed.

def remove_mask(image_url, mask_url) do
    # Start both downloads in parallel
    original_task = Task.async(fn -> Req.get(image_url) end)
    mask_task = Task.async(fn -> Req.get(mask_url) end)

    # Wait for both downloads to complete
    with {:ok, %Req.Response{body: original_data}} <- Task.await(original_task),
         {:ok, %Req.Response{body: mask_data}} <- Task.await(mask_task),
         {:ok, original} <- Image.from_binary(original_data),
         {:ok, mask_image} <- Image.from_binary(mask_data),
         {:ok, mask} <- Image.to_colorspace(mask_image, :bw),
         {:ok, masked_image} <- Image.add_alpha(original, mask[0]),
         {:ok, final_image_data} <- Image.write(masked_image, :memory, suffix: ".png")

Thanks again for your help @kipcole9 ! I really appreciate it!