Segmentation fault when composing a larger image of many smaller images

[MarcusRiemer]

tl;dr: Clone https://github.com/MarcusRiemer/elixir-tiled, run mix deps.get && mix test test/tiled/map_test.exs --trace and you will probably see a crash.

I have a very much WIP state of a renderer for Tiled .tmj files. These are essentially JSON files that contain instructions on how to compose an image of many small "tiles". The current implementation is very much WIP and not optimized at all, there is no caching and each tile that is to be rendered is effectively a new image created by Image.crop. So this might be related to #191 but so far I could only observe the issue when actively writing the image. Or I am too sleep deprived to recognize ...

Reducing the amount of tiles rendered (or maybe cropped?) prevents the crash, hence the following section:

  def render_layer!(%Vix.Vips.Image{} = map_image, %__MODULE__{} = map, %TileLayer{} = layer) do
    layer.data
    |> Enum.with_index()
    |> Enum.filter(fn {tile, _coord_index} -> tile > 0 end)
    # |> Enum.take(50) # Rendering more than 50 tiles on a single layer crashes the phoenix server on the second rendering of a controller
    |> Enum.reduce(map_image, fn {tile, coord_index}, %Vix.Vips.Image{} = accu_image ->
      {render_x, render_y} = Tiled.Coordinate.point_from_index(map, layer.width, coord_index)
      {:ok, rendered_tile_image} = tile_image(map, tile)

      Image.compose!(accu_image, rendered_tile_image, x: render_x, y: render_y)
    end)
  end

So I copied together the code from the private project, published it and recognized it's well past midnight where I live, so I will leave it at that for the moment.

[kipcole9]

Thanks for the report - and especially for the reproducible case. It's morning where I am but I have a bunch of commitment over the next several hours - I'll dive in after that.

[kipcole9]

A quick run and what I'm seeing on my MacBook Pro M1 Max is that some runs pass and some do not. We both know that's not fun to debug. I'll get into the details in a few hours.

[MarcusRiemer]

First things first: Thank you for your time and please don't feel rushed or under obligation in any way! I very much appreciate you providing this library as it is.

For the moment I can only confirm that this also fails on x64 Linux devices: Github Actions in CI and my personal Linux device both crash with a segfault.

[MarcusRiemer]

I implemented rudimentary caching to not call crop for every single tile that is placed on the resulting map. Then added a testcase with a large map that simply renders 30 * 30 identical tiles. On my machine this crashes reproducible:

> mix test test/tiled/map_test.exs:125
Running ExUnit with seed: 925133, max_cases: 16
Excluding tags: [:test]
Including tags: [location: {"test/tiled/map_test.exs", 125}]

21:35:40.971 [debug] Loading tiled map from test/tiled/fixtures/007_all_identical_30x30.tmj
21:35:40.980 [debug] Loading tileset from test/tiled/fixtures/kenney_roguelike_spritesheet.tsj

[... SNIP loads of Cache Hit messages SNIP ...]

[1]    40711 bus error

So at the core this boils down to only three images floating around in the system:

1. The target image
2. The tileset image
3. A cropped portion of the tileset image

Image 3 is then placed numerous times on the target image 1. And at some point this crashes. I sadly don't have the time to look further into this at this moment. But I hope this narrows the issue down in a helpful way.

[kipcole9]

Very helpful and much appreciated (debugging NIF crashes is no fun - and it's not really a skill I have either!). I'll dive into it this week to see if I can get to the bottom of this. I may, in the first instance, see if I can reproduce in a python script (using pyvips). If that also crashes then it's most likely in libvips. If not, it may be in vix.

[MarcusRiemer]

I sadly didn't have the time to revisit this. I had a brief hope that maybe the passage of time fixed this with newer versions of the underlying libraries or erlang itself. Sadly no :-(

So I sat down to make a truly minimal reproduction case that could be handily converted to Python. And then I wanted to convert it to Python, but in the meantime life kicked in and all I have at the moment is this testcase:

defmodule Tiled.ElixirImageTest do
  use ExUnit.Case, async: false

  for map_tile_size <- 1..15 do
    test "Render the same tile on very place on a #{map_tile_size} * #{map_tile_size} map" do
      tile_px_size = 16

      tileset = Image.open!("./test/tiled/fixtures/kenney_tilemap.png")
      tile = Image.crop!(tileset, 17, 0, tile_px_size, tile_px_size)

      target = Image.new!(unquote(map_tile_size) * tile_px_size, unquote(map_tile_size) * tile_px_size, color: [0, 0, 0, 0])

      index_locations = for x <- 0..unquote(map_tile_size), y <- 0..unquote(map_tile_size), do: {x, y}

      result = Enum.reduce(index_locations, target, fn {x, y}, accu_image ->
        Image.compose!(accu_image, tile, x: x * tile_px_size, y: y * tile_px_size)
      end)

      assert result
    end
  end
end

Running it in order (mix test test/tiled/elixir_image_test.exs --trace --seed 0) on my machine yields:

Tiled.ElixirImageTest [test/tiled/elixir_image_test.exs]
  * test Render the same tile on very place on a 1 * 1 map (44.2ms) [L#5]
  * test Render the same tile on very place on a 2 * 2 map (1.4ms) [L#5]
  * test Render the same tile on very place on a 3 * 3 map (1.7ms) [L#5]
  * test Render the same tile on very place on a 4 * 4 map (1.8ms) [L#5]
  * test Render the same tile on very place on a 5 * 5 map (3.6ms) [L#5]
  * test Render the same tile on very place on a 6 * 6 map (4.4ms) [L#5]
  * test Render the same tile on very place on a 7 * 7 map (5.8ms) [L#5]
  * test Render the same tile on very place on a 8 * 8 map (7.6ms) [L#5]
  * test Render the same tile on very place on a 9 * 9 map [L#5][1]    292677 segmentation fault (core dumped)  mix test test/tiled/elixir_image_test.exs --trace --seed 0

Interestingly running it with a size of exactly e.g. 30 * 30 will pass the test, but fail afterwards with a segmentation fault. The PNG file in question was obtained from kenney.nl and is attached.

[MarcusRiemer]

I had an idea during lunchbreak and asked Claude AI to convert the test to a standalone python program using pyvips. It came up with the following result that does not crash:

#!/usr/bin/env python3
"""
Standalone Python program to render tiles on a map using pyvips.
Converted from Elixir test code.
"""

import pyvips
import sys
import os

def main():
    # Configuration
    map_tile_size = 15
    tile_px_size = 16
    
    # Path to the tileset image
    tileset_path = "./test/tiled/fixtures/kenney_tilemap.png"
    
    # Check if the tileset file exists
    if not os.path.exists(tileset_path):
        print(f"Error: Tileset file not found at {tileset_path}")
        print("Please run this script from the project root directory")
        sys.exit(1)
    
    try:
        # Load the tileset image
        print(f"Loading tileset from {tileset_path}")
        tileset = pyvips.Image.new_from_file(tileset_path)
        
        # Crop a single tile (16x16 pixels starting at position 17,0)
        print(f"Cropping tile from position (17, 0) with size {tile_px_size}x{tile_px_size}")
        tile = tileset.extract_area(17, 0, tile_px_size, tile_px_size)
        
        # Create target image (15x15 tiles = 240x240 pixels)
        target_width = map_tile_size * tile_px_size
        target_height = map_tile_size * tile_px_size
        print(f"Creating target image: {target_width}x{target_height} pixels")
        
        # Create a transparent background (RGBA with alpha=0)
        target = pyvips.Image.black(target_width, target_height, bands=4)
        target = target.cast(pyvips.BandFormat.USHORT)  # Use 16-bit for better precision
        
        # Place the same tile at every position on the map
        print(f"Rendering {map_tile_size}x{map_tile_size} tiles...")
        result = target
        
        for x in range(map_tile_size):
            for y in range(map_tile_size):
                x_pos = x * tile_px_size
                y_pos = y * tile_px_size
                result = result.insert(tile, x_pos, y_pos)
        
        # Save the result
        output_path = "tile_renderer_output.png"
        print(f"Saving result to {output_path}")
        result.write_to_file(output_path)
        
        print("Success! Tile rendering completed.")
        print(f"Output saved to: {output_path}")
        
    except Exception as e:
        print(f"Error: {e}")
        sys.exit(1)

if __name__ == "__main__":
    main()