Epic: Terrain — automatic per-cluster collision loading driven by entity positions

[martinjms]

Type: epic / interface and implementation plan. Replaces the original framing with the per-engine MapProvider interface decided in the 2026-05-03 architecture sessions.

Quick Summary

• 🟡 Goal: game developers never insert terrain geometry into a cluster's physics world by hand. The cluster runtime owns chunk-loading; the game implements a per-engine MapProvider that the runtime calls.
• Game owns the storage decision (object storage / SpacetimeDB / on-disk / procedural / hybrid) and the authoring tool (UE editor / Unity scene / Godot scene / Blender / voxel editor / generator).
• Arcane owns the interface (MapProvider per engine), the chunks-needed computation (driven by entity positions), and the integration with the cluster's physics engine (handing collision data to Rapier / Chaos / PhysX / Bullet / Jolt).
• The same interface supports static mesh terrain, voxel terrain, and procedural terrain — implementation differs per game.
• Per-engine API discipline applies (per entity-model.md §8): each engine plugin defines its own MapProvider in its native language. No shared MapProvider type across Rust/UE/Unity/Godot.

Why this matters

Terrain is the substrate that physics needs to do anything useful. A cluster owning 3 players in the eastern desert must have eastern-desert terrain colliders in its physics world for raycasts and entity-vs-floor collision to work. It must NOT have polar-icecap terrain loaded — irrelevant cost. As entities migrate (PGP affinity may relocate a player westward), the terrain set must update.

Having game developers manage this is a non-starter:

• They'd hardcode chunk loads in on_tick, defeating dynamic clustering.
• Every game would reinvent the chunk-load logic, badly.
• Terrain ownership conflicts with cluster ownership unless centrally coordinated.
• Cross-cluster terrain coherence (raycasts near chunk boundaries) needs a single source of truth.

Arcane should compute the right terrain set automatically and orchestrate loading via a game-implemented provider.

What "terrain" is in Arcane

Terrain is the only persistent world layer that's not an entity. Three terrain shapes need to work in Arcane, all through the same interface:

Terrain shape	Storage (game's choice)	Mutability	Example
Static / mesh	Object storage (S3, CDN, on-disk asset bundle)	Read-only at runtime	UE level export, Unity scene, glTF import
Voxel	SpacetimeDB (voxel grid as durable state — every block edit persists)	Mutable per-block, durable across sessions	Minecraft, Valheim, Astroneer
Procedural / hybrid	Seed in SpacetimeDB; geometry generated on demand; modifications stored as durable diffs	Effectively mutable via overrides	No Man's Sky, procedural sandboxes

If a game has destructible structures that are entity-flavored (a wall placed by a player that takes damage), those are entities, not terrain edits. Voxel chunks are terrain, just with mutable durable state at chunk granularity rather than per-block entity_ids.

The interface — MapProvider

Per-engine plugin defines its own MapProvider. The Rapier-Rust shape:

pub trait RapierMapProvider: Send + Sync {
    /// Compute which chunks need to be loaded given the cluster's currently
    /// owned entity positions. Pure function; called per cluster tick or
    /// on entity arrival/departure events.
    fn chunks_in_range(&self, entity_positions: &[Vec3]) -> Vec<ChunkId>;

    /// Fetch the collision geometry for a chunk. Implementation reads from
    /// wherever the game stores it — game's choice.
    fn load_chunk(&self, chunk_id: ChunkId) -> Result<ChunkCollision, MapError>;

    /// Optional: called when the runtime is notified (via Redis pub/sub on
    /// the existing replication channel) that a chunk's state changed in
    /// another cluster. Default impl is no-op; mutable-map providers
    /// (voxel, destructible) override to invalidate caches and trigger
    /// reload from the local chunk source. Static-map providers ignore.
    fn on_chunk_changed(&self, _chunk_id: ChunkId) {}

    /// Optional: clean up when a chunk is no longer needed by the cluster.
    fn unload_chunk(&self, _chunk_id: ChunkId) {}
}

pub enum ChunkCollision {
    TriMesh { vertices: Vec<Vec3>, indices: Vec<[u32; 3]> },
    HeightField { width: usize, height: usize, samples: Vec<f32> },
    // Voxel games typically convert to TriMesh via greedy meshing
    // before returning, or expose per-block boxes if blocks are sparse.
}

pub struct ChunkId(pub u64);  // game-defined opaque id; 64 bits is fine for any practical world

The UE plugin defines IArcaneUnrealMapProvider returning UE-native collision (UStaticMesh* / ULandscapeComponent* / equivalent). Unity defines its own with Mesh / MeshCollider. Godot defines its own returning ConcavePolygonShape3D. Same conceptual contract, different language and types — per the per-engine API discipline.

Per-cluster runtime contract

1. Cluster startup: instantiate the game's MapProvider. Optionally read a manifest from SpacetimeDB to know which chunks exist.
2. On entity arrival (via add_entity / first-sight): call provider.chunks_in_range(...) with the new owned-entity-position set, diff against currently-loaded chunks, call provider.load_chunk(...) for newly-needed ones, insert resulting collision geometry into the cluster's physics world as fixed bodies.
3. On entity departure (pending_removals / migration out): same diff; remove chunks no longer needed by any owned entity; cleanup their physics bodies.
4. Per tick (low-frequency): optionally recompute the chunks-needed set as entities move. Chunks shift slowly relative to tick rate, so this can be coarse (e.g., every 10 ticks).
5. Chunks loaded as RigidBodyType::Fixed bodies (per-engine equivalent). Solver-skipped; only AABB tracking. Per-tick cost essentially zero.

Where things live (typical layout)

Data	Storage
Static map content (mesh files, prebaked geometry)	Object storage / asset bundle / on-disk — game's choice
Voxel terrain content	SpacetimeDB (durable per-chunk row)
Map manifest (chunk catalog, version pointers)	SpacetimeDB row(s) — small, always available, read at cluster startup
Mutable per-chunk state (destruction events on a mesh chunk, voxel diffs, runtime modifications)	SpacetimeDB for durability + Redis pub/sub for real-time cross-cluster sync
Per-chunk entities (placed structures, drops)	SpacetimeDB — already entities, already durable

Cross-cluster coordination for mutable terrain

For mutable terrain (voxel chunks, destructible mesh chunks, runtime-edited geometry), cross-cluster sync uses the same Redis pub/sub channel pattern Arcane already uses for entity replication. SpacetimeDB is the durable source of truth; Redis is the real-time replication layer.

When Cluster X modifies a chunk:

1. Cluster X writes the change to SpacetimeDB (durable transactional row).
2. Cluster X publishes the modification on Redis (real-time, on the existing replication channel).
3. Cluster Y, with the same chunk loaded for its own owned entities, receives the Redis notification.
4. Cluster Y's MapProvider::on_chunk_changed(chunk_id) is called by the runtime.
5. Cluster Y's provider invalidates its local cache and reloads the chunk's collision geometry (from SpacetimeDB if needed, or from a local cache that's just been refreshed).
6. On any cluster restart: rehydrate from SpacetimeDB; Redis events from before the restart are already reflected in durable state.

This is the same write-to-SpacetimeDB-and-publish-on-Redis pattern Arcane uses for EntityStateDelta. Voxel chunks and destructible-terrain modifications are conceptually just another kind of cross-cluster game state that's both immediate and durable. Don't propose "SpacetimeDB pub/sub" or "Redis as durable storage" — both are wrong by Arcane's design (memory: project_redis_vs_spacetimedb_split.md).

Engine-native streaming bridging

The MapProvider's role differs by engine. For engines with sophisticated native streaming systems (UE / Unity / Godot), the plugin's MapProvider wraps the native system rather than re-implementing chunk loading. For engines without native streaming (Rapier-Rust pure, voxel games regardless of engine), the plugin's MapProvider is fully implemented by the game.

Engine	Native streaming system	Plugin's MapProvider role
Unreal	World Partition (UE5) — automatic spatial cell load/unload, async asset loading, distance streaming	Plugin's default impl wraps World Partition: ChunksInRange → WP cell ids → WorldPartitionSubsystem::LoadCells(...). Plugin reads collision from whatever WP loaded. Game devs author in UE Editor; for typical mesh-terrain games they don't override anything.
Unity	SceneManager (additive scene loading) + Addressables	Plugin's default impl wraps SceneManager.LoadSceneAsync(..., LoadSceneMode.Additive). Provider returns scene names / addressable keys; Unity handles asset acquisition.
Godot	Scene loading (.tscn) + ResourceLoader	Plugin's default impl wraps ResourceLoader.load_threaded_request(...) and adds resulting nodes to the scene tree.
Rapier-Rust pure / voxel / custom	None native	Game implements full MapProvider: reads from SpacetimeDB chunks / generator / asset bundles and returns collision data directly.

So for traditional mesh-terrain games on UE/Unity/Godot, the dev workflow is "author levels in the engine editor; plugin handles everything." For voxel/procedural/custom games, the dev implements a custom MapProvider that reads from wherever they store map data.

Map types — capability checklist

Each engine plugin needs these capabilities. Start from the basics; layer up as games need them. The first row is V1 baseline for any engine plugin; subsequent rows are added when the first game using that pattern is built.

• Static mesh (engine-native streaming bridge) — baseline
  • Read-only at runtime; map authored in engine editor (UE Editor / Unity Scene / Godot scene / Blender export for Rapier).
  • Plugin's default MapProvider impl wraps the engine's native streaming.
  • No cross-cluster mutation coordination needed.
  • Cross-cluster boundaries handled by loading neighbor chunks locally (cheap for static data).
• Voxel maps
  • Voxel data in SpacetimeDB (per-chunk durable row).
  • Chunk modifications: write durable to SpacetimeDB + publish on Redis pub/sub channel.
  • MapProvider on_chunk_changed(chunk_id) called when neighbor cluster modifies a chunk this cluster has loaded.
  • Plugin provides voxel-chunk hooks; game implements the voxel-specific load + meshing logic.
• Procedural / seed-based maps
  • Seed in SpacetimeDB.
  • Geometry generated on demand from seed (deterministic across clusters).
  • Modifications stored as durable diffs in SpacetimeDB; published on Redis like voxel.
  • Game implements generator + diff-overlay logic.
• Instanced maps (raid dungeon, BR round, sharded realm)
  • Cluster carries an instance_id in its config / SpacetimeDB row.
  • Game's MapProvider scopes chunk loads to that instance id.
  • Different instances are independent — no cross-instance map coherence.
• Hybrid (composite provider) — multiple map types in one game
  • E.g., UE static base world + voxel destructible regions; or static main world + procedural dungeons.
  • Game implements a composite MapProvider that routes by chunk_id range / region / instance.
  • Plugin documents the pattern; no special framework needed.
• Cross-cluster boundary visibility for raycasts (advanced)
  • Each cluster loads a small overlap of neighbor chunks locally for queries that cross the boundary.
  • Coordinated with the cross-cluster physics work (separate epic).

The plugin doesn't need to implement all rows on day 1. Plugin needs:

1. The interface definition (so games CAN implement any pattern).
2. Default implementation for the basic case (row 1).
3. Documentation + example game patterns for the others, lit up as concrete games adopt them.

What Arcane does NOT provide

• A map asset format. Game decides.
• An authoring tool. Game uses its engine's editor or generates procedurally.
• A default storage backend. Game picks SpacetimeDB / object storage / on-disk / procedural / hybrid.
• Voxel-specific or mesh-specific implementations. The interface is uniform; game's MapProvider impl is type-aware.
• Terrain replication. Map content is the same on every server (every node has the manifest + can read storage); not on the wire.

Open design questions

• Default MapProvider impls? Arcane could ship a few common cases (e.g., "TriMesh from glTF in S3"), reducing boilerplate for typical mesh-terrain games. Voxel and procedural impls stay game-specific. Probably worth a small companion library, deferred until first user.
• Chunk-id schema. Game-defined opaque u64 is the proposed default. Alternatives: structured (x, y, z) tuple, hierarchical / quadtree id, hash-based. Game's choice; runtime treats it as opaque.
• Cross-cluster boundaries. When an entity is near a chunk boundary, the neighbor cluster's chunks must be visible for raycasts. Same kinematic-proxy mechanism as cross-cluster entities; or a shared read-only chunk cache; or replicate chunk-id sets across clusters. Deferred to the cross-cluster physics work.
• Hot reload / live edits to static mesh terrain. Out of scope. Map is read-only at runtime in this epic. Live editing for voxel games is straightforward (it's just mutable durable state); for mesh games, deferred.
• Coordination with PGP clustering. Does the clustering model need to know about chunks, or is chunk loading purely a downstream consequence of entity positions? Likely the latter: clustering stays affinity-driven; terrain loading is a runtime side-effect. Note: spatial-bound entities (Fixed body kind, structures, walls) DO depend on chunk ownership for migration semantics — that's the related clustering-binding work, separate epic.

Out of scope

• Map authoring tooling (separate concern; uses the game engine's own editor).
• Destructible terrain (modeled as entities, not terrain edits — see entity-model.md).
• Cross-cluster terrain coherence specifics (separate epic once cross-cluster physics work begins).
• Per-pool terrain optimization (e.g., a logic-only ambient pool may not need terrain at all if its members never raycast).

Acceptance criteria

• MapProvider trait shape designed and documented for the Rapier-Rust plugin.
• ChunkCollision enum (or equivalent) defined with TriMesh + HeightField variants.
• Per-cluster terrain-loading runtime implemented in arcane-infra::rapier_cluster — agnostic of where chunks come from (game's MapProvider decides).
• Cross-cluster boundary handling proposed (implementation may be a follow-up).
• UE plugin equivalent: IArcaneUnrealMapProvider interface designed in parallel; tracked in the Unreal Cluster Node epic (#124).
• Demo: a small open-area demo (in arcane-demos/) where entities walk on terrain, fall into pits, and shoot bullets that hit walls — all without the demo author writing chunk-loading code. The author writes a MapProvider impl that reads a glTF; everything else is plumbing.

Reference

• Architecture doc: docs/architecture/entity-model.md §7 (terrain) and §8 (per-engine API discipline).
• Builds on: #117 (Rapier minimum integration), #118 (per-entity colliders + contact events), the Rapier physics work in flight via PR #123.
• Related: PGP / Affinity Clustering — entities migrate by affinity; chunk-load set follows.
• Related: #33 (heterogeneous node tiers — terrain loading is per-cluster regardless of node kind).
• Related: #8 (cluster physics backends — terrain colliders live in whichever physics engine the cluster uses).
• Parallel work: #124 (Unreal Cluster Node) — UE-native MapProvider design lives in that epic; same conceptual contract.

Memory anchor for future sessions

• Per-engine API pattern (project_per_engine_api_pattern.md in memory): each engine plugin defines its own MapProvider in its native language. Arcane does not try to share MapProvider types across Rust/UE/Unity/Godot.
• Unified entity model (project_unified_entity_model.md in memory): terrain is the one thing that's not an entity. Every persistent in-world thing (player / NPC / projectile / structure / dropped item) is an entity; every persistent in-world substrate is terrain.
• Wire format stays engine-neutral; terrain doesn't cross clusters via wire format because every cluster reads the same map content (manifest + storage).