Pointmap contains Inf values causing NaN scale in reconstruction

[Skylarkkkk]

Awosome Work! But when I running SAM3D object reconstruction on my data, the scale computation produces NaN values, causing the reconstruction pipeline to fail with IndexError: list index out of range in make_scene().

Root Cause
The depth estimation model (MoGe) generates pointmaps that contain infinite values (Inf) in regions outside the valid depth range. When use_scene_scale=True is configured, the scale computation in ObjectCentricSSI._compute_scale_and_shift() uses the entire pointmap including these Inf values,resulting in NaN scale values.

And I run a little analysis:

• Pointmap total pixels: 196,608
• Pixels containing Inf: 113,649 (58%)
• Valid finite points in mask region: 1,989 (100% finite)
• Scale computation uses full pointmap → produces NaN

Location
File: sam3d_objects/data/dataset/tdfy/img_and_mask_transforms.py
Lines: 574-578

if self.use_scene_scale == True:
    # Normalize by the scene scale
    points_centered = pointmap_flat - shift.unsqueeze(-1)  # Uses entire pointmap with Inf
    max_dims = points_centered.abs().max(dim=0).values     # max() returns Inf
    scale = max_dims.nanmedian(dim=-1).values              # nanmedian() can't handle Inf → NaN

Impact

• All reconstructions with use_scene_scale=True fail when pointmap contains Inf values
• Causes complete reconstruction failure, not just degraded quality

Solution
Filter out non-finite values (Inf, -Inf, NaN) before computing scale. If no finite points remain in the scene, fallback to object-based scale computation.

Proposed Fix

if self.use_scene_scale == True:
    # Normalize by the scene scale
    points_centered = pointmap_flat - shift.unsqueeze(-1)
    # Filter out non-finite values before computing scale
    finite_mask = torch.isfinite(points_centered).all(dim=0)
    if finite_mask.sum().item() == 0:
        # Fallback to object scale if no finite points in scene
        logger.warning("No finite points in scene pointmap; using object scale instead")
        shifted_mask_points = mask_points - shift.unsqueeze(-1)
        norm = shifted_mask_points.norm(dim=0)
        scale = norm.nanmedian(dim=-1).values
    else:
        points_centered_finite = points_centered[:, finite_mask]
        max_dims = points_centered_finite.abs().max(dim=0).values
        scale = max_dims.median(dim=-1).values

After fix:

• my 21 test masks produce valid finite scale values
• Scale range: [0.027, 1.201] (reasonable values)
• No more NaN or Inf in scale computation
• Reconstruction pipeline completes successfully