Support for consumer GPUs (10GB-12GB VRAM) and Quantization

[helloimflo]

Hi team,

Thank you for this amazing work on tttLRM! I am very interested in testing your model for single-image to 3D Gaussian Splatting reconstruction.

I am currently using an NVIDIA RTX 3080 with 10GB of VRAM. Given the "Large" nature of the model, I am concerned about OOM (Out of Memory) issues.

Do you plan to release a quantized version (4-bit or 8-bit) of the checkpoints?

Are there specific inference parameters you recommend to minimize VRAM usage for single-view reconstruction?

Would you consider providing a "Small" or "Tiny" version of the model for users with consumer-grade hardware?

Best regards.

[cwchenwang]

Thanks for your interest in our work! Our model is hard to be compatible with 10GB VRAM. For quantization, it is generally hard for 3D models because it might affect the final quality a lot. Compromises I can think of is using multiple 3080 GPUS with sequence parallel or reduce the resolution of input images (which might affect the final quality). Also, our model might not be suitable for single-view reconstruction tasks. I apologize that I might not conider the "Small" or "Tiny" version since I currently don't have resources for that.

[helloimflo]

Thanks for your honest feedback regarding VRAM and the "Single-view" suitability.

I was curious about the single-image examples shown on your project page. If the model isn't specifically optimized for single-view, did you use a multi-view diffusion model (like MVDream or Wonder3D) as a pre-process to generate synthetic views, or is the model capable of "hallucinating" the full 3DGS structure from just one set of input tokens?

I've tried using Video-Generation models to create orbital views before feeding them into reconstruction pipelines, but they often introduce unwanted character movement. Your results look much more geometrically consistent.

Thanks again for the clarification!

[cwchenwang]

Yes, we use a multi-view diffusion model (based on video model) to generate multi-view first. However, that model is not able to be released due to commercial use.