The State of 3D Model Previews

[CyanVoxel]

Thumbnail and preview support for 3D models (see #351) including STL and OBJ files has been an ideal feature for TagStudio to have since its inception, however several difficulties involving the available libraries and modules for this have unfortunately put this feature into limbo. Because of this I’ve made the decision to remove the version estimates for this feature from the roadmap and create a centralized discussion here in order to coordinate where to go next with this feature.

The Problems with Open3D

The recent(ish) push for 3D model preview support was kicked off when Open3D finally added support for Python 3.12, the minimum version that TagStudio supports. Unfortunately this was not the magic bullet I was hoping for and the Open3D Python module seems to suffer from extremely reduced performance that is both non-hardware accelerated and locked to a single processing thread. Not only that, but the Open3D Python module seems to always be about an entire version behind the newest Python version and has been a bottleneck for us to update and support newer versions - despite not even having adopted the module yet (see #1047). Because of this I don’t believe that Open3D is a good fit for the project and including it as a dependency would hold back the entire project from using newer Python versions and modules.

The Problems with Other Libraries

As far as I’ve been able to determine in my research and testing, no other established 3D rendering libraries available for Python are able to function either with usable performance or without needing to spawn their own GUI window (PyOpenGL, PyAssimp, Matplotlib, PyQt, etc.). Other solutions such as bundling PyGame or relying on a Blender installation are also just too bloated for what this feature is.

What Now?

Unfortunately the options from here are limited. One possibility that myself and @xarvex have discussed is creating a dedicated GPU-accelerated preview renderer in Rust along with bindings for Python, allowing TagStudio to interact with it. This however is a lofty and intensive solution, and GPU programming isn’t exactly in either of our wheelhouses. This is more of a "technically this would be possible" idealized solution rather than a "we are 100% going to spend the time and effort on this". This approach is also open for anyone else willing to take a stab at it, if you so choose.

It would also be possible to create a rough standalone renderer using one of the less ideal solutions mentioned above that could be used as an external dependency to TagStudio, and would come with whatever quirks apply to the implementation. This would at least give users the option to have some form of thumbnails over nothing at all, but it wouldn’t be a long term solution.

This discussion is both an announcement on the state of where this feature is stuck, as well as a call for alternative implementations that may overcome the issues we’ve faced so far.

[JonatanNevo]

@CyanVoxel Does the deticated renderer needs to have python bindings? Can't we just find/build some lean offline renderer, package against it's binary and just use it in a subprocess?
This can be easier than making a realtime renderer with python bindings