## Update: The official TreeAIBox plugin is now published at
NRCan/TreeAIBox on GitHub
Please use that repository for the latest features.

Please use that repository for the latest features.

cc-TreeAIBox-plugin

AI-Enhanced Toolset for 3D Tree Processing: A CloudCompare Plugin (alpha 0.1)

This is a Python Plugin for CloudCompare. You can register the python file TreeAIBox.py as the main program. This is just a fresh prototype with bugs and imperfections. I will release a stable version after enough tests. The code format also needs standardization.

Note (Jan-21-2025): The main program on tree segmentation will be uploaded by the end of Feburary 2025.

Note: Tested only in Windows OS, RTX3090 (VRAM: 24GB) With CUDA; I am currently making the TreeAIBox installable akin to the 3DFin plugin, thanks to Romain Janvier's suggestion!

Requirements

There are several Python library dependencies:

pip install torch torchvision torchaudio --index-url https://download.pytorch.org/whl/cu121
pip install scikit-image==0.22.0 timm numpy-groupies numpy-indexed scikit-learn circle-fit

Make sure the scikit-image version is correct. The numpy version should be 1.xx (below 2.0). You can either use the Python package manager of the CloudCompare, or navigate to the default python interpreter of the CloudCompare and run the pip install commands via CLI.

Important: If you have installed CloudCompare to Program Files, there might be issues with pip being prohibited from installing to the default site-package folder. Please run pip command or launch the CloudCompare as the administrator.

Basic Modules (Updated Aug 22, 2024)

WoodCls

This module classifies the stem and branch components from a tree scan based on the 3D SegFormer deep learning model.

• Please download the model before applying the classification.
• The classification results will be presented as scalar fields of stemcls and branchcls.
• More trained AI models will be added in the future.

QSM

This module reconstructs wood architecture from a tree scan. There's much space for improvement.

• The initial segmentation isolates point clusters.
• Stem and branch have customizable scale parameters (MeanShift bandwidth).
• Clusters should be large enough to cover the stem cross-section.
• After skeletonization:
  • Wood architecture will be saved as an XML file
  • Meshes will be saved as an OBJ file
  • Skeleton curves are drawn on CloudCompare

Future upgrades may include an AI-based version of this tool.

Known Issues

1. Occasionally flashing: Sometimes the program will flash due to the computation overhead. Please check first if your pytorch+cuda is installed properly. Note that pytorch may not be compatible with most recent CUDA versions (e.g. 12.4+) which might need be downgraded to a previous version (e.g. 12.1).

3. Curve issues:

   • The node connection sometimes is poor. The current version of my QSM is sensitive to the point gaps;
   • The joints between two branches sometimes do not follow the point cloud forms

4. Short stems and branches: The final curves and meshes do not fully capture the length of stems and branches. This is because centroids from initial segments were used as nodes, whereas branch tips need to be added as tree nodes.

5. Mesh visualization: Need to visualize the meshes directly, instead of saving and loading the meshes.

6. Performance on certain tree types: Current algorithm works poorly on branchy and fat trees (tropical trees).

UPDATES

(Sept 20, 2024)I've updated WoodCls.py with a main function and created a model folder containing the trained branchcls model. To use your own data, simply replace the las/laz files in the data folder and execute the main function from WoodCls.py if you encounter any issues on the CloudCompare platform. The output will be laz files stored in a newly created output folder located in the same directory as WoodCls.py.

TreeAIBoxQSM_demo.mp4