Skip to content

SAM2.1 Bacteria - Pre-trained features model release#39

Merged
bentaculum merged 22 commits intoweigertlab:mainfrom
C-Achard:cy/release-model
Nov 18, 2025
Merged

SAM2.1 Bacteria - Pre-trained features model release#39
bentaculum merged 22 commits intoweigertlab:mainfrom
C-Achard:cy/release-model

Conversation

@C-Achard
Copy link
Contributor

@C-Achard C-Achard commented Sep 23, 2025

Hello again !

This PR adds code to run inference for models trained with pre-trained features from foundation models directly with the Trackastra API.

Updated models, feature extraction and custom WRFeat code is hosted here and can be installed as an optional dependency.
The aim is to add the new models in the least disruptive way possible, but I still had to make a few changes throughout the code; let me know if the implementation needs tweaked or could be more lightweight.

(Does not include training code as that would be far more cumbersome; the fork can be used as-is for training regardless.)


  • Feature extraction model should use the same device as tracking model
  • All pre-trained feature models imports should be handled case-by-case

Main changes :

  • New optional dependency for custom models, feature extraction and windowed region features creation,
  • Small additions to WRFeat, model API and model for compatibility.
  • A few tweaks, notably making batch size more accessible in inference and directly passing model weights name (if one wants to use latest or another checkpoint that isn't model.pt)

Best,
Cyril

@C-Achard C-Achard mentioned this pull request Sep 23, 2025
11 tasks
@C-Achard
Copy link
Contributor Author

@bentaculum Working on a few remaining issues, but should be ready soon; let me know how we can handle sharing model weights !

@C-Achard C-Achard changed the title Pre-trained features model release SAM2.1 Bacteria - Pre-trained features model release Oct 28, 2025
C-Achard and others added 5 commits October 28, 2025 10:35
- rename model
- add inference test on MPS and CUDA. CPU too slow.
- use trackastra-et-ultra fork for memory-efficient feature computation
- update Readme
@bentaculum
Copy link
Member

@C-Achard thanks once more for this nice addition. I added some changes, most importantly to have the SAM2 part of the tests, and some other small ones.

@C-Achard
Copy link
Contributor Author

Thanks @bentaculum ! Let me know if I can help with anything else

@bentaculum bentaculum merged commit 77b4577 into weigertlab:main Nov 18, 2025
10 checks passed
@bentaculum
Copy link
Member

@C-Achard is your master thesis online somewhere to point to for advertising the new model?

@C-Achard
Copy link
Contributor Author

@bentaculum I've uploaded it on the pretrained-feats repo here :
https://github.com/C-Achard/Trackastra-et-Ultra/releases/download/manuscript/Cyril_Achard_Trackastra_NX_MA_Thesis_15_08_25.pdf

Hope that works, thanks again for all your help !

@bentaculum
Copy link
Member

Thanks @C-Achard.

I just learned that pypi does not support dependencies on Github repos: https://github.com/weigertlab/trackastra/actions/runs/20122805873

Last thing hopefully :D, can you please merge my PR C-Achard/Trackastra-et-Ultra#1, upload trackastra_pretrained_feats to pypi, and set the optional dependency in here to that?

@C-Achard
Copy link
Contributor Author

Hi again @bentaculum ; sorry for missing the PR, it's been merged.

The dependency has been updated here - #51 and the pretained_feats package is on PyPi now.

Let me know if you need anything else !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants