Which weights to use for fastest MobileSAMv2 segmentation?

[misrolox]

I have an NVIDIA GeForce RTX 4050 with 6GB of memory. I always get the error "torch.OutOfMemoryError: CUDA out of memory." when running inference on MobileSAMv2. Specifically:

torch.OutOfMemoryError: CUDA out of memory. Tried to allocate 184.00 MiB. GPU 0 has a total capacity of 5.64 GiB of which 95.25 MiB is free. Including non-PyTorch memory, this process has 3.57 GiB memory in use. Of the allocated memory 3.40 GiB is allocated by PyTorch, and 47.38 MiB is reserved by PyTorch but unallocated. If reserved but unallocated memory is large try setting PYTORCH_CUDA_ALLOC_CONF=expandable_segments:True to avoid fragmentation.  See documentation for Memory Management  (https://pytorch.org/docs/stable/notes/cuda.html#environment-variables)

Is this expected with my GPU?

I try to run the MobileSAMv2 demo with the encoder "efficientvit_l2". If I try running with "tiny_vit", I get the error which is mentioned in another issue:

/usr/local/lib/python3.10/dist-packages/timm/models/layers/__init__.py:48: FutureWarning: Importing from timm.models.layers is deprecated, please import via timm.layers
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.layers", FutureWarning)
/usr/local/lib/python3.10/dist-packages/timm/models/registry.py:4: FutureWarning: Importing from timm.models.registry is deprecated, please import via timm.models
  warnings.warn(f"Importing from {__name__} is deprecated, please import via timm.models", FutureWarning)
/home/repos/MobileSAM/MobileSAMv2/tinyvit/tiny_vit.py:656: UserWarning: Overwriting tiny_vit_5m_224 in registry with tinyvit.tiny_vit.tiny_vit_5m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/repos/MobileSAM/MobileSAMv2/tinyvit/tiny_vit.py:656: UserWarning: Overwriting tiny_vit_11m_224 in registry with tinyvit.tiny_vit.tiny_vit_11m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/repos/MobileSAM/MobileSAMv2/tinyvit/tiny_vit.py:656: UserWarning: Overwriting tiny_vit_21m_224 in registry with tinyvit.tiny_vit.tiny_vit_21m_224. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/repos/MobileSAM/MobileSAMv2/tinyvit/tiny_vit.py:656: UserWarning: Overwriting tiny_vit_21m_384 in registry with tinyvit.tiny_vit.tiny_vit_21m_384. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/repos/MobileSAM/MobileSAMv2/tinyvit/tiny_vit.py:656: UserWarning: Overwriting tiny_vit_21m_512 in registry with tinyvit.tiny_vit.tiny_vit_21m_512. This is because the name being registered conflicts with an existing name. Please check if this is not expected.
  return register_model(fn_wrapper)
/home/repos/MobileSAM/MobileSAMv2/efficientvit/models/nn/ops.py:395: FutureWarning: `torch.cuda.amp.autocast(args...)` is deprecated. Please use `torch.amp.autocast('cuda', args...)` instead.
  @autocast(enabled=False)
Traceback (most recent call last):
  File "[path_here]", line 154, in <module>
    mobile_sam_v2 = MobileSAMV2()
  File "[path_here]", line 31, in __init__
    self.mobilesamv2, self.obj_aware_model = self.create_model()
  File "[path_here]", line 43, in create_model
    mobilesamv2.image_encoder = sam_model_registry['tiny_vit']('./weight/l2.pt')
  File "/home/repos/MobileSAM/MobileSAMv2/mobilesamv2/build_sam.py", line 115, in build_sam_vit_t_encoder
    mobile_sam.load_state_dict(state_dict['model'],strict=False)
KeyError: 'model'

[Shanzhaguoo]

@misrolox Hi, I met the same problem. Did you solve it?