Error during Finetuning (AttributeError: 'CLIPTextEmbeddings' object has no attribute 'embedding_forward')

[lostnull]

Got the following error while trying to finetune the model. Python package versions used are same as in the environment.yaml file

Traceback (most recent call last):
  File "<string>", line 1, in <module>
  File "/usr/lib/python3.8/multiprocessing/spawn.py", line 116, in spawn_main
    exitcode = _main(fd, parent_sentinel)
  File "/usr/lib/python3.8/multiprocessing/spawn.py", line 126, in _main
    self = reduction.pickle.load(from_parent)
  File "/root/fine-tuning/ProSpect/.venv/lib/python3.8/site-packages/torch/nn/modules/module.py", line 1177, in __getattr__
    raise AttributeError("'{}' object has no attribute '{}'".format(
AttributeError: 'CLIPTextEmbeddings' object has no attribute 'embedding_forward'

Python package versions

• taming-transformers-rom1504==0.0.6
• transformers==4.18.0
• torch==1.10.2+cu111
• torch-fidelity==0.3.0
• torchmetrics==0.6.0
• torchvision==0.11.3+cu111
• pytorch-lightning==1.5.9

[lostnull]

Update: It's working fine with single GPU training, but for multi GPU the error persists

[carvychen]

hi, sorry for bothering you. Do you know about the input param "opt.embedding_manager_ckpt"?

[lostnull]

@carvychen

It's the path to the embedding checkpoint which u want to initialize from. If you are training a new embedding, you won't need to set this parameter but if you want to use a previously trained embedding for the new training, you can set the respective path.

Example
python main.py --base configs/stable-diffusion/v1-finetune.yaml -t --actual_resume ./models/sd/v1-5-pruned.ckpt -n obj --data_root data/obj --embedding_manager_ckpt logs/obj-v1_prospect.pt

Where 'logs/obj-v1_prospect.pt' is the embedding checkpoint from a previous training.

[carvychen]

Thanks for your kind help.