eval composite_n

[Yizhxy]

i use your code and trained models,but got the results(have changed temperature values)，is there anything wrong ？

[ParagonLight]

Hi, could you give me more detailed information about this try?

[Yizhxy]

Hi, could you give me more detailed information about this try?
of course,i only changed the config，meteora_ckpt_path: 'ckpt/llama3_8b/llama3_8b_meteora/top_1',and 'self.top_k = 1 self.T = 15',and the command is ‘nohup python -u eval_model.py --task composite_3 --batch_size 16 > log/eval_cp3.log’

[Yizhxy]

When I use topk=2 everything is fine

[ParagonLight]

When I use topk=2 everything is fine

You run the test of composite? Yeah, this test should be done by top-2. Since the LLM itself requires time to switch the LoRA during the generation. When k=2, the second LoRA adapter could participate the generation and gradually influence the result.