Running Chai for LLama 8B model

[schopra6]

Hi, I tried running Chai setup with Llama 8B model weights from huggingface. After running the example provided , it starts generating gibberish text. Here is the repository if you want to have a look at the changes https://github.com/schopra6/chai.

[bilgeacun]

Can you try setting the args.prune_layer to 19 and see how it works please? I think we missed including this parameter config in the repo.