Model tokenizer is the same to reference model.

[Harry-mic]

Hello, Thanks for your awesome work!

I notice that the tokenizers of value model and ref_model are set the same in tinyllama_hh.py line 106:

tokenizer = ref_tokenizer =AutoTokenizer.from_pretrained(vas_config.model_name)

So I wonder if the value model and ref_model in this setting must use the same tokenizer. Are other LLMs such as Mistral-7B as ref-model and tiny-llama as value model effective?

Thanks for your reply and time !