Lionstudy

Study tuning lion optimizer's hyperparameters on LLM fine-tuning tasks under fixed budget.

Name		Name	Last commit message	Last commit date
Latest commit History 58 Commits
__pycache__		__pycache__
lion-pytorch		lion-pytorch
mylion_pytorch		mylion_pytorch
tokenizer		tokenizer
BertTrain.ipynb		BertTrain.ipynb
Bertfinetune.ipynb		Bertfinetune.ipynb
GLUE.py		GLUE.py
GLUEGPT.py		GLUEGPT.py
GLUEGPT2.py		GLUEGPT2.py
GLUEforQQP.py		GLUEforQQP.py
GPT2.ipynb		GPT2.ipynb
GPT2train.ipynb		GPT2train.ipynb
GPTTrain.ipynb		GPTTrain.ipynb
LICENSE		LICENSE
README.md		README.md
Yeger.ipynb		Yeger.ipynb
colabatchsize32schedulernooptimizerLionLR1e-05acc.npy		colabatchsize32schedulernooptimizerLionLR1e-05acc.npy
colabatchsize32schedulernooptimizerLionLR1e-05loss.npy		colabatchsize32schedulernooptimizerLionLR1e-05loss.npy
colabatchsize32schedulernooptimizerLionLR1e-05metric.npy		colabatchsize32schedulernooptimizerLionLR1e-05metric.npy
log1.txt		log1.txt
logQQP.txt		logQQP.txt
logQQP2.txt		logQQP2.txt
logQQP3.txt		logQQP3.txt
plot.ipynb		plot.ipynb
tokenizer.zip		tokenizer.zip

Provide feedback