Energy Efficiency

[rafecchang]

Objective:

• Reduce model size

Steps:

1. Train a disterbert model on the optimized hyperparameters. Use this as a baseline model.
2. Apply purning (likely hyperparameter optimization for only the purning-related hyperparameters.)
3. Apply post-training quantization (likely hyperparameter optimization for only the quantization-related hyperparameters.)
4. Fine tune with hyperparameter optimization