Add temperature curve support

[devdaniel]

Adds a temperature_curve utility and allows for passing some additional parameters to better control temperature during inferencing. This allows the temperature to interpolate (currently linear or cosine) over the course of generation. This helps to mitigate precision loss accumulation on longer runs or with extreme topk and cfg_scale values by starting warm and cooling down.

Added to examples/run_music_generation.py

• --temperature_end (default: none) If set, will adjust the temperature throughout the run from the starting value in --temperature to this value according to the set schedule.
• --temperature_schedule (default: "linear") Will use this schedule to define the temperature curve. Currently implemented curves are "linear" or "cosine".

[frink]

Wonder if a variation of this could be used to help with genderization issues?

[devdaniel]

Wonder if a variation of this could be used to help with genderization issues?

This is intended to help improve stability throughout the generation. Starting warm allows it to be more creative, and cooling down helps reduce the drift that gets multiplied every 30s.

The gender issue is a tagging/classification issue unrelated to this.

[frink]

I didn't understand it initially. This is actually a really cool idea.