Time-varying training sample bias metric

[brianhenn]

From a conversation with Chris, we proposed logging the time- or sample-varying bias in training. For example, we might want to know how well we learn climate-dependent variables like OLR as they change across a dataset with historical or 1%-CO2 driven warming, particularly as we try to avoid having "clock variables" as inputs.

Since validation usually is for a short time period, we would probably want to compute this metric over the training samples. This would likely involve binning/annualizing the training bias of samples along with their time coordinate value, so that we could log how this bias is varying over time.