Auxiliary output semantics

[ixjlyons]

dump_aux_fn is a useful "hook" to help figure out why a regression test is failing without having to modify test code or deal with loading/visualizing data separately, particularly when the diff printout doesn't quite cut it. I was able to extend NDArraysRegressionFixture to pass in plotting callbacks in check(), but it's still a bit limited. Here's an example:

def plotter(sim):
    fig, ax = plt.subplots()
    ax.plot(sim.t, sim.x[0])
    return ax


def test_something(custom_ndarrays_regression):
    sim = run_sim()
    custom_ndarrays_regression.check(
        sim,
        plotters=dict(x0=plotter),  # internally wrapped to satisfy dump_aux_fn interface
    )

The source/expected auxiliary files are currently only generated if the source/expected data file doesn't already exist (or with regen-all), which indicates the intention is to commit those auxiliary outputs. However, at times it might be necessary to add new auxiliary outputs or change them in some way to better see what's going on with a test failure. In the example above, maybe I need to set fixed axis limits so you can flip back and forth between expected and obtained to see a change more clearly. In this use case, it would be nice to always re-generate the source/expected auxiliary outputs with whatever the current dump_aux_fn is, loading the existing source/expected data. A minor side benefit is you are always hitting your dump_aux_fn code path, helping prevent it from becoming stale, albeit with a performance hit.

This is I think a one-line addition to perform_regression_check, but I realize it may be considered a breaking change if people are using dump_aux_fn and committing those files. None of the existing fixtures seem to use it, but people could always be extending the fixture implementations to do so.

Would an additional force_aux_regen=False argument be acceptable? Happy to submit a PR.