How do you postprocess an existing experiment with a different data processor?

[sbrandstaeter]

Lately, I find myself frequently wanting to reevaluate the results of previous simulation experiments—specifically those run with queens—using a different data processor, or sometimes the same one but with a different configuration.

For example, I might have initially extracted displacement at a specific location in a solid mechanics simulation, but now I want to analyze displacement at a different location.

I’m curious if others run into this situation as well. How do you handle it without resorting to lots of boilerplate code or writing manual loops over the experiment?

Would a built-in, more accessible way to reprocess experiments be something worth implementing?

[leahaeusel]

I have also run into this problem before.

For my purposes, I have written a simple Recycle driver that checks if a simulation with the same job ID and inputs is already there and then processes the data.

I have been meaning to add this functionality to main but haven't had the time yet. I definitely think it is something that we should add to the code.

[gilrrei]

A recap of our in-person (@leahaeusel, @sbrandstaeter, @maxdinkel, @shervas) meeting regarding this topic:

• The idea is to run a data processor on existing data
• as @maxdinkel pointed out correctly, this is sensible for open-loop scenarios, you can't restart an iterative process with a different model, as this would lead to different outputs
• We already have a similar functionality to gather information from the metadata file in each simulation folder, see this

I think a utility to run a data processor on existing data is a nice idea! I like it. Just as a warning, it would be a good idea to enable the extraction in multi-query fashion. From my experience, if you have thousands of simulations to process, this can take a long time