Aggregating arrays / lists into a table / csv

[natecdavis]

What's the best way, when running a loop in a DataFlow ,to aggregate many arrays / lists into a single table? Each iteration of the loop produces one array / list, and ultimately I'd like to save the table aggregating all of these to a CSV.

[liamfratturo]

Use an Accumulator to terminate your loop, then pass the output of the Accumulator into the 'Rows' input of a Table Creator module. From there, you can use the CSV Writer to produce an output file.

If the operations performed in your loop are available in SQL, you could consider using the Table Creator on your input data, and then transforming it via a Table Query rather than a loop, but this may or may not be possible depending on your specific use case.

[natecdavis]

Thanks for the response. The issue appears to be that I was trying to aggregate the accumulated output of a Python Script Executor module but hadn't figured out the right way to make that properly "accumulatable". Outputting each array as a table row (via the composable_table_out parameter within the Python Script Executor) and then aggregating the accumulated output with the Table Set Operator module appears to have done the trick.