support configurable output data writers in recorder

[ahmed-shariff]

This PR refactors the recorder infrastructure to support configurable output data writers. The IDataWriter interface is now generic on a new IDataWriterInfo, allowing data writers to receive configuration via info objects (FileDataWriterInfo, NetworkDataWriterInfo) when instantiated. StartRecordingInternal and StartRecording accept an array of IDataWriterInfo to flexibly control recording destinations and configurations at runtime. This enables future extensibility, such as adding new writer types or enabling user selection of outputs. Default behavior remains writing to a file when no outputs are specified. Added info classes for file and network writers to contain configuration like file paths and network endpoints.

An alternative to having IDataWriterInfo is to directly pass the IDataWriter instances, and pass the record instance through an Initialize method.

This is primarily motivated by wanting to control when data recording starts and stops and as an extension also specify where and how the data would be recorded.

If necessary, the implementations allows adding support to configuring the writers from settings as well.

Let me know what you think about this.

[cjaverliat]

I think it would be better to directly give a list of IDataWriter[] as input to StartRecordingInternal, avoiding this switch to be changed for every new IDataWriter type. Additionnally, this would get rid of the DataWriterInfo class as the DataWriter could be constructed directly outside of the recorder and it wouldn't be the responsibility of the recorder to construct the writers.

[ahmed-shariff]

I would agree that it would be a better approach. However, the record object needs to be injected. Let me try restructuring that to see what it could look like.

[ahmed-shariff]

I refactored to allow for injecting the recorder. Let me know what you think.

[cjaverliat]

Thanks for taking the time to work on this :)
I like the direction toward more flexible and configurable data writers.

A few thoughts after going through the changes:

• Regarding the idea of using an Initialize method on IDataWriter: that approach feels a bit non-standard for writer interfaces. It might be cleaner to pass the necessary metadata or config (like Record info) directly when creating the writer. That said, I realize doing so might require some refactoring of existing classes.

• I think the current architecture, as we designed it, with Record, Recorder, RecorderContext, and DataDispatcher might be unecessarily complex, and that’s making it harder to support multiple data writers cleanly. For example, Record currently holds the serialized data chunks in memory, which are then pulled by the DataDispatcher to be written to the file/network data writers. The Recorder methods like Recorder.RecordTimelessSample, Recorder.RecordMarker, etc simply deflect the calls to _context.CurrentRecord.RecordTimelessSample, _context.CurrentRecord.RecordMarker and so on. It might be worth considering merging some of these classes. I think a simpler structure, where the Recorder contains its recorder modules (currently stored in RecorderContext), a structure to hold the data buffers, and handles recording logic directly, could simplify things and make multiple writer support cleaner.

• Specifically, the Record class might not be needed anymore. If we move the buffer storage into Recorder, then methods like RecordTimelessSample or RecordMarker could just write directly to those buffers, without routing through _context.CurrentRecord.Record....

Of course, this implies bigger change to the PR, I would be happy to discuss further or help break it down if we want to pursue this refactor and simplification.

[ahmed-shariff]

Regarding the idea of using an Initialize method on IDataWriter: that approach feels a bit non-standard for writer interfaces.

I agree, I find dependency injection to nearly always gets messy.

It might be cleaner to pass the necessary metadata or config (like Record info) directly when creating the writer.

I see your point about Recorder, Record, RecordContext and DataDispatcher. From a cursory glance, it should be possible to consolidate these classes into one? However, in the context of this PR, the record is being passed only to get the metadata. For now I'll try to restructure this to use a metadata setter.

[ahmed-shariff]

My understanding is that RecordMetaData itself cannot be made available to the writer objects when they are created, as the metadata is set up when the recording starts. Hence, only the metadata is injected. This way, all other setups during the construction of the writer are left as is. Let me know if this makes more sense for now.

I can take a stab at the refactor you mentioned above in another PR.