Draft documentation of Croissant metadata process / workflow for MC2

[aclayton555]

Based on pilot process in #208 and expansion in #226 and #227, we should develop an initial draft of the Croissant metadata process, related automations, relevant tables, etc. in place.

To the extent possible, it would be good to develop this in a generalized manner with reference to project-specific processes (e.g. MC2, NF) as a resource for other projects (e.g. Orion notes that Bridge2AI might also be interested in this, so it would be helpful for that team to reference the efforts that have already been established). Some reference documentation will also be helpful for the Tech team to reference in the context of their exploration of how to approach download of external datasets (see https://sagebionetworks.jira.com/browse/TECH-186), as it is expected that the learnings and technical aspects of this process may inform that approach.

Let Ashley know if the preference is to start this in Google docs, or if a placeholder page in Confluence would be helpful!

[aclayton555]

Documentation for Croissant metadata generation (incomplete as of now): https://sagebionetworks.jira.com/wiki/pages/resumedraft.action?draftId=4408311829&draftShareId=364bbb4f-6320-4ab3-b3ee-3e59bfd0c461

@aditya-nath-sage I don't seem to be able to access this. Can you clarify which Confluence space this is in?

[aditya-nath-sage]

Just added you as an editor to the page (it is a draft for now, you me and Orion have edit access so far). Let me know if you still are having issues! I placed it in the [Hidden] SOPs section of the CCKP docs

[aditya-nath-sage]

Added note about dataset collection versioning (from exploration that tom and I did on why there were version mismatches in multiple tables from the croissant dataset collection)