Missing consensus review workflow documentations

[JonnyTran]

[FotisSik]

hey @JonnyTran !

Thanks for sharing your fork of Argilla. I noticed it’s intended to support a review workflow, which is exactly what our team has been looking into.

I tried running your version locally, but I wasn’t able to find any UI or documentation on how to start or use the review process. I’d really appreciate it if you could clarify how it’s supposed to work.

If there’s anything I missed in the code or any usage notes you could point me to, that would be super helpful.

Thanks again for your work on this!

Best regards,
Fotis

[JonnyTran]

Hi @FotisSik, thanks for reaching out!

Currently, the documentation page is missing for the consensus review workflow in Extralit - but new contributors are actively working on fixing this. The two features we have added on top of Argilla are:

1. Admin users can view responses from multiple annotators across multiple types of questions in a record. Admin users can determine consensus and submit a "Validated" record.
2. A dataset record aggregation module to select the Validated records among other Submitted records.

I'll ping you here in this Issue when the documentation is out and the workflow is refined. Additionally, I'd invite you to consider sponsoring our ongoing development to ensure this feature continue to evolve with your research needs.