Part of our API accepts human rationale annotations to evaluate explanations on plausibility metrics. It is reasonable to assume that human workers will annotate text with such rationales using open-source tools such as Label Studio.
In this regard, we might want to support off-the-shelf some of the output formats these platforms have. E.g:
- Label Studio: list of spans, where each span is a tuple with (starting index, ending index)
Part of our API accepts human rationale annotations to evaluate explanations on plausibility metrics. It is reasonable to assume that human workers will annotate text with such rationales using open-source tools such as Label Studio.
In this regard, we might want to support off-the-shelf some of the output formats these platforms have. E.g: