diff --git a/docs/source/includes/tags/chat.md b/docs/source/includes/tags/chat.md new file mode 100644 index 000000000000..210745f94ee3 --- /dev/null +++ b/docs/source/includes/tags/chat.md @@ -0,0 +1,12 @@ +### Parameters + +| Param | Type | Description | +| --- | --- | --- | +| name | string | Name of the element | +| value | string | Data field containing an array of chat messages or empty array | +| [messageroles] | string | Comma-separated list of roles that the user can create and send messages on behalf of. Default is "user" if the `llm` parameter is set; default is "user,assistant" if not. | +| [editable] | boolean \| string | Whether messages are editable. Use true/false, or a comma-separated list of roles that are editable | +| [minmessages] | string \| number | Minimum total number of messages required to submit | +| [maxmessages] | string \| number | Maximum total number of messages allowed | +| [llm] | string | Model used to enable automatic assistant replies, format: `/` | + diff --git a/docs/source/tags/chat.md b/docs/source/tags/chat.md new file mode 100644 index 000000000000..2a41b43150d7 --- /dev/null +++ b/docs/source/tags/chat.md @@ -0,0 +1,137 @@ +--- +title: Chat +type: tags +hide_menu: true +order: 302 +meta_title: Chat Tag for Conversational Transcripts +meta_description: Display and extend chat transcripts; optionally request assistant replies from an LLM. Supports message editing controls and min/max limits. +--- + +The `Chat` tag displays a conversational transcript and lets annotators extend it by adding new messages: + +![Screenshot](/images/tags/chat.png) + +Use with the following data types: JSON array of message objects + +!!! error Enterprise + This tag is only available for Label Studio Enterprise and Starter Cloud users. + + +### Use with an LLM + +Optionally, the tag can request automatic replies from an LLM. + +To use an LLM, you need to do two things: + +1. Add a model provider API key to your organization. + +2. Once you have added an API key for a model provider, set the `llm` attribute on the `` tag to the model you want to use. + + The `llm` attribute must use the format `/`. For example, `llm="openai/gpt-5"`. + +!!! note + Starter Cloud users have limited access to the LLM feature for chats. + + In Starter Cloud, you cannot add model providers. However, you will have access to out-of-the-box OpenAI models with $5.00 in credits for testing. If you want to add your own models and use interactive chats more extensively, you can [upgrade to Enterprise](https://humansignal.com/pricing/). + +### Editing messages + +You can allow annotators to edit the messages that they enter and, if applicable, responses from the LLM. + +Set the `editable` parameter to `"true"` or to a list of roles that should be editable. To edit a message, hover over it to view the edit icon. + +Annotators cannot edit messages from the imported task data. + +![Screenshot](/images/tags/chat-edit.png) + +{% insertmd includes/tags/chat.md %} + +## Examples + +### Example `` tag + +The following labeling configuration is the most basic implementation of the Chat tag. Adding a self-referencing `toName` paramenter allows you to use it without any other control tags. + +This labeling configuration would allow an annotator to submit messages from different roles that they select in a drop down message. + +```xml + + + +``` + +You can extend this configuration by allowing auto-replies from an LLM and adding control tags to evaluate the messages, as seen in the example below. + +### Example labeling config + +Evaluate assistant responses: + +```xml + + + + + + + + + + + + + + + + + + +
Overall quality of this conversation
+ + + + + +``` + +### Example input data + +!!! attention + The chat messages that you import are not selectable. This means that you cannot edit them or apply annotations (ratings, choices, etc) to them. + + You can only select and annotate messages that are added to the chat by an annotator or that are imported as predictions. + +The example JSON input data below is called in the `value="$chat"` parameter. + +- `role` — speaker identifier; supported roles: `user`, `assistant`, `system`, `tool`, `developer` +- `content` — message text + + +```json +{ + "data": { + "chat": [ + { + "role": "user", + "content": "Start with a kick-off message to validate the quality of it based on further conversation" + }, + { + "role": "assistant", + "content": "A response from the LLM" + } + ] + } +} +``` + +To work with a blank chat and have your annotator add all messages, simply import: + +```json +{ + "data": { + "chat": [] + } +} +``` \ No newline at end of file diff --git a/docs/source/tags/hypertext.md b/docs/source/tags/hypertext.md index 6fde2df78fd7..3f48342d2ca7 100644 --- a/docs/source/tags/hypertext.md +++ b/docs/source/tags/hypertext.md @@ -1,7 +1,7 @@ --- title: HyperText type: tags -order: 302 +order: 303 meta_title: Hypertext Tags for Hypertext Markup (HTML) meta_description: Label Studio Hypertext Tags customize Label Studio for hypertext markup (HTML) for machine learning and data science projects. --- diff --git a/docs/source/tags/image.md b/docs/source/tags/image.md index 56633ce8408b..bdb1f7a0aba9 100644 --- a/docs/source/tags/image.md +++ b/docs/source/tags/image.md @@ -1,7 +1,7 @@ --- title: Image type: tags -order: 303 +order: 304 meta_title: Image Tags for Images meta_description: Customize Label Studio with the Image tag to annotate images for computer vision machine learning and data science projects. --- diff --git a/docs/source/tags/list.md b/docs/source/tags/list.md index d5422258faea..cfe49ac4c96b 100644 --- a/docs/source/tags/list.md +++ b/docs/source/tags/list.md @@ -1,7 +1,7 @@ --- title: List type: tags -order: 304 +order: 305 meta_title: List Tag displays items of the same type, like articles, search results, etc. meta_description: Customize Label Studio by displaying similar items from task data for machine learning and data science projects. --- diff --git a/docs/source/tags/paragraphs.md b/docs/source/tags/paragraphs.md index 986068c10fe9..f1737d136d9c 100644 --- a/docs/source/tags/paragraphs.md +++ b/docs/source/tags/paragraphs.md @@ -1,7 +1,7 @@ --- title: Paragraphs type: tags -order: 305 +order: 306 meta_title: Paragraph Tags for Paragraphs meta_description: Customize Label Studio with the Paragraphs tag to annotate paragraphs for NLP and NER machine learning and data science projects. --- diff --git a/docs/source/tags/pdf.md b/docs/source/tags/pdf.md index aff2c0f3a7c7..36dc9cb60061 100644 --- a/docs/source/tags/pdf.md +++ b/docs/source/tags/pdf.md @@ -1,7 +1,7 @@ --- title: PDF type: tags -order: 302 +order: 307 meta_title: PDF tag for loading PDF documents meta_description: Label Studio PDF tag for loading PDF documents for machine learning and data science projects. --- diff --git a/docs/source/tags/table.md b/docs/source/tags/table.md index 84502cd89863..6d688a4841a4 100644 --- a/docs/source/tags/table.md +++ b/docs/source/tags/table.md @@ -1,7 +1,7 @@ --- title: Table type: tags -order: 306 +order: 308 meta_title: Table Tag to Display Keys & Values in Tables meta_description: Customize Label Studio by displaying key-value pairs in tasks for machine learning and data science projects. --- diff --git a/docs/source/tags/text.md b/docs/source/tags/text.md index dfceebb2ca93..21f744f6a45e 100644 --- a/docs/source/tags/text.md +++ b/docs/source/tags/text.md @@ -1,7 +1,7 @@ --- title: Text type: tags -order: 308 +order: 309 meta_title: Text Tags for Text Objects meta_description: Customize Label Studio with the Text tag to annotate text for NLP and NER machine learning and data science projects. --- diff --git a/docs/source/tags/timeseries.md b/docs/source/tags/timeseries.md index ee3b4560e841..9a7a948bd922 100644 --- a/docs/source/tags/timeseries.md +++ b/docs/source/tags/timeseries.md @@ -1,7 +1,7 @@ --- title: TimeSeries type: tags -order: 309 +order: 310 meta_title: Time Series Tags for Time Series Data meta_description: Customize Label Studio with the TimeSeries tag to annotate time series data for machine learning and data science projects. --- diff --git a/docs/source/tags/video.md b/docs/source/tags/video.md index 6304c93705fb..8673f9c88a9b 100644 --- a/docs/source/tags/video.md +++ b/docs/source/tags/video.md @@ -1,7 +1,7 @@ --- title: Video type: tags -order: 310 +order: 311 meta_title: Video Tag for Video Labeling meta_description: Customize Label Studio with the Video tag for basic video annotation tasks for machine learning and data science projects. --- diff --git a/docs/themes/v2/source/images/tags/chat-edit.png b/docs/themes/v2/source/images/tags/chat-edit.png new file mode 100644 index 000000000000..3dc81b9a0213 Binary files /dev/null and b/docs/themes/v2/source/images/tags/chat-edit.png differ diff --git a/docs/themes/v2/source/images/tags/chat.png b/docs/themes/v2/source/images/tags/chat.png new file mode 100644 index 000000000000..da2e9b33c73e Binary files /dev/null and b/docs/themes/v2/source/images/tags/chat.png differ diff --git a/web/libs/core/src/lib/utils/schema/tags.json b/web/libs/core/src/lib/utils/schema/tags.json index 2a260784952c..d2e5da804ccb 100644 --- a/web/libs/core/src/lib/utils/schema/tags.json +++ b/web/libs/core/src/lib/utils/schema/tags.json @@ -195,6 +195,54 @@ } } }, + "Chat": { + "name": "Chat", + "description": "The `Chat` tag displays a conversational transcript and lets annotators\nextend it with new messages during labeling. The initial transcript is\nprovided from task data via the `value` attribute.\n\nOptionally, the tag can request automatic replies from an LLM model. To do so,\nset the `llm` attribute to a model in the format `/`.\n\nMessages can be edited by clicking the edit button that appears on hover for\nuser-created messages (messages from annotation results). System messages from\ntask data cannot be edited.\n\nUse with the following data types: JSON array of message objects.\n\nMessage object format (task data):\n- `role` — speaker identifier; supported roles: `user`, `assistant`, `system`, `tool`, `developer`\n- `content` — message text\n\nExample task data:\n```json\n{\n \"dialog\": [\n {\"role\": \"system\", \"content\": \"Welcome to the assistant.\"},\n {\"role\": \"user\", \"content\": \"Hello!\"}\n ]\n}\n```", + "attrs": { + "name": { + "name": "name", + "description": "Name of the element", + "type": "string", + "required": true + }, + "value": { + "name": "value", + "description": "Data field containing an array of chat messages or empty array", + "type": "string", + "required": true + }, + "messageroles": { + "name": "messageroles", + "description": "Comma-separated list of roles that the user can create and send messages on behalf of. Default is \"user\" if the `llm` parameter is set; default is \"user,assistant\" if not.", + "type": "string", + "required": false + }, + "editable": { + "name": "editable", + "description": "Whether messages are editable. Use true/false, or a comma-separated list of roles that are editable", + "type": ["true", "false"], + "required": false + }, + "minmessages": { + "name": "minmessages", + "description": "Minimum total number of messages required to submit", + "type": ["string", "number"], + "required": false + }, + "maxmessages": { + "name": "maxmessages", + "description": "Maximum total number of messages allowed", + "type": ["string", "number"], + "required": false + }, + "llm": { + "name": "llm", + "description": "Model used to enable automatic assistant replies, format: `/`", + "type": "string", + "required": false + } + } + }, "HyperText": { "name": "HyperText", "description": "The `HyperText` tag displays hypertext markup for labeling. Use for labeling HTML-encoded text and webpages for NER and NLP projects.\n\nUse with the following data types: HTML.", @@ -3022,6 +3070,7 @@ "children": [ "Audio", "Channel", + "Chat", "HyperText", "Image", "List", diff --git a/web/libs/editor/src/stores/types.d.ts b/web/libs/editor/src/stores/types.d.ts index 56b476e265b0..43876bee3081 100644 --- a/web/libs/editor/src/stores/types.d.ts +++ b/web/libs/editor/src/stores/types.d.ts @@ -113,6 +113,7 @@ type MSTAnnotation = { draft?: RawResult[]; result?: RawResult[]; }; + areas: Map; regions: MSTRegion[]; results: MSTResult[]; type: "annotation" | "prediction"; diff --git a/web/libs/editor/src/tags/object/Chat.js b/web/libs/editor/src/tags/object/Chat.js new file mode 100644 index 000000000000..5e3f6e46a8e8 --- /dev/null +++ b/web/libs/editor/src/tags/object/Chat.js @@ -0,0 +1,59 @@ +/** + * The `Chat` tag displays a conversational transcript and lets annotators + * extend it with new messages during labeling. The initial transcript is + * provided from task data via the `value` attribute. + * + * Optionally, the tag can request automatic replies from an LLM model. To do so, + * set the `llm` attribute to a model in the format `/`. + * + * Messages can be edited by clicking the edit button that appears on hover for + * user-created messages (messages from annotation results). System messages from + * task data cannot be edited. + * + * Use with the following data types: JSON array of message objects. + * + * Message object format (task data): + * - `role` — speaker identifier; supported roles: `user`, `assistant`, `system`, `tool`, `developer` + * - `content` — message text + * + * Example task data: + * ```json + * { + * "dialog": [ + * {"role": "system", "content": "Welcome to the assistant."}, + * {"role": "user", "content": "Hello!"} + * ] + * } + * ``` + * + * @example + * + * + * + * + * @example + * + * + * + * + * + * @name Chat + * @regions ChatRegion + * @meta_title Chat Tag for Conversational Transcripts + * @meta_description Display and extend chat transcripts; optionally request assistant replies from an LLM. Supports message editing controls and min/max limits. + * + * @param {string} name Name of the element + * @param {string} value Data field containing an array of chat messages or empty array + * @param {string} [messageroles] Comma-separated list of roles that the user can create and send messages on behalf of. Default is "user" if the `llm` parameter is set; default is "user,assistant" if not. + * @param {boolean|string} [editable] Whether messages are editable. Use true/false, or a comma-separated list of roles that are editable + * @param {string|number} [minmessages] Minimum total number of messages required to submit + * @param {string|number} [maxmessages] Maximum total number of messages allowed + * @param {string} [llm] Model used to enable automatic assistant replies, format: `/` + */ + +export const ChatModel = {};