Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
12 changes: 12 additions & 0 deletions docs/source/includes/tags/chat.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,12 @@
### Parameters

| Param | Type | Description |
| --- | --- | --- |
| name | <code>string</code> | Name of the element |
| value | <code>string</code> | Data field containing an array of chat messages or empty array |
| [messageroles] | <code>string</code> | Comma-separated list of roles that the user can create and send messages on behalf of. Default is "user" if the `llm` parameter is set; default is "user,assistant" if not. |
| [editable] | <code>boolean</code> \| <code>string</code> | Whether messages are editable. Use true/false, or a comma-separated list of roles that are editable |
| [minmessages] | <code>string</code> \| <code>number</code> | Minimum total number of messages required to submit |
| [maxmessages] | <code>string</code> \| <code>number</code> | Maximum total number of messages allowed |
| [llm] | <code>string</code> | Model used to enable automatic assistant replies, format: `<provider>/<model>` |

137 changes: 137 additions & 0 deletions docs/source/tags/chat.md
Original file line number Diff line number Diff line change
@@ -0,0 +1,137 @@
---
title: Chat
type: tags
hide_menu: true
order: 302
meta_title: Chat Tag for Conversational Transcripts
meta_description: Display and extend chat transcripts; optionally request assistant replies from an LLM. Supports message editing controls and min/max limits.
---

The `Chat` tag displays a conversational transcript and lets annotators extend it by adding new messages:

![Screenshot](/images/tags/chat.png)

Use with the following data types: JSON array of message objects

!!! error Enterprise
This tag is only available for Label Studio Enterprise and Starter Cloud users.


### Use with an LLM

Optionally, the tag can request automatic replies from an LLM.

To use an LLM, you need to do two things:

1. Add a model provider API key to your organization.

2. Once you have added an API key for a model provider, set the `llm` attribute on the `<Chat>` tag to the model you want to use.

The `llm` attribute must use the format `<provider>/<model>`. For example, `llm="openai/gpt-5"`.

!!! note
Starter Cloud users have limited access to the LLM feature for chats.

In Starter Cloud, you cannot add model providers. However, you will have access to out-of-the-box OpenAI models with $5.00 in credits for testing. If you want to add your own models and use interactive chats more extensively, you can [upgrade to Enterprise](https://humansignal.com/pricing/).

### Editing messages

You can allow annotators to edit the messages that they enter and, if applicable, responses from the LLM.

Set the `editable` parameter to `"true"` or to a list of roles that should be editable. To edit a message, hover over it to view the edit icon.

Annotators cannot edit messages from the imported task data.

![Screenshot](/images/tags/chat-edit.png)

{% insertmd includes/tags/chat.md %}

## Examples

### Example `<Chat>` tag

The following labeling configuration is the most basic implementation of the Chat tag. Adding a self-referencing `toName` paramenter allows you to use it without any other control tags.

This labeling configuration would allow an annotator to submit messages from different roles that they select in a drop down message.

```xml
<View>
<Chat name="chat" value="$chat" toName="chat" />
</View>
```

You can extend this configuration by allowing auto-replies from an LLM and adding control tags to evaluate the messages, as seen in the example below.

### Example labeling config

Evaluate assistant responses:

```xml
<View>
<Style>
.htx-chat{flex-grow:1}
.htx-chat-sidepanel{flex:300px 0 0;display:flex;flex-direction:column;border-left:2px solid #ccc;padding-left:16px}
</Style>
<View style="display:flex;width:100%;gap:1em">
<Chat name="chat" value="$chat" llm="openai/gpt-4.1-nano" minMessages="4" maxMessages="10" editable="true" />
<View className="htx-chat-sidepanel">
<View style="position:sticky;top:14px">
<!-- Invitation/explanation on how to evaluate -->
<View visibleWhen="no-region-selected">
<Text name="_3" value="Click on a message to rate specific parts of the conversation"/>
</View>
<!-- Evaluate assistant messages -->
<View visibleWhen="region-selected" whenRole="assistant">
<Text name="_1" value="Rate the response" />
<Rating name="response_rating" toName="chat" perRegion="true" />
</View>
</View>
<!-- Evaluate the whole conversation -->
<View style="margin-top:auto;height:130px">
<Header size="4">Overall quality of this conversation</Header>
<Rating name="rating" toName="chat" />
</View>
</View>
</View>
</View>
```

### Example input data

!!! attention
The chat messages that you import are not selectable. This means that you cannot edit them or apply annotations (ratings, choices, etc) to them.

You can only select and annotate messages that are added to the chat by an annotator or that are imported as predictions.

The example JSON input data below is called in the `value="$chat"` parameter.

- `role` — speaker identifier; supported roles: `user`, `assistant`, `system`, `tool`, `developer`
- `content` — message text


```json
{
"data": {
"chat": [
{
"role": "user",
"content": "Start with a kick-off message to validate the quality of it based on further conversation"
},
{
"role": "assistant",
"content": "A response from the LLM"
}
]
}
}
```

To work with a blank chat and have your annotator add all messages, simply import:

```json
{
"data": {
"chat": []
}
}
```
2 changes: 1 addition & 1 deletion docs/source/tags/hypertext.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: HyperText
type: tags
order: 302
order: 303
meta_title: Hypertext Tags for Hypertext Markup (HTML)
meta_description: Label Studio Hypertext Tags customize Label Studio for hypertext markup (HTML) for machine learning and data science projects.
---
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tags/image.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Image
type: tags
order: 303
order: 304
meta_title: Image Tags for Images
meta_description: Customize Label Studio with the Image tag to annotate images for computer vision machine learning and data science projects.
---
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tags/list.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: List
type: tags
order: 304
order: 305
meta_title: List Tag displays items of the same type, like articles, search results, etc.
meta_description: Customize Label Studio by displaying similar items from task data for machine learning and data science projects.
---
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tags/paragraphs.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Paragraphs
type: tags
order: 305
order: 306
meta_title: Paragraph Tags for Paragraphs
meta_description: Customize Label Studio with the Paragraphs tag to annotate paragraphs for NLP and NER machine learning and data science projects.
---
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tags/pdf.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: PDF
type: tags
order: 302
order: 307
meta_title: PDF tag for loading PDF documents
meta_description: Label Studio PDF tag for loading PDF documents for machine learning and data science projects.
---
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tags/table.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Table
type: tags
order: 306
order: 308
meta_title: Table Tag to Display Keys & Values in Tables
meta_description: Customize Label Studio by displaying key-value pairs in tasks for machine learning and data science projects.
---
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tags/text.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Text
type: tags
order: 308
order: 309
meta_title: Text Tags for Text Objects
meta_description: Customize Label Studio with the Text tag to annotate text for NLP and NER machine learning and data science projects.
---
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tags/timeseries.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: TimeSeries
type: tags
order: 309
order: 310
meta_title: Time Series Tags for Time Series Data
meta_description: Customize Label Studio with the TimeSeries tag to annotate time series data for machine learning and data science projects.
---
Expand Down
2 changes: 1 addition & 1 deletion docs/source/tags/video.md
Original file line number Diff line number Diff line change
@@ -1,7 +1,7 @@
---
title: Video
type: tags
order: 310
order: 311
meta_title: Video Tag for Video Labeling
meta_description: Customize Label Studio with the Video tag for basic video annotation tasks for machine learning and data science projects.
---
Expand Down
Binary file added docs/themes/v2/source/images/tags/chat-edit.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/themes/v2/source/images/tags/chat.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
49 changes: 49 additions & 0 deletions web/libs/core/src/lib/utils/schema/tags.json
Original file line number Diff line number Diff line change
Expand Up @@ -195,6 +195,54 @@
}
}
},
"Chat": {
"name": "Chat",
"description": "The `Chat` tag displays a conversational transcript and lets annotators\nextend it with new messages during labeling. The initial transcript is\nprovided from task data via the `value` attribute.\n\nOptionally, the tag can request automatic replies from an LLM model. To do so,\nset the `llm` attribute to a model in the format `<provider>/<model>`.\n\nMessages can be edited by clicking the edit button that appears on hover for\nuser-created messages (messages from annotation results). System messages from\ntask data cannot be edited.\n\nUse with the following data types: JSON array of message objects.\n\nMessage object format (task data):\n- `role` — speaker identifier; supported roles: `user`, `assistant`, `system`, `tool`, `developer`\n- `content` — message text\n\nExample task data:\n```json\n{\n \"dialog\": [\n {\"role\": \"system\", \"content\": \"Welcome to the assistant.\"},\n {\"role\": \"user\", \"content\": \"Hello!\"}\n ]\n}\n```",
"attrs": {
"name": {
"name": "name",
"description": "Name of the element",
"type": "string",
"required": true
},
"value": {
"name": "value",
"description": "Data field containing an array of chat messages or empty array",
"type": "string",
"required": true
},
"messageroles": {
"name": "messageroles",
"description": "Comma-separated list of roles that the user can create and send messages on behalf of. Default is \"user\" if the `llm` parameter is set; default is \"user,assistant\" if not.",
"type": "string",
"required": false
},
"editable": {
"name": "editable",
"description": "Whether messages are editable. Use true/false, or a comma-separated list of roles that are editable",
"type": ["true", "false"],
"required": false
},
"minmessages": {
"name": "minmessages",
"description": "Minimum total number of messages required to submit",
"type": ["string", "number"],
"required": false
},
"maxmessages": {
"name": "maxmessages",
"description": "Maximum total number of messages allowed",
"type": ["string", "number"],
"required": false
},
"llm": {
"name": "llm",
"description": "Model used to enable automatic assistant replies, format: `<provider>/<model>`",
"type": "string",
"required": false
}
}
},
"HyperText": {
"name": "HyperText",
"description": "The `HyperText` tag displays hypertext markup for labeling. Use for labeling HTML-encoded text and webpages for NER and NLP projects.\n\nUse with the following data types: HTML.",
Expand Down Expand Up @@ -3022,6 +3070,7 @@
"children": [
"Audio",
"Channel",
"Chat",
"HyperText",
"Image",
"List",
Expand Down
1 change: 1 addition & 0 deletions web/libs/editor/src/stores/types.d.ts
Original file line number Diff line number Diff line change
Expand Up @@ -113,6 +113,7 @@ type MSTAnnotation = {
draft?: RawResult[];
result?: RawResult[];
};
areas: Map<string, MSTRegion>;
regions: MSTRegion[];
results: MSTResult[];
type: "annotation" | "prediction";
Expand Down
59 changes: 59 additions & 0 deletions web/libs/editor/src/tags/object/Chat.js
Original file line number Diff line number Diff line change
@@ -0,0 +1,59 @@
/**
* The `Chat` tag displays a conversational transcript and lets annotators
* extend it with new messages during labeling. The initial transcript is
* provided from task data via the `value` attribute.
*
* Optionally, the tag can request automatic replies from an LLM model. To do so,
* set the `llm` attribute to a model in the format `<provider>/<model>`.
*
* Messages can be edited by clicking the edit button that appears on hover for
* user-created messages (messages from annotation results). System messages from
* task data cannot be edited.
*
* Use with the following data types: JSON array of message objects.
*
* Message object format (task data):
* - `role` — speaker identifier; supported roles: `user`, `assistant`, `system`, `tool`, `developer`
* - `content` — message text
*
* Example task data:
* ```json
* {
* "dialog": [
* {"role": "system", "content": "Welcome to the assistant."},
* {"role": "user", "content": "Hello!"}
* ]
* }
* ```
*
* @example
* <View>
* <Chat name="chat" value="$dialog" />
* </View>
*
* @example
* <View>
* <!-- Allow composing both user and assistant messages; auto-reply using an LLM model -->
* <Chat
* name="conversation" value="$dialog"
* messageroles="user,assistant" llm="openai/gpt-5"
* minMessages="4" maxMessages="20"
* editable="user,assistant"
* />
* </View>
*
* @name Chat
* @regions ChatRegion
* @meta_title Chat Tag for Conversational Transcripts
* @meta_description Display and extend chat transcripts; optionally request assistant replies from an LLM. Supports message editing controls and min/max limits.
*
* @param {string} name Name of the element
* @param {string} value Data field containing an array of chat messages or empty array
* @param {string} [messageroles] Comma-separated list of roles that the user can create and send messages on behalf of. Default is "user" if the `llm` parameter is set; default is "user,assistant" if not.
* @param {boolean|string} [editable] Whether messages are editable. Use true/false, or a comma-separated list of roles that are editable
* @param {string|number} [minmessages] Minimum total number of messages required to submit
* @param {string|number} [maxmessages] Maximum total number of messages allowed
* @param {string} [llm] Model used to enable automatic assistant replies, format: `<provider>/<model>`
*/

export const ChatModel = {};
Loading