-
Notifications
You must be signed in to change notification settings - Fork 613
Add comprehensive agentic memory API documentation for OpenSearch 3.3 #11169
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged. Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer. When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review. |
- Add long-term memory APIs: search, update, delete - Add memory history APIs: search, delete - Add general memory APIs: get, update, delete, search by type - Update add-memory API to include data type memory and structured data - Update index with complete API structure - Support for conversation memory, data memory, and trace data - Include namespace-based organization and tagging system - Remove experimental warning as agentic memory is GA in 3.3 Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
@dhrubo-os Thank you for the PR! When you connect it to an issue, you have to leave "Closes", "Fixes", or "Resolves" so it's properly linked (in this case, |
Hey @dhrubo-os , let me know when this PR is ready for doc review. We have 3 days left before the release. |
- Remove redundant API files in favor of unified memory APIs - Delete legacy files: delete-long-term-memory.md, delete-memory-history.md, search-long-term-memory.md, search-memory-history.md, update-long-term-memory.md, search-memory.md, delete-memory.md, update-memory.md, get-memory.md - Enhance unified APIs with comprehensive examples for all memory types (sessions, working, long-term, history) - Add metadata field documentation to add-memory.md and get-memory-by-type.md - Update response examples and field documentation with actual API responses - Update index.md to reflect clean unified API structure - Fix memory type parameter names (sessions vs session) for consistency Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
- Add missing update-memory-container.md and search-memory-container.md files - Update create-memory-container.md with advanced features from Postman collection: * Multiple strategy support (SEMANTIC, USER_PREFERENCE, SUMMARY) * Strategy configuration with llm_result_path * Index configuration (index_prefix, use_system_index) * Remove experimental warning (GA in 3.3) - Remove experimental feature reference from index.md - All APIs now match actual implementation from Postman collection Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
- Add default llm_result_path value: $.output.message.content[0].text (Bedrock Converse format) - Document LLM connector requirement for system_prompt and user_prompt parameters - Reference ml-commons PR opensearch-project#4283 for standardized connector format - Clarify that default is optimized for Bedrock Converse API Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
Yes please start reviewing the PR. Thanks |
|
||
## Path parameters | ||
|
||
| Parameter | Data type | Description | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Currently we have a separate column Required/Optional
, can we follow the same format?
{: .warning} | ||
|
||
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create a memory in one of the following modes (controlled by the `infer` parameter): | ||
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create memories in two types: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create memories in two types: | |
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can provide memory payload in two types: |
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create a memory in one of the following modes (controlled by the `infer` parameter): | ||
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create memories in two types: | ||
|
||
- **Conversation memory** -- Stores conversational messages between users and assistants. Can be processed (when `infer` is `true`) to extract facts or stored as raw messages. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- **Conversation memory** -- Stores conversational messages between users and assistants. Can be processed (when `infer` is `true`) to extract facts or stored as raw messages. | |
- **conversational** -- Stores conversational messages between users and assistants. |
|
||
- **Conversation memory** -- Stores conversational messages between users and assistants. Can be processed (when `infer` is `true`) to extract facts or stored as raw messages. | ||
|
||
- **Data memory** -- Stores structured, non-conversational data such as agent state, checkpoints, or reference information. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- **Data memory** -- Stores structured, non-conversational data such as agent state, checkpoints, or reference information. | |
- **data** -- Stores extra messages, structured, non-conversational data such as agent state, checkpoints, or reference information. |
|
||
Memory processing modes (controlled by the `infer` parameter): | ||
|
||
- Fact memory -- A processed representation of the message. The large language model (LLM) associated with the memory container extracts and stores key factual information or knowledge from the original text. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- Fact memory -- A processed representation of the message. The large language model (LLM) associated with the memory container extracts and stores key factual information or knowledge from the original text. | |
- **infer=true** -- Use large language model (LLM) to extract key information or knowledge from the messages. |
|
||
- Fact memory -- A processed representation of the message. The large language model (LLM) associated with the memory container extracts and stores key factual information or knowledge from the original text. | ||
|
||
- Raw message memory -- The unprocessed message content. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- **infer=false** -- Only store raw messages and data in working memory. |
`session_id` | String | Optional | The session ID associated with the memory. | ||
`agent_id` | String | Optional | The agent ID associated with the memory. | ||
`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true`. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message. | ||
`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`. | |
`messages` | List | Conditional | A list of messages for conversational payload. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `payload_type` of `conversational`. |
`agent_id` | String | Optional | The agent ID associated with the memory. | ||
`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true`. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message. | ||
`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`. | ||
`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`. | |
`structured_data` | Map<String, Object> | Conditional | Structured data content for data memory. Required for `memory_type` of `data`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a new row for binary_data
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
And metadata
also?
`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true`. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message. | ||
`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`. | ||
`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`. | ||
`memory_type` | String | Required | The type of memory: `conversation` or `data`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
`memory_type` | String | Required | The type of memory: `conversation` or `data`. | |
`payload_type` | String | Required | The type of payload: `conversational` or `data`. |
`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`. | ||
`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`. | ||
`memory_type` | String | Required | The type of memory: `conversation` or `data`. | ||
`namespace` | Object | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
`namespace` | Object | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`). | |
`namespace` | List<String> | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Explain more details
If session_id
not exists in namespace
will create a new session and use the new session's id
`memory_type` | String | Required | The type of memory: `conversation` or `data`. | ||
`namespace` | Object | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`). | ||
`metadata` | Object | Optional | Additional metadata for the memory (e.g., `status`, `branch`, custom fields). | ||
`session_id` | String | Optional | The session ID associated with the memory. Deprecated in favor of using `namespace.session_id`. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Remove these two lines: 45, 46
`metadata` | Object | Optional | Additional metadata for the memory (e.g., `status`, `branch`, custom fields). | ||
`session_id` | String | Optional | The session ID associated with the memory. Deprecated in favor of using `namespace.session_id`. | ||
`agent_id` | String | Optional | The agent ID associated with the memory. Deprecated in favor of using `namespace.agent_id`. | ||
`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true` for conversation memory, `false` for data memory. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true` for conversation memory, `false` for data memory. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message. | |
`infer` | Boolean | Optional | Controls whether use LLM to extract key information from messages. Default is `false`. When `true`, the LLM extracts key information from the original text and stores it as the memory. |
`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`. | ||
`memory_type` | String | Required | The type of memory: `conversation` or `data`. | ||
`namespace` | Object | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`). | ||
`metadata` | Object | Optional | Additional metadata for the memory (e.g., `status`, `branch`, custom fields). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add a new row for tags
## Example request | ||
## Example requests | ||
|
||
### Conversation memory |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
### Conversation memory | |
### Conversational payload |
"topic": "personal info" | ||
}, | ||
"infer": true, | ||
"memory_type": "conversation" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"memory_type": "conversation" | |
"payload_type": "conversational" |
} | ||
``` | ||
|
||
### Data memory |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
### Data memory | |
### Data payload | |
Store agent state in working memory: |
"topic": "agent_state" | ||
}, | ||
"infer": false, | ||
"memory_type": "data" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"memory_type": "data" | |
"payload_type": "data" |
} | ||
``` | ||
|
||
### Trace data memory |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
### Trace data memory | |
Store agent trace data in working memory: |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
I didn't clearly follow this suggestion. you want me to remove ### Trace data memory
section?
"data_type": "trace" | ||
}, | ||
"infer": false, | ||
"memory_type": "conversation" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
"memory_type": "conversation" | |
"payload_type": "conversation" |
`embedding_model_id` | String | Optional | The embedding model ID. | ||
`embedding_dimension` | Integer | Optional | The dimension of the embedding model. Required if `embedding_model_type` is `TEXT_EMBEDDING`. | ||
`llm_id` | String | Optional | The LLM model ID for processing and inference. | ||
`index_prefix` | String | Optional | Custom prefix for the memory indices. If not specified, a default prefix is used. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Explain more details for default prefix
- If
use_system_index
istrue
, usedefault
as index prefix - If
use_system_index
isfalse
, use 8 bit random UUID as index prefix
Field | Data type | Required/Optional | Description | ||
:--- | :--- | :--- | :--- | ||
`type` | String | Required | The strategy type: `SEMANTIC`, `USER_PREFERENCE`, or `SUMMARY`. | ||
`namespace` | Array | Required | Array of namespace dimensions for organizing memories (e.g., `["user_id"]`, `["agent_id"]`). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Suggest add more fields to show we can support multiple fields in namespace
`namespace` | Array | Required | Array of namespace dimensions for organizing memories (e.g., `["user_id"]`, `["agent_id"]`). | |
`namespace` | Array | Required | Array of namespace dimensions for organizing memories (e.g., `["user_id"]`, `["agent_id", "session_id"]`). |
:--- | :--- | :--- | :--- | ||
`type` | String | Required | The strategy type: `SEMANTIC`, `USER_PREFERENCE`, or `SUMMARY`. | ||
`namespace` | Array | Required | Array of namespace dimensions for organizing memories (e.g., `["user_id"]`, `["agent_id"]`). | ||
`configuration` | Object | Optional | Strategy-specific configuration. See [the strategy `configuration` object](#the-strategy-configuration-object). |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
`configuration` | Object | Optional | Strategy-specific configuration. See [the strategy `configuration` object](#the-strategy-configuration-object). | |
`configuration` | Map<String, Object> | Optional | Strategy-specific configuration. See [the strategy `configuration` object](#the-strategy-configuration-object). |
|
||
Field | Data type | Required/Optional | Description | ||
:--- | :--- | :--- | :--- | ||
`llm_result_path` | String | Optional | JSONPath to extract LLM results. Default is `"$.output.message.content[0].text"` for Bedrock Converse API format. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
We also support
system_prompt
, so user can override the default strategy prompt.llm_id
, user can set different LLM for each strategy
"type": "SEMANTIC", | ||
"namespace": ["agent_id"], | ||
"configuration": { | ||
"llm_result_path": "$.output.message.content[0].text" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
All of these three strategies shows llm_result_path
example.
Let's also show the system_prompt
and llm_id
example?
--- | ||
|
||
# Delete memory by type and ID | ||
**Introduced 3.2** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**Introduced 3.2** | |
**Introduced 3.3** |
--- | ||
|
||
# Get memory by type and ID | ||
**Introduced 3.2** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**Introduced 3.2** | |
**Introduced 3.3** |
}, | ||
"metadata": { | ||
"status": "checkpoint", | ||
"branch": { |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Let's show the real get response. This metadata
value will be stored as string, not a json object
"branch": "{\"root_event_id\":\"228nadfs879mtgk\",\"branch_name\":\"high\"}",
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
If user need to store the data as Json object, suggest store it in structured_data
.
"structured_data": {
"branch": {
"root_event_id": "228nadfs879mtgk",
"branch_name": "high"
},
"status": "checkpoint"
},
"memory": "A comprehensive security investigation was performed across multiple data sources including 55 OpenSearch indices, 50 CloudTrail events, 22 VPC Flow logs, 38 WAF events, 74 CloudWatch log groups, active CloudWatch alarms, and OpenSearch cluster security configuration." | ||
}, | ||
"namespace": { | ||
"agent_id": "soap-user" |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Fix this part ? This means user id , not agent id?
{ | ||
"persistent": { | ||
"plugins.ml_commons.agentic_memory_enabled": true | ||
"plugins.ml_commons.agentic_memory_enabled": false |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add to this page https://docs.opensearch.org/latest/ml-commons-plugin/cluster-settings/ ?
--- | ||
|
||
# Update memory by type and ID | ||
**Introduced 3.2** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**Introduced 3.2** | |
**Introduced 3.3** |
--- | ||
|
||
# Update memory container | ||
**Introduced 3.2** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**Introduced 3.2** | |
**Introduced 3.3** |
--- | ||
|
||
# Search memory containers | ||
**Introduced 3.2** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**Introduced 3.2** | |
**Introduced 3.3** |
|
||
The request body supports standard OpenSearch query DSL. Common fields include: | ||
|
||
| Field | Data type | Description | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we support standard OpenSearch query, seems no need to show this.
OpenSearch DLS supports more fields like _source
, aggs
etc
--- | ||
|
||
# Search memories by type | ||
**Introduced 3.2** |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
**Introduced 3.2** | |
**Introduced 3.3** |
|
||
The request body supports standard OpenSearch query DSL. Common fields include: | ||
|
||
| Field | Data type | Description | |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
As we support standard OpenSearch query, seems no need to show this.
- Update add-memory.md terminology: memory_type -> payload_type, conversational/data types - Add Required/Optional column format to parameter tables - Update version labels from 3.2 to 3.3 for new unified APIs - Add binary_data and tags fields with correct data types - Fix metadata field representation as string in response examples - Add detailed explanations for namespace session creation behavior - Update create-memory-container.md with system_prompt and llm_id strategy fields - Add detailed index_prefix default behavior explanation - Show multiple namespace field examples in strategy configuration Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
- Add plugins.ml_commons.agentic_memory_enabled setting to cluster-settings.md - Enables/disables agentic memory functionality for AI agents - Includes session, working, long-term memory, and memory history features Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
|
||
- Raw message memory -- The unprocessed message content. | ||
- **data** -- Stores extra messages, structured, non-conversational data such as agent state, checkpoints, or reference information. | ||
|
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Add some description before showing infer=true/false ?
User can use infer
to choose to extract key information from raw messages or not.
- Raw message memory -- The unprocessed message content. | ||
- **data** -- Stores extra messages, structured, non-conversational data such as agent state, checkpoints, or reference information. | ||
|
||
- **infer=true** -- Use large language model (LLM) to extract key information or knowledge from the messages. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
- **infer=true** -- Use large language model (LLM) to extract key information or knowledge from the messages. | |
- **infer=true** -- Use large language model (LLM) to extract key information or knowledge from the messages. The extracted information will be stored as long-term memory. |
Description
Describe what this change achieves.
Issues Resolved
Closes #11162
Version
3.3
Frontend features
If you're submitting documentation for an OpenSearch Dashboards feature, add a video that shows how a user will interact with the UI step by step. A voiceover is optional.
Checklist
For more information on following Developer Certificate of Origin and signing off your commits, please check here.