Skip to content

Conversation

dhrubo-os
Copy link
Contributor

@dhrubo-os dhrubo-os commented Oct 1, 2025

  • Add long-term memory APIs: search, update, delete
  • Add memory history APIs: search, delete
  • Add general memory APIs: get, update, delete, search by type
  • Update add-memory API to include data type memory and structured data
  • Update index with complete API structure
  • Support for conversation memory, data memory, and trace data
  • Include namespace-based organization and tagging system
  • Remove experimental warning as agentic memory is GA in 3.3

Description

Describe what this change achieves.

Issues Resolved

Closes #11162

Version

3.3

Frontend features

If you're submitting documentation for an OpenSearch Dashboards feature, add a video that shows how a user will interact with the UI step by step. A voiceover is optional.

Checklist

  • By submitting this pull request, I confirm that my contribution is made under the terms of the Apache 2.0 license and subject to the Developers Certificate of Origin.
    For more information on following Developer Certificate of Origin and signing off your commits, please check here.

Copy link

github-actions bot commented Oct 1, 2025

Thank you for submitting your PR. The PR states are In progress (or Draft) -> Tech review -> Doc review -> Editorial review -> Merged.

Before you submit your PR for doc review, make sure the content is technically accurate. If you need help finding a tech reviewer, tag a maintainer.

When you're ready for doc review, tag the assignee of this PR. The doc reviewer may push edits to the PR directly or leave comments and editorial suggestions for you to address (let us know in a comment if you have a preference). The doc reviewer will arrange for an editorial review.

- Add long-term memory APIs: search, update, delete
- Add memory history APIs: search, delete
- Add general memory APIs: get, update, delete, search by type
- Update add-memory API to include data type memory and structured data
- Update index with complete API structure
- Support for conversation memory, data memory, and trace data
- Include namespace-based organization and tagging system
- Remove experimental warning as agentic memory is GA in 3.3

Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
@kolchfa-aws
Copy link
Collaborator

@dhrubo-os Thank you for the PR! When you connect it to an issue, you have to leave "Closes", "Fixes", or "Resolves" so it's properly linked (in this case, Closes #11162). I updated the PR description.

@kolchfa-aws kolchfa-aws added Tech review PR: Tech review in progress release-notes PR: Include this PR in the automated release notes v3.3.0 labels Oct 2, 2025
@kolchfa-aws
Copy link
Collaborator

Hey @dhrubo-os , let me know when this PR is ready for doc review. We have 3 days left before the release.

- Remove redundant API files in favor of unified memory APIs
- Delete legacy files: delete-long-term-memory.md, delete-memory-history.md, search-long-term-memory.md, search-memory-history.md, update-long-term-memory.md, search-memory.md, delete-memory.md, update-memory.md, get-memory.md
- Enhance unified APIs with comprehensive examples for all memory types (sessions, working, long-term, history)
- Add metadata field documentation to add-memory.md and get-memory-by-type.md
- Update response examples and field documentation with actual API responses
- Update index.md to reflect clean unified API structure
- Fix memory type parameter names (sessions vs session) for consistency

Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
- Add missing update-memory-container.md and search-memory-container.md files
- Update create-memory-container.md with advanced features from Postman collection:
  * Multiple strategy support (SEMANTIC, USER_PREFERENCE, SUMMARY)
  * Strategy configuration with llm_result_path
  * Index configuration (index_prefix, use_system_index)
  * Remove experimental warning (GA in 3.3)
- Remove experimental feature reference from index.md
- All APIs now match actual implementation from Postman collection

Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
- Add default llm_result_path value: $.output.message.content[0].text (Bedrock Converse format)
- Document LLM connector requirement for system_prompt and user_prompt parameters
- Reference ml-commons PR opensearch-project#4283 for standardized connector format
- Clarify that default is optimized for Bedrock Converse API

Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
@dhrubo-os
Copy link
Contributor Author

Hey @dhrubo-os , let me know when this PR is ready for doc review. We have 3 days left before the release.

Yes please start reviewing the PR. Thanks


## Path parameters

| Parameter | Data type | Description |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

https://docs.opensearch.org/latest/ml-commons-plugin/api/agentic-memory-apis/create-memory-container/

Currently we have a separate column Required/Optional, can we follow the same format?

{: .warning}

Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create a memory in one of the following modes (controlled by the `infer` parameter):
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create memories in two types:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create memories in two types:
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can provide memory payload in two types:

Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create a memory in one of the following modes (controlled by the `infer` parameter):
Use this API to add an agentic memory to a [memory container]({{site.url}}{{site.baseurl}}/ml-commons-plugin/api/agentic-memory-apis/create-memory-container). You can create memories in two types:

- **Conversation memory** -- Stores conversational messages between users and assistants. Can be processed (when `infer` is `true`) to extract facts or stored as raw messages.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Conversation memory** -- Stores conversational messages between users and assistants. Can be processed (when `infer` is `true`) to extract facts or stored as raw messages.
- **conversational** -- Stores conversational messages between users and assistants.


- **Conversation memory** -- Stores conversational messages between users and assistants. Can be processed (when `infer` is `true`) to extract facts or stored as raw messages.

- **Data memory** -- Stores structured, non-conversational data such as agent state, checkpoints, or reference information.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **Data memory** -- Stores structured, non-conversational data such as agent state, checkpoints, or reference information.
- **data** -- Stores extra messages, structured, non-conversational data such as agent state, checkpoints, or reference information.


Memory processing modes (controlled by the `infer` parameter):

- Fact memory -- A processed representation of the message. The large language model (LLM) associated with the memory container extracts and stores key factual information or knowledge from the original text.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- Fact memory -- A processed representation of the message. The large language model (LLM) associated with the memory container extracts and stores key factual information or knowledge from the original text.
- **infer=true** -- Use large language model (LLM) to extract key information or knowledge from the messages.


- Fact memory -- A processed representation of the message. The large language model (LLM) associated with the memory container extracts and stores key factual information or knowledge from the original text.

- Raw message memory -- The unprocessed message content.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **infer=false** -- Only store raw messages and data in working memory.

`session_id` | String | Optional | The session ID associated with the memory.
`agent_id` | String | Optional | The agent ID associated with the memory.
`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true`. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message.
`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`.
Copy link
Contributor

@ylwu-amzn ylwu-amzn Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`.
`messages` | List | Conditional | A list of messages for conversational payload. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `payload_type` of `conversational`.

`agent_id` | String | Optional | The agent ID associated with the memory.
`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true`. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message.
`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`.
`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`.
`structured_data` | Map<String, Object> | Conditional | Structured data content for data memory. Required for `memory_type` of `data`.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a new row for binary_data

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

And metadata also?

`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true`. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message.
`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`.
`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`.
`memory_type` | String | Required | The type of memory: `conversation` or `data`.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`memory_type` | String | Required | The type of memory: `conversation` or `data`.
`payload_type` | String | Required | The type of payload: `conversational` or `data`.

`messages` | List | Conditional | A list of messages for conversation memory. Each message requires `content` and may include a `role` (commonly, `user` or `assistant`) when `infer` is set to `true`. Required for `memory_type` of `conversation`.
`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`.
`memory_type` | String | Required | The type of memory: `conversation` or `data`.
`namespace` | Object | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`namespace` | Object | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`).
`namespace` | List<String> | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explain more details

If session_id not exists in namespace will create a new session and use the new session's id

`memory_type` | String | Required | The type of memory: `conversation` or `data`.
`namespace` | Object | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`).
`metadata` | Object | Optional | Additional metadata for the memory (e.g., `status`, `branch`, custom fields).
`session_id` | String | Optional | The session ID associated with the memory. Deprecated in favor of using `namespace.session_id`.
Copy link
Contributor

@ylwu-amzn ylwu-amzn Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Remove these two lines: 45, 46

`metadata` | Object | Optional | Additional metadata for the memory (e.g., `status`, `branch`, custom fields).
`session_id` | String | Optional | The session ID associated with the memory. Deprecated in favor of using `namespace.session_id`.
`agent_id` | String | Optional | The agent ID associated with the memory. Deprecated in favor of using `namespace.agent_id`.
`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true` for conversation memory, `false` for data memory. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`infer` | Boolean | Optional | Controls whether the LLM infers context from messages. Default is `true` for conversation memory, `false` for data memory. When `true`, the LLM extracts factual information from the original text and stores it as the memory. When `false`, the memory contains the unprocessed message and you must explicitly specify the `role` in each message.
`infer` | Boolean | Optional | Controls whether use LLM to extract key information from messages. Default is `false`. When `true`, the LLM extracts key information from the original text and stores it as the memory.

`structured_data` | Object | Conditional | Structured data content for data memory. Required for `memory_type` of `data`.
`memory_type` | String | Required | The type of memory: `conversation` or `data`.
`namespace` | Object | Optional | Namespace context for organizing memories (e.g., `user_id`, `session_id`, `agent_id`).
`metadata` | Object | Optional | Additional metadata for the memory (e.g., `status`, `branch`, custom fields).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add a new row for tags

## Example request
## Example requests

### Conversation memory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Conversation memory
### Conversational payload

"topic": "personal info"
},
"infer": true,
"memory_type": "conversation"
Copy link
Contributor

@ylwu-amzn ylwu-amzn Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"memory_type": "conversation"
"payload_type": "conversational"

}
```

### Data memory
Copy link
Contributor

@ylwu-amzn ylwu-amzn Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Data memory
### Data payload
Store agent state in working memory:

"topic": "agent_state"
},
"infer": false,
"memory_type": "data"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"memory_type": "data"
"payload_type": "data"

}
```

### Trace data memory
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
### Trace data memory
Store agent trace data in working memory:

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I didn't clearly follow this suggestion. you want me to remove ### Trace data memory section?

"data_type": "trace"
},
"infer": false,
"memory_type": "conversation"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
"memory_type": "conversation"
"payload_type": "conversation"

`embedding_model_id` | String | Optional | The embedding model ID.
`embedding_dimension` | Integer | Optional | The dimension of the embedding model. Required if `embedding_model_type` is `TEXT_EMBEDDING`.
`llm_id` | String | Optional | The LLM model ID for processing and inference.
`index_prefix` | String | Optional | Custom prefix for the memory indices. If not specified, a default prefix is used.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Explain more details for default prefix

  • If use_system_index is true, use default as index prefix
  • If use_system_index is false, use 8 bit random UUID as index prefix

Field | Data type | Required/Optional | Description
:--- | :--- | :--- | :---
`type` | String | Required | The strategy type: `SEMANTIC`, `USER_PREFERENCE`, or `SUMMARY`.
`namespace` | Array | Required | Array of namespace dimensions for organizing memories (e.g., `["user_id"]`, `["agent_id"]`).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggest add more fields to show we can support multiple fields in namespace

Suggested change
`namespace` | Array | Required | Array of namespace dimensions for organizing memories (e.g., `["user_id"]`, `["agent_id"]`).
`namespace` | Array | Required | Array of namespace dimensions for organizing memories (e.g., `["user_id"]`, `["agent_id", "session_id"]`).

:--- | :--- | :--- | :---
`type` | String | Required | The strategy type: `SEMANTIC`, `USER_PREFERENCE`, or `SUMMARY`.
`namespace` | Array | Required | Array of namespace dimensions for organizing memories (e.g., `["user_id"]`, `["agent_id"]`).
`configuration` | Object | Optional | Strategy-specific configuration. See [the strategy `configuration` object](#the-strategy-configuration-object).
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
`configuration` | Object | Optional | Strategy-specific configuration. See [the strategy `configuration` object](#the-strategy-configuration-object).
`configuration` | Map<String, Object> | Optional | Strategy-specific configuration. See [the strategy `configuration` object](#the-strategy-configuration-object).


Field | Data type | Required/Optional | Description
:--- | :--- | :--- | :---
`llm_result_path` | String | Optional | JSONPath to extract LLM results. Default is `"$.output.message.content[0].text"` for Bedrock Converse API format.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

We also support

  1. system_prompt , so user can override the default strategy prompt.
  2. llm_id, user can set different LLM for each strategy

"type": "SEMANTIC",
"namespace": ["agent_id"],
"configuration": {
"llm_result_path": "$.output.message.content[0].text"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

All of these three strategies shows llm_result_path example.
Let's also show the system_prompt and llm_id example?

---

# Delete memory by type and ID
**Introduced 3.2**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Introduced 3.2**
**Introduced 3.3**

---

# Get memory by type and ID
**Introduced 3.2**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Introduced 3.2**
**Introduced 3.3**

},
"metadata": {
"status": "checkpoint",
"branch": {
Copy link
Contributor

@ylwu-amzn ylwu-amzn Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Let's show the real get response. This metadata value will be stored as string, not a json object

"branch": "{\"root_event_id\":\"228nadfs879mtgk\",\"branch_name\":\"high\"}",

Copy link
Contributor

@ylwu-amzn ylwu-amzn Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If user need to store the data as Json object, suggest store it in structured_data.

  "structured_data": {
    "branch": {
      "root_event_id": "228nadfs879mtgk",
      "branch_name": "high"
    },
    "status": "checkpoint"
  },

"memory": "A comprehensive security investigation was performed across multiple data sources including 55 OpenSearch indices, 50 CloudTrail events, 22 VPC Flow logs, 38 WAF events, 74 CloudWatch log groups, active CloudWatch alarms, and OpenSearch cluster security configuration."
},
"namespace": {
"agent_id": "soap-user"
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Fix this part ? This means user id , not agent id?

{
"persistent": {
"plugins.ml_commons.agentic_memory_enabled": true
"plugins.ml_commons.agentic_memory_enabled": false
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

---

# Update memory by type and ID
**Introduced 3.2**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Introduced 3.2**
**Introduced 3.3**

---

# Update memory container
**Introduced 3.2**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Introduced 3.2**
**Introduced 3.3**

---

# Search memory containers
**Introduced 3.2**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Introduced 3.2**
**Introduced 3.3**


The request body supports standard OpenSearch query DSL. Common fields include:

| Field | Data type | Description |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we support standard OpenSearch query, seems no need to show this.

OpenSearch DLS supports more fields like _source , aggs etc

---

# Search memories by type
**Introduced 3.2**
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
**Introduced 3.2**
**Introduced 3.3**


The request body supports standard OpenSearch query DSL. Common fields include:

| Field | Data type | Description |
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

As we support standard OpenSearch query, seems no need to show this.

- Update add-memory.md terminology: memory_type -> payload_type, conversational/data types
- Add Required/Optional column format to parameter tables
- Update version labels from 3.2 to 3.3 for new unified APIs
- Add binary_data and tags fields with correct data types
- Fix metadata field representation as string in response examples
- Add detailed explanations for namespace session creation behavior
- Update create-memory-container.md with system_prompt and llm_id strategy fields
- Add detailed index_prefix default behavior explanation
- Show multiple namespace field examples in strategy configuration

Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
- Add plugins.ml_commons.agentic_memory_enabled setting to cluster-settings.md
- Enables/disables agentic memory functionality for AI agents
- Includes session, working, long-term memory, and memory history features

Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>
Signed-off-by: Dhrubo Saha <dhrubo@amazon.com>

- Raw message memory -- The unprocessed message content.
- **data** -- Stores extra messages, structured, non-conversational data such as agent state, checkpoints, or reference information.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Add some description before showing infer=true/false ?

User can use infer to choose to extract key information from raw messages or not.

- Raw message memory -- The unprocessed message content.
- **data** -- Stores extra messages, structured, non-conversational data such as agent state, checkpoints, or reference information.

- **infer=true** -- Use large language model (LLM) to extract key information or knowledge from the messages.
Copy link
Contributor

@ylwu-amzn ylwu-amzn Oct 10, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
- **infer=true** -- Use large language model (LLM) to extract key information or knowledge from the messages.
- **infer=true** -- Use large language model (LLM) to extract key information or knowledge from the messages. The extracted information will be stored as long-term memory.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
release-notes PR: Include this PR in the automated release notes Tech review PR: Tech review in progress v3.3.0
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[DOC] Agentic Memory enhancements for GA release
4 participants