From cfdf7db83e2cc6945e3989cf86d2274ef4965eee Mon Sep 17 00:00:00 2001 From: Andrew Brookins Date: Fri, 13 Jun 2025 16:26:51 -0700 Subject: [PATCH 1/4] Add TASK_MEMORY.md for task tracking --- TASK_MEMORY.md | 26 ++++++++++++++++++++++++++ 1 file changed, 26 insertions(+) create mode 100644 TASK_MEMORY.md diff --git a/TASK_MEMORY.md b/TASK_MEMORY.md new file mode 100644 index 0000000..a8aa628 --- /dev/null +++ b/TASK_MEMORY.md @@ -0,0 +1,26 @@ +# Task Memory + +**Created:** 2025-06-13 16:26:51 +**Branch:** feature/write-a-new + +## Requirements + +Write a new CHANGELOG.md file. Include two versions: 0.0.1, whose description says basically, 'Initial release: basic short-term and long-term memory.' Then 0.9.0, which should act like a diff between that initial release and the latest set of features. It doesn't have to be perfect: mention stabilized APIs and concepts, with 'short-term' memory changing to 'working memory.' Detail the ability of clients to use working and long-term memory as tools (for the LLM) with the client SDK. Detail the availability of a REST and MCP API. Explain the new flow of memory into working memory that is then moved in the background to long-term memory. Detail the ability of working memory to contain arbitrary JSON data in addition to the memory structure. And anythin gelse interesting from the docs. + +## Development Notes + +*Update this section as you work on the task. Include:* +- *Progress updates* +- *Key decisions made* +- *Challenges encountered* +- *Solutions implemented* +- *Files modified* +- *Testing notes* + +### Work Log + +- [2025-06-13 16:26:51] Task setup completed, TASK_MEMORY.md created + +--- + +*This file serves as your working memory for this task. Keep it updated as you progress through the implementation.* From d0c42fa3fffd28147309538cc7f29acd5c75f734 Mon Sep 17 00:00:00 2001 From: Andrew Brookins Date: Fri, 13 Jun 2025 17:06:30 -0700 Subject: [PATCH 2/4] Add a changelog --- CHANGELOG.md | 63 ++++++++++++++++++++++++++++++++++++++++++++++++++ TASK_MEMORY.md | 26 --------------------- 2 files changed, 63 insertions(+), 26 deletions(-) create mode 100644 CHANGELOG.md delete mode 100644 TASK_MEMORY.md diff --git a/CHANGELOG.md b/CHANGELOG.md new file mode 100644 index 0000000..9a1374d --- /dev/null +++ b/CHANGELOG.md @@ -0,0 +1,63 @@ +# Changelog + +All notable changes to this project will be documented in this file. + +The format is based on [Keep a Changelog](https://keepachangelog.com/en/1.0.0/), +and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0.html). + +## [0.9.0] - 2025-06-13 + +*Changes from the initial release:** + +### Architecture Evolution +- **Working Memory (formerly Short-term Memory)**: + - Renamed from "short-term memory" to "working memory" to better reflect its purpose + - Enhanced with automatic promotion system that moves structured memories to long-term storage in background + - Added support for arbitrary JSON data storage alongside memory structures + - Improved automatic conversation summarization in working memory + + +- **Working Memory -> Long-term Memory Promotion**: + - Implemented seamless flow from working memory to long-term memory via background task processing + - Agent only has to think about working memory, long-term memory is managed automatically (but can be managed manually, too) + - Structured memories automatically promoted with vector embeddings and metadata indexing + - Deduplication and compaction systems for long-term memory management + - Background task worker system using Docket for reliable memory processing + +### Client SDK and Tooling + - Both working and long-term memory available as tools for LLM integration (LLM can choose to persist a long-term memory or search for long-term memories) + - Memory-enriched prompt generation via `/v1/memory/prompt` endpoint + - Unified search across both memory types with metadata filtering + - Support for namespace isolation and session management + +### Search and Retrieval + - Vector-based similarity search using OpenAI embeddings + - Rich filtering system by session, namespace, topics, entities, timestamps + - Hybrid search combining semantic similarity with metadata filtering + - RedisVL integration for high-performance vector operations with Redis + +### Enhanced Memory Classification: + - Semantic memories for facts and preferences + - Episodic memories for time-bound events with event dates (requires a timeframe) + - Message memories for conversation records + - Automatic topic modeling and entity recognition either using BERTopic or a configured LLM + - Rich metadata extraction and indexing + +### Authentication and Security + - OAuth2/JWT Bearer token authentication with JWKS validation + - Multi-provider support (Auth0, AWS Cognito, Okta, Azure AD) + - Role-based access control using JWT claims + - Development mode with configurable auth bypass + +### Operational Features +- **Comprehensive CLI Interface**: + - Commands for server management (`api`, `mcp`, `task-worker`) + - Database operations (`rebuild-index`) + - Background task scheduling and management + - Health monitoring and diagnostics + + +## [0.0.1] + +### Initial Release - 2025-04-07 +- Initial release with basic short-term and long-term memory functionality diff --git a/TASK_MEMORY.md b/TASK_MEMORY.md deleted file mode 100644 index a8aa628..0000000 --- a/TASK_MEMORY.md +++ /dev/null @@ -1,26 +0,0 @@ -# Task Memory - -**Created:** 2025-06-13 16:26:51 -**Branch:** feature/write-a-new - -## Requirements - -Write a new CHANGELOG.md file. Include two versions: 0.0.1, whose description says basically, 'Initial release: basic short-term and long-term memory.' Then 0.9.0, which should act like a diff between that initial release and the latest set of features. It doesn't have to be perfect: mention stabilized APIs and concepts, with 'short-term' memory changing to 'working memory.' Detail the ability of clients to use working and long-term memory as tools (for the LLM) with the client SDK. Detail the availability of a REST and MCP API. Explain the new flow of memory into working memory that is then moved in the background to long-term memory. Detail the ability of working memory to contain arbitrary JSON data in addition to the memory structure. And anythin gelse interesting from the docs. - -## Development Notes - -*Update this section as you work on the task. Include:* -- *Progress updates* -- *Key decisions made* -- *Challenges encountered* -- *Solutions implemented* -- *Files modified* -- *Testing notes* - -### Work Log - -- [2025-06-13 16:26:51] Task setup completed, TASK_MEMORY.md created - ---- - -*This file serves as your working memory for this task. Keep it updated as you progress through the implementation.* From 1c43ab6e9ad1ef32501b465a078150e5cd5c2948 Mon Sep 17 00:00:00 2001 From: Andrew Brookins Date: Fri, 13 Jun 2025 17:28:04 -0700 Subject: [PATCH 3/4] Update CHANGELOG.md --- CHANGELOG.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 9a1374d..1a5f1ed 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -16,10 +16,10 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 - Added support for arbitrary JSON data storage alongside memory structures - Improved automatic conversation summarization in working memory - -- **Working Memory -> Long-term Memory Promotion**: +- **Long-term Memory Promotion**: - Implemented seamless flow from working memory to long-term memory via background task processing - Agent only has to think about working memory, long-term memory is managed automatically (but can be managed manually, too) + - Use any LangChain `VectorStore` subclass for long-term storage, defaults to `RedisVectorStore` - Structured memories automatically promoted with vector embeddings and metadata indexing - Deduplication and compaction systems for long-term memory management - Background task worker system using Docket for reliable memory processing From a0fd969afca5c9f78f1811c8ec72836a997d76ed Mon Sep 17 00:00:00 2001 From: Andrew Brookins Date: Fri, 13 Jun 2025 17:31:48 -0700 Subject: [PATCH 4/4] Update CHANGELOG.md --- CHANGELOG.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/CHANGELOG.md b/CHANGELOG.md index 1a5f1ed..72f1bc9 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -7,7 +7,7 @@ and this project adheres to [Semantic Versioning](https://semver.org/spec/v2.0.0 ## [0.9.0] - 2025-06-13 -*Changes from the initial release:** +*Changes from the initial release:* ### Architecture Evolution - **Working Memory (formerly Short-term Memory)**: