Feature Request: Integrate Stateful Memory Extraction with Category & Tag Support for LLMs

[princetechs]

Summary

Add a built-in mechanism in Raix to support stateful long-term memory management for large language models (LLMs), enabling:

• Extraction of salient user information during conversations
• Categorization of memory entries by type (e.g., favorite_food, appointment)
• Tagging with relevant keywords for improved semantic search and filtering
• Returning structured JSON output containing both the LLM’s conversational response and memory update in a single call

Motivation

Statefulness is a crucial capability for conversational AI, allowing the bot to remember user preferences, important facts, and contextual knowledge across interactions. Currently, Raix focuses on LLM integration but lacks a structured memory management system that can:

• Extract and categorize user memory from conversation dynamically
• Support advanced retrieval via categories and semantic tags
• Enable efficient pruning, summarization, and update of memories

Implementing this feature will help developers build smarter, context-aware AI assistants with minimal extra effort.

Proposed Design

• Introduce a memory extraction prompt template with clear instructions for the LLM to return a JSON containing:

  • response — natural reply to the user

  • memory_update — memory object or null

    • category (string): short descriptor of the memory type
    • content (string): concise memory text
    • tags (array of strings): keywords for semantic search

• Provide helper methods/utilities for:

  • Injecting this prompt into chat conversations automatically
  • Parsing and validating the JSON response
  • Managing memory entries (store, update, prune) with categories and tags

• Optionally integrate with vector databases for semantic search and retrieval

Benefits

• Reduces boilerplate for memory handling in AI chatbots using Raix
• Enables more precise memory retrieval and contextual continuity
• Makes Raix more competitive and aligned with cutting-edge LLM statefulness research (e.g., Mem0)

Additional Context

• See related research: [Mem0: Building Production-Ready AI Agents with Scalable Long-Term Memory](https://arxiv.org/abs/2504.19413)
• Example prompt format for memory extraction and response generation included in the README