Skip to content

Enahancement memory#1

Merged
gitnasr merged 13 commits intomainfrom
enahancement-memory
Jun 14, 2025
Merged

Enahancement memory#1
gitnasr merged 13 commits intomainfrom
enahancement-memory

Conversation

@gitnasr
Copy link
Copy Markdown
Collaborator

@gitnasr gitnasr commented Jun 14, 2025

Summary by CodeRabbit

  • New Features

    • Introduced structured user management with personalized preferences, context, and memory handling.
    • Added embedding-based conversation repetition detection and retrieval of relevant memories for enhanced responses.
    • Implemented token-limited, role-based conversation context with dynamic summarization of older interactions.
  • Improvements

    • Enhanced conversation storage with embeddings, topics, and token counts for improved context accuracy.
    • Refined error handling and detailed logging for robustness.
    • Replaced previous memory management with a new class supporting advanced context and memory operations.
  • Documentation

    • Added a detailed system prompt defining the assistant’s persona, multilingual support, and operational guidelines.
  • Chores

    • Updated dependencies to include sentence-transformers and scikit-learn for embedding and similarity computations.

@gitnasr gitnasr requested a review from Copilot June 14, 2025 14:53
@gitnasr gitnasr self-assigned this Jun 14, 2025
@coderabbitai
Copy link
Copy Markdown

coderabbitai bot commented Jun 14, 2025

Caution

Review failed

The pull request is closed.

Walkthrough

The codebase was refactored to introduce class-based, structured management of users, conversations, context, and memory. New models and database schema changes support richer metadata, embedding-based similarity checks, and user preferences. Message processing now leverages these enhancements for context construction, repetition detection, and memory summarization, guided by a comprehensive system prompt.

Changes

File(s) Change Summary
database.py Refactored to encapsulate all DB operations in a ConversationDB class; added user, context, and memory management with embedding similarity checks; improved error handling; introduced singleton instance.
database_config.py Added/extended SQLAlchemy models: User, UserMemory, expanded Conversation and UserContext with embeddings, preferences, topics, and relationships; added serialization and helper methods.
main.py Replaced WhatsAppWizard with MemoryManager class; enhanced message processing with context/memory integration, repetition detection, and summarization; updated webhook endpoint and system prompt loading; added thread pool executor lifecycle events.
requirements.txt Added dependencies: sentence-transformers, scikit-learn; cleaned formatting.
system_prompt.txt Added comprehensive system prompt file defining assistant persona, capabilities, and operational guidelines.

Sequence Diagram(s)

sequenceDiagram
    participant User
    participant API (/webhook)
    participant MemoryManager
    participant ConversationDB
    participant OpenRouterAPI

    User->>API: Sends message (with user_id, language)
    API->>MemoryManager: process_message(user_id, message, language)
    MemoryManager->>ConversationDB: get_or_create_user(user_id)
    MemoryManager->>ConversationDB: get_user_preferences(user_id)
    MemoryManager->>ConversationDB: get_recent_conversations(user_id)
    MemoryManager->>ConversationDB: get_relevant_memories(user_id, embedding)
    MemoryManager->>ConversationDB: check_repetition(user_id, message_embedding)
    alt Repetition detected
        MemoryManager-->>API: Return repeated response
    else Not repeated
        MemoryManager->>OpenRouterAPI: Generate AI response (with context, memories)
        OpenRouterAPI-->>MemoryManager: AI response
        MemoryManager->>ConversationDB: add_conversation(user_id, message, response, ...)
        MemoryManager->>ConversationDB: update_user_context(user_id, ...)
        MemoryManager-->>API: Return AI response
    end
    API-->>User: Deliver response
Loading

Poem

In the warren of code, a new order’s begun,
With memories, context, and embeddings spun.
Users and chats now neatly arrayed,
Repetition detected, summaries made.
A prompt for the wizard, so clever and spry—
Hopping through data, with a glint in its eye!
🐇✨


📜 Recent review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between d0b806a and 14f0787.

📒 Files selected for processing (3)
  • .github/workflows/docker-build.yml (3 hunks)
  • database_config.py (2 hunks)
  • main.py (2 hunks)
✨ Finishing Touches
  • 📝 Generate Docstrings

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share
🪧 Tips

Chat

There are 3 ways to chat with CodeRabbit:

  • Review comments: Directly reply to a review comment made by CodeRabbit. Example:
    • I pushed a fix in commit <commit_id>, please review it.
    • Explain this complex logic.
    • Open a follow-up GitHub issue for this discussion.
  • Files and specific lines of code (under the "Files changed" tab): Tag @coderabbitai in a new review comment at the desired location with your query. Examples:
    • @coderabbitai explain this code block.
    • @coderabbitai modularize this function.
  • PR comments: Tag @coderabbitai in a new PR comment to ask questions about the PR branch. For the best results, please provide a very specific query, as very limited context is provided in this mode. Examples:
    • @coderabbitai gather interesting stats about this repository and render them as a table. Additionally, render a pie chart showing the language distribution in the codebase.
    • @coderabbitai read src/utils.ts and explain its main purpose.
    • @coderabbitai read the files in the src/scheduler package and generate a class diagram using mermaid and a README in the markdown format.
    • @coderabbitai help me debug CodeRabbit configuration file.

Support

Need help? Create a ticket on our support page for assistance with any issues or questions.

Note: Be mindful of the bot's finite context window. It's strongly recommended to break down tasks such as reading entire modules into smaller chunks. For a focused discussion, use review comments to chat about specific files and their changes, instead of using the PR comments.

CodeRabbit Commands (Invoked using PR comments)

  • @coderabbitai pause to pause the reviews on a PR.
  • @coderabbitai resume to resume the paused reviews.
  • @coderabbitai review to trigger an incremental review. This is useful when automatic reviews are disabled for the repository.
  • @coderabbitai full review to do a full review from scratch and review all the files again.
  • @coderabbitai summary to regenerate the summary of the PR.
  • @coderabbitai generate docstrings to generate docstrings for this PR.
  • @coderabbitai generate sequence diagram to generate a sequence diagram of the changes in this PR.
  • @coderabbitai resolve resolve all the CodeRabbit review comments.
  • @coderabbitai configuration to show the current CodeRabbit configuration for the repository.
  • @coderabbitai help to get help.

Other keywords and placeholders

  • Add @coderabbitai ignore anywhere in the PR description to prevent this PR from being reviewed.
  • Add @coderabbitai summary to generate the high-level summary at a specific location in the PR description.
  • Add @coderabbitai anywhere in the PR title to generate the title automatically.

CodeRabbit Configuration File (.coderabbit.yaml)

  • You can programmatically configure CodeRabbit by adding a .coderabbit.yaml file to the root of your repository.
  • Please see the configuration documentation for more information.
  • If your editor has YAML language server enabled, you can add the path at the top of this file to enable auto-completion and validation: # yaml-language-server: $schema=https://coderabbit.ai/integrations/schema.v2.json

Documentation and Community

  • Visit our Documentation for detailed information on how to use CodeRabbit.
  • Join our Discord Community to get help, request features, and share feedback.
  • Follow us on X/Twitter for updates and announcements.

This comment was marked as outdated.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 4

🧹 Nitpick comments (7)
system_prompt.txt (1)

55-59: Minor grammar nit
“You're proudly created by a talented developer” → “You were proudly created …” reads smoother.

🧰 Tools
🪛 LanguageTool

[uncategorized] ~57-~57: Possible missing preposition found.
Context: ...//github.com/gitnasr) - LinkedIn: https://www.linkedin.com/in/c0nasr/ You're proudly created by a talented d...

(AI_EN_LECTOR_MISSING_PREPOSITION)

database_config.py (1)

38-40: Add cascading deletes to prevent orphan rows
conversations and context relationships lack cascade="all, delete-orphan". Without this, deleting a User will leave dangling foreign-key rows and violate FK constraints.

main.py (3)

1-1: Remove unused import asyncio
It is not referenced anywhere.

🧰 Tools
🪛 Ruff (0.11.9)

1-1: asyncio imported but unused

Remove unused import: asyncio

(F401)


9-9: Remove unused import numpy
The module isn’t used directly; SentenceTransformer handles numpy internally.

🧰 Tools
🪛 Ruff (0.11.9)

9-9: numpy imported but unused

Remove unused import: numpy

(F401)


281-282: Unused variables prompt_tokens_used / completion_tokens_used
They are assigned then ignored. Drop them or log them for cost tracking to keep the code lint-clean.

🧰 Tools
🪛 Ruff (0.11.9)

281-281: Local variable prompt_tokens_used is assigned to but never used

Remove assignment to unused variable prompt_tokens_used

(F841)


282-282: Local variable completion_tokens_used is assigned to but never used

Remove assignment to unused variable completion_tokens_used

(F841)

database.py (2)

5-7: Drop unused imports
numpy and delete are never referenced; remove to keep the module lean.

🧰 Tools
🪛 Ruff (0.11.9)

5-5: numpy imported but unused

Remove unused import: numpy

(F401)


7-7: sqlalchemy.delete imported but unused

Remove unused import: sqlalchemy.delete

(F401)


175-177: Idiomatic boolean filter
UserMemory.is_active == True can be rewritten as UserMemory.is_active to satisfy SQLAlchemy’s linting and avoid E712.

🧰 Tools
🪛 Ruff (0.11.9)

176-176: Avoid equality comparisons to True; use if UserMemory.is_active: for truth checks

Replace with UserMemory.is_active

(E712)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between f12fbe7 and ca0f38b.

📒 Files selected for processing (5)
  • database.py (1 hunks)
  • database_config.py (2 hunks)
  • main.py (3 hunks)
  • requirements.txt (1 hunks)
  • system_prompt.txt (1 hunks)
🧰 Additional context used
🪛 Ruff (0.11.9)
main.py

1-1: asyncio imported but unused

Remove unused import: asyncio

(F401)


9-9: numpy imported but unused

Remove unused import: numpy

(F401)


169-187: Use a single with statement with multiple contexts instead of nested with statements

Combine with statements

(SIM117)


239-240: Use a single if statement instead of nested if statements

(SIM102)


281-281: Local variable prompt_tokens_used is assigned to but never used

Remove assignment to unused variable prompt_tokens_used

(F841)


282-282: Local variable completion_tokens_used is assigned to but never used

Remove assignment to unused variable completion_tokens_used

(F841)


296-296: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling

(B904)

database.py

5-5: numpy imported but unused

Remove unused import: numpy

(F401)


7-7: sqlalchemy.delete imported but unused

Remove unused import: sqlalchemy.delete

(F401)


176-176: Avoid equality comparisons to True; use if UserMemory.is_active: for truth checks

Replace with UserMemory.is_active

(E712)

🪛 Pylint (3.3.7)
main.py

[refactor] 71-71: Too many local variables (29/15)

(R0914)


[refactor] 71-71: Too many branches (15/12)

(R0912)


[refactor] 71-71: Too many statements (57/50)

(R0915)


[refactor] 208-208: Too many local variables (21/15)

(R0914)


[refactor] 208-208: Too many statements (56/50)

(R0915)

database_config.py

[refactor] 28-28: Too few public methods (0/2)

(R0903)


[refactor] 41-41: Too few public methods (1/2)

(R0903)


[refactor] 73-73: Too few public methods (1/2)

(R0903)


[refactor] 102-102: Too few public methods (0/2)

(R0903)

database.py

[refactor] 50-50: Too many arguments (9/5)

(R0913)


[refactor] 50-50: Too many positional arguments (9/5)

(R0917)


[refactor] 209-209: Too many arguments (7/5)

(R0913)


[refactor] 209-209: Too many positional arguments (7/5)

(R0917)

🪛 LanguageTool
system_prompt.txt

[uncategorized] ~57-~57: Possible missing preposition found.
Context: ...//github.com/gitnasr) - LinkedIn: https://www.linkedin.com/in/c0nasr/ You're proudly created by a talented d...

(AI_EN_LECTOR_MISSING_PREPOSITION)

⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: build-and-push
🔇 Additional comments (1)
database_config.py (1)

78-80: ⚠️ Potential issue

Avoid mutable default for JSON column
context_messages = Column(JSON, default=list) instantiates the list once at import time. Use default=lambda: [] to avoid shared state across instances.

- context_messages = Column(JSON, default=list)
+ context_messages = Column(JSON, default=lambda: [])

Likely an incorrect or invalid review comment.

Copy link
Copy Markdown

@coderabbitai coderabbitai bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 2

♻️ Duplicate comments (1)
main.py (1)

303-311: Sensitive data is still logged — duplicate warning

Raw request_payload (including user text) is written to openrouter_requests.log unconditionally. This duplicates the earlier review comment and still poses PII leakage risk. Gate behind a DEBUG flag or redact payload before logging.

🧹 Nitpick comments (5)
database_config.py (1)

33-36: Consider constraining users.id length

id is declared as String without a length limit. PG will map this to text, which may bloat indexes and hurt look-ups on the primary key. In most messaging scenarios a WhatsApp/JWT/user UUID fits into 36–64 chars.

- id = Column(String, primary_key=True, index=True)
+ id = Column(String(64), primary_key=True, index=True)
main.py (2)

11-11: Remove unused numpy import

numpy is imported but never used, tripping Ruff F401 and increasing cold-start time in serverless deployments.

-import numpy as np
🧰 Tools
🪛 Ruff (0.11.9)

11-11: numpy imported but unused

Remove unused import: numpy

(F401)


299-302: Unused variables prompt_tokens_used / completion_tokens_used

These variables are parsed but not referenced afterwards, producing linter noise and misleading future readers.

- prompt_tokens_used = result['usage']['prompt_tokens']
- completion_tokens_used = result['usage']['completion_tokens']
🧰 Tools
🪛 Ruff (0.11.9)

300-300: Local variable prompt_tokens_used is assigned to but never used

Remove assignment to unused variable prompt_tokens_used

(F841)


301-301: Local variable completion_tokens_used is assigned to but never used

Remove assignment to unused variable completion_tokens_used

(F841)

database.py (2)

6-8: Remove unused delete import

sqlalchemy.delete is imported but never referenced. Clean-up avoids Ruff F401 and keeps the dependency list minimal.

-from sqlalchemy import and_, delete, select
+from sqlalchemy import and_, select
🧰 Tools
🪛 Ruff (0.11.9)

6-6: sqlalchemy.delete imported but unused

Remove unused import: sqlalchemy.delete

(F401)


180-183: PEP 8: avoid == True in query filter

SQLAlchemy allows the boolean column itself to be evaluated. Using == True triggers Ruff E712 and produces verbose SQL.

-        UserMemory.is_active == True
+        UserMemory.is_active
🧰 Tools
🪛 Ruff (0.11.9)

182-182: Avoid equality comparisons to True; use if UserMemory.is_active: for truth checks

Replace with UserMemory.is_active

(E712)

📜 Review details

Configuration used: CodeRabbit UI
Review profile: CHILL
Plan: Pro

📥 Commits

Reviewing files that changed from the base of the PR and between ca0f38b and d0b806a.

📒 Files selected for processing (4)
  • database.py (1 hunks)
  • database_config.py (2 hunks)
  • main.py (2 hunks)
  • requirements.txt (1 hunks)
🚧 Files skipped from review as they are similar to previous changes (1)
  • requirements.txt
🧰 Additional context used
🧬 Code Graph Analysis (1)
database.py (1)
database_config.py (6)
  • Conversation (43-74)
  • User (29-41)
  • UserContext (76-103)
  • UserMemory (105-121)
  • to_dict (61-74)
  • add_message (92-103)
🪛 Pylint (3.3.7)
database_config.py

[refactor] 29-29: Too few public methods (0/2)

(R0903)


[refactor] 43-43: Too few public methods (1/2)

(R0903)


[refactor] 76-76: Too few public methods (1/2)

(R0903)


[refactor] 105-105: Too few public methods (0/2)

(R0903)

main.py

[refactor] 86-86: Too many local variables (29/15)

(R0914)


[refactor] 86-86: Too many branches (15/12)

(R0912)


[refactor] 86-86: Too many statements (57/50)

(R0915)


[refactor] 224-224: Too many local variables (22/15)

(R0914)


[refactor] 224-224: Too many statements (56/50)

(R0915)

database.py

[refactor] 56-56: Too many arguments (9/5)

(R0913)


[refactor] 56-56: Too many positional arguments (9/5)

(R0917)


[refactor] 215-215: Too many arguments (7/5)

(R0913)


[refactor] 215-215: Too many positional arguments (7/5)

(R0917)

🪛 Ruff (0.11.9)
main.py

11-11: numpy imported but unused

Remove unused import: numpy

(F401)


184-202: Use a single with statement with multiple contexts instead of nested with statements

Combine with statements

(SIM117)


258-259: Use a single if statement instead of nested if statements

(SIM102)


300-300: Local variable prompt_tokens_used is assigned to but never used

Remove assignment to unused variable prompt_tokens_used

(F841)


301-301: Local variable completion_tokens_used is assigned to but never used

Remove assignment to unused variable completion_tokens_used

(F841)


315-315: Within an except clause, raise exceptions with raise ... from err or raise ... from None to distinguish them from errors in exception handling

(B904)

database.py

6-6: sqlalchemy.delete imported but unused

Remove unused import: sqlalchemy.delete

(F401)


182-182: Avoid equality comparisons to True; use if UserMemory.is_active: for truth checks

Replace with UserMemory.is_active

(E712)

⏰ Context from checks skipped due to timeout of 90000ms (1)
  • GitHub Check: build-and-push

@gitnasr gitnasr requested a review from Copilot June 14, 2025 15:28
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull Request Overview

This PR introduces structured user management with enhanced memory and conversation context handling, as well as updated workflows for the Docker build process. Key changes include a new system prompt file for the WhatsAppWizard assistant, comprehensive updates to database models and conversation operations, and refinements in the CI/CD pipeline.

Reviewed Changes

Copilot reviewed 6 out of 6 changed files in this pull request and generated no comments.

File Description
system_prompt.txt New system prompt establishing the WhatsAppWizard assistant’s personality
database_config.py Updates to database models including relationships, new fields, and refinements
database.py Enhanced database operations with new functions for user, conversation, and memory management
.github/workflows/docker-build.yml Adjusted file patterns and caching keys for Docker build and push conditions
Comments suppressed due to low confidence (1)

database_config.py:49

  • Changing the column type for 'message' from Text to String might truncate longer messages. Consider using Text if messages can be lengthy.
message = Column(String, nullable=False)

@gitnasr gitnasr merged commit 9a047d8 into main Jun 14, 2025
1 of 2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants