Skip to content

feat: add retention policy for agent chats#4039

Open
0vertake wants to merge 23 commits intosuperplanehq:mainfrom
0vertake:feat/agent-chat-retention
Open

feat: add retention policy for agent chats#4039
0vertake wants to merge 23 commits intosuperplanehq:mainfrom
0vertake:feat/agent-chat-retention

Conversation

@0vertake
Copy link
Copy Markdown
Collaborator

@0vertake 0vertake commented Apr 8, 2026

Summary

  • Hourly background task in the agent service deletes expired chats per organization, using each org's retention_window_days from the usage service
  • Falls back to AGENT_CHAT_RETENTION_DAYS env var (default 14) when the usage service is unavailable or returns no policy for an org
  • Setting AGENT_CHAT_RETENTION_DAYS=0 disables the fallback but still respects per-org policies from the usage service
  • Deletes are batched (500 rows per transaction) with a race-safe outer WHERE clause for READ COMMITTED isolation
  • Adds composite index on (org_id, updated_at) for efficient per-org cleanup queries
  • Fixes missing await on UsageLimitChecker.close()

Closes #3818

@superplanehq-integration
Copy link
Copy Markdown

👋 Commands for maintainers:

  • /sp start - Start an ephemeral machine (takes ~30s)
  • /sp stop - Stop a running machine (auto-executed on pr close)

@0vertake 0vertake requested a review from lucaspin April 8, 2026 13:07
Comment thread agent/tests/test_session_store.py Outdated
Comment thread agent/src/ai/session_store.py Outdated
Comment thread agent/db/structure.sql Outdated
Comment thread agent/src/ai/session_store.py Outdated
@lucaspin
Copy link
Copy Markdown
Contributor

@0vertake now that the agent service is talking with the usage service, I think the retention policy configuration should be based on the limit for retention policy for the organization. @forestileao can you help here?

@forestileao
Copy link
Copy Markdown
Collaborator

@0vertake now that the agent service is talking with the usage service, I think the retention policy configuration should be based on the limit for retention policy for the organization. @forestileao can you help here?

I will need to add a retention policy limit to agent chats on usage service. Then he will only need to grab this value to cleanup stuff.

@lucaspin
Copy link
Copy Markdown
Contributor

@forestileao can't we just use the same retention policy we have there for everything else already?

@forestileao
Copy link
Copy Markdown
Collaborator

@forestileao can't we just use the same retention policy we have there for everything else already?

We can, no problem. Unless we want to make different policies, but I don't think we need

Comment thread agent/src/ai/usage_limit_checker.py Outdated
@0vertake 0vertake force-pushed the feat/agent-chat-retention branch from e398aae to 0c9823c Compare April 13, 2026 13:06
Comment thread agent/db/migrations/20260413132301_add-agent-chats-updated-at-index.up.sql Outdated
Remove agent chats from the database based on each organization's
retention window from the usage service, falling back to a configurable
default (AGENT_CHAT_RETENTION_DAYS, default 14 days, 0 to disable).

- Add per-org delete_expired_chats_for_org to SessionStore with batched
  DELETEs and a race-safe outer WHERE clause for READ COMMITTED isolation
- Query each org's retention_window_days via the usage service gRPC;
  fall back to the env var default when the service is unavailable
- Add background async task (chat_retention.py) wired into the
  FastAPI lifespan, checking once per hour
- Add updated_at index on agent_chats for query performance
- Fix missing await on UsageLimitChecker.close()
- Add unit tests for the store method and the retention loop

Closes superplanehq#3818

Signed-off-by: Milos Jovanovic <milosjovanovic519@gmail.com>
Made-with: Cursor
@0vertake 0vertake force-pushed the feat/agent-chat-retention branch from 0c9823c to 8e6974d Compare April 13, 2026 13:45
Comment thread agent/src/ai/usage_limit_checker.py Outdated
Signed-off-by: Milos Jovanovic <milosjovanovic519@gmail.com>
Signed-off-by: Milos Jovanovic <milosjovanovic519@gmail.com>
Comment thread agent/src/ai/usage_limit_checker.py
Signed-off-by: Milos Jovanovic <milosjovanovic519@gmail.com>
Signed-off-by: Milos Jovanovic <milosjovanovic519@gmail.com>
Comment thread agent/src/ai/chat_retention.py
Comment thread agent/src/ai/chat_retention.py Outdated
Comment thread agent/src/ai/session_store.py Outdated
Comment thread agent/src/ai/chat_retention.py Outdated
Copy link
Copy Markdown
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Fix All in Cursor

❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Reviewed by Cursor Bugbot for commit 2feebfb. Configure here.

Comment thread agent/src/ai/session_store.py Outdated
Comment thread agent/tests/test_session_store.py Outdated
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Retention policy for agent chats

4 participants