Skip to content

feat(context): introduce GroupRequest and ContextGroupId types for group management#2043

Open
rtb-12 wants to merge 41 commits intomasterfrom
feat/context-management-proposal
Open

feat(context): introduce GroupRequest and ContextGroupId types for group management#2043
rtb-12 wants to merge 41 commits intomasterfrom
feat/context-management-proposal

Conversation

@rtb-12
Copy link
Contributor

@rtb-12 rtb-12 commented Feb 19, 2026

[core] Implement hierarchical context group management

Description

Implements the full Context Groups feature across core/crates/, enabling administrators to organize contexts into groups with shared application targets, coordinated upgrades, and membership management. Built across 6 phases on top of the context-config contract types (ContextGroupId, GroupRequest, AppKey).

Phase 1 — Storage Foundation

  • 5 new key structs in store/src/key/group.rs: GroupMeta (0x20), GroupMember (0x21), GroupContextIndex (0x22), ContextGroupRef (0x23), GroupUpgradeKey (0x24) — all with AsKeyParts/FromKeyParts impls and roundtrip tests
  • Value types: GroupMetaValue, GroupUpgradeValue, GroupUpgradeStatus with Borsh serialization
  • Primitive types in primitives/src/context.rs: UpgradePolicy (Automatic / LazyOnAccess / Coordinated), GroupMemberRole (Admin / Member), GroupInvitationPayload

Phase 2 — Group CRUD + Membership

  • group_store.rs (~390 LOC): Complete storage helper layer — CRUD for group metadata, members, context indices, and upgrade records with efficient key-only iteration for counting
  • 6 actor handlers: create_group, delete_group, add_group_members, remove_group_members, get_group_info, list_group_members
  • 6 admin API routes with corresponding server handlers

Phase 3 — Context-Group Integration

  • create_context.rs: Pre-validates group membership + app override; post-creation registers context in group index
  • delete_context.rs: Unregisters context from group on deletion
  • list_group_contexts: New paginated endpoint for listing a group's contexts

Phase 4 — Upgrade Propagation

  • Canary-first upgrade strategy: Upgrades the first context as a canary; on success, spawns an async propagator for the remaining contexts
  • propagate_upgrade(): Async fn that iterates group contexts, calls update_application() per context, persists progress after each step
  • Status tracking: GroupUpgradeStatus::InProgress { total, completed, failed }Completed { completed_at }
  • 3 API endpoints: POST /upgrade, GET /upgrade/status, POST /upgrade/retry

Phase 5 — Advanced Policies + Crash Recovery

  • Lazy-on-access upgrade: execute.rs transparently upgrades stale contexts before method execution when UpgradePolicy::LazyOnAccess is set
  • Crash recovery: ContextManager::started() scans for InProgress upgrades and re-spawns propagators
  • Auto-retry: Failed context upgrades are retried up to 3× with exponential backoff (5s, 10s, 20s)

Phase 6 — Group Invitations + Join Flow

  • GroupInvitationPayload: Borsh-serialized + base58-encoded invitation token (mirrors ContextInvitationPayload pattern)
  • Targeted (invitee_identity: Some) and open (invitee_identity: None) invitation modes
  • Join validation: Verifies inviter is still admin, checks invitee identity match, validates expiration
  • 2 API endpoints: POST /groups/:id/invite, POST /groups/join

Post-implementation fixes (PR review)

  • Phase 1: Fixed off-by-one in propagate_upgrade completed counter on retry/recovery paths
  • Phase 2: Fixed swallowed store errors, stale GroupContextIndex on re-registration, orphaned upgrade records on group deletion, Duration::new panic on malformed Borsh, switched to ValidatedJson for upgrade input
  • Phase 3: Rewrote count_group_admins/count_group_contexts without Vec allocation, added offset/limit to enumerate_group_contexts, batched member cleanup in delete_group
  • Phase 4: Decoupled GroupUpgradeValue/GroupUpgradeStatus store types from calimero-context-primitives API boundary — introduced GroupUpgradeInfo and primitives-local GroupUpgradeStatus with From conversion impls
  • Phase 5: Added count_group_members() for efficient member counting, inlined value read in enumerate_in_progress_upgrades to reuse store handle

New API routes

POST   /admin-api/groups                              → CreateGroup
GET    /admin-api/groups/:group_id                    → GetGroupInfo
DELETE /admin-api/groups/:group_id                    → DeleteGroup
POST   /admin-api/groups/:group_id/members            → AddGroupMembers
POST   /admin-api/groups/:group_id/members/remove     → RemoveGroupMembers
GET    /admin-api/groups/:group_id/members            → ListGroupMembers
GET    /admin-api/groups/:group_id/contexts           → ListGroupContexts
POST   /admin-api/groups/:group_id/upgrade            → UpgradeGroup
GET    /admin-api/groups/:group_id/upgrade/status     → GetGroupUpgradeStatus
POST   /admin-api/groups/:group_id/upgrade/retry      → RetryGroupUpgrade
POST   /admin-api/groups/:group_id/invite             → CreateGroupInvitation
POST   /admin-api/groups/join                         → JoinGroup

Test plan

  • cargo check --workspace — compiles cleanly
  • cargo fmt --check — no formatting issues
  • cargo clippy -- -A warnings — no errors
  • cargo test -p calimero-context — 16 tests pass
  • cargo test -p calimero-context-primitives — 3 tests pass
  • cargo test -p calimero-store — key roundtrip + value serialization tests pass
  • End-to-end group lifecycle tests via meroctl against a running merod node
  • Contract integration tests in contracts/ repo

Documentation update

N/A — these are internal protocol types. Public API documentation should be added once the full group management feature is shipped end-to-end.


Note

High Risk
High risk because this introduces new persistent Store column/keyspace for groups, adds new admin API surface, and changes runtime execution to optionally perform lazy upgrades before method execution; mistakes could impact upgrade safety and context availability.

Overview
Adds Context Groups as a first-class feature: new ContextGroupId/AppKey types and GroupRequest contract/config structs, plus node-side group CRUD, membership management, and context↔group indexing stored in a new Store Column::Group keyspace.

Introduces group-wide application upgrade orchestration (canary + propagator with progress tracking, auto-retry/backoff, manual retry, and crash recovery on startup) and a LazyOnAccess policy that can trigger a context update_application before executing user methods.

Extends the admin server with /groups/* endpoints (create/delete/info, members, contexts, upgrade/status/retry, invite/join) and integrates group membership into context lifecycle (optional group_id on create with app-target override + membership checks, unregister on delete). Also hardens store iteration to skip mismatched-size keys when multiple key types share a column.

Written by Cursor Bugbot for commit e5643bf. This will update automatically on new commits. Configure here.

Copy link

@meroreviewer meroreviewer bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 100% | Review time: 167.3s

💡 1 suggestions. See inline comments.


🤖 Generated by AI Code Reviewer | Review ID: review-3e67acf7

Copy link

@meroreviewer meroreviewer bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 100% | Review time: 227.1s

💡 3 suggestions. See inline comments.


🤖 Generated by AI Code Reviewer | Review ID: review-078e5b50

Copy link

@meroreviewer meroreviewer bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 100% | Review time: 203.4s

💡 2 suggestions, 📝 1 nitpicks. See inline comments.


🤖 Generated by AI Code Reviewer | Review ID: review-86d4a70f

@github-actions
Copy link

E2E Blockchain Proposals Failed

The following proposal workflow(s) failed:

  • near

Please check the workflow logs for more details.

Copy link

@meroreviewer meroreviewer bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 100% | Review time: 197.8s

🟡 2 warnings, 💡 2 suggestions. See inline comments.


🤖 Generated by AI Code Reviewer | Review ID: review-04819256

…etaValue, GroupUpgradeValue, and GroupUpgradeStatus structures for enhanced group upgrade management
Copy link

@meroreviewer meroreviewer bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 89% | Review time: 319.2s

🟡 2 warnings, 💡 4 suggestions. See inline comments.


🤖 Generated by AI Code Reviewer | Review ID: review-f10665bd

Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 2 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

…y is removed

When the canary context is removed from the group between canary
completion and propagator enumeration, the fresh enumeration yields
N-1 contexts but the skip_context filter removes nothing (canary is
gone). Starting completed at initial_completed=1 then produces
completed=N which exceeds total=N-1, a logically invalid state.

Check whether the canary was actually present in the fresh enumeration
before counting it toward completed.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link

@meroreviewer meroreviewer bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 100% | Review time: 312.1s

🟡 2 warnings, 💡 6 suggestions. See inline comments.


🤖 Generated by AI Code Reviewer | Review ID: review-bbf56092

Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Two fixes:

1. The retry loop checked `attempt >= MAX_AUTO_RETRIES` after
   incrementing, so only 2 retries (5s, 10s) ran instead of the
   documented 3 (5s, 10s, 20s). Changed to `attempt > MAX_AUTO_RETRIES`.

2. The manual retry handler could spawn a new propagator while the
   previous one was still sleeping in its exponential backoff, causing
   conflicting status writes and double-counted completions. Added an
   `active_propagators` guard to ContextManager that is set when a
   propagator is spawned and cleared when it finishes. The retry handler
   rejects requests while a propagator is active.

Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
Copy link
Contributor

@cursor cursor bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Cursor Bugbot has reviewed your changes and found 3 potential issues.

Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.

Copy link

@meroreviewer meroreviewer bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

🤖 AI Code Reviewer

Reviewed by 3 agents | Quality score: 83% | Review time: 385.2s

🔴 2 critical, 🟡 3 warnings, 💡 5 suggestions. See inline comments.


🤖 Generated by AI Code Reviewer | Review ID: review-eac9a89c

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant