Skip to content

fix(discord): handle_message blocks event loop during streaming, starving other sessions #429

@thepagent

Description

@thepagent

Description

When one session is streaming a long response, all other incoming Discord messages are blocked until the streaming completes. This happens even though the session pool correctly uses per-connection locks (fixed in #257).

Steps to Reproduce

  1. Deploy OpenAB with pool_max=3
  2. Send a message that triggers a long streaming response (30s+)
  3. While streaming, send messages from other users/threads
  4. Observe that messages B and C are not processed until A finishes

Expected Behavior

All sessions should stream concurrently. A long response in one session should not block other sessions.

Root Cause

openab/src/discord.rs

Lines 430 to 436 in 67a8e30

if let Err(e) = self
.router
.handle_message(&adapter, &thread_channel, &sender, &prompt, extra_blocks, &trigger_msg)
.await
{
error!("handle_message error: {e}");
}

Affected code: src/discord.rs#L430-L436

discord.rs awaits handle_message() directly in the serenity event handler:

async fn message(&self, ctx: Context, msg: Message) {
    // ... filtering ...
    self.router.handle_message(...).await  // ← blocks event loop
}

handle_message runs the entire streaming loop (which can take 30+ seconds) before returning. Since serenity dispatches events sequentially through the handler, no other messages are processed until the current one finishes.

┌──────────────────────────────────────────────────────────┐
│ Serenity Event Loop                                      │
│                                                          │
│  msg A arrives ──> handle_message().await                 │
│                    ┌────────────────────────────────┐    │
│                    │ streaming response (30s+)...   │    │
│                    └────────────────────────────────┘    │
│                                                          │
│  msg B arrives ──> ⏳ QUEUED (event loop blocked)         │
│  msg C arrives ──> ⏳ QUEUED (event loop blocked)         │
│                                                          │
│  A finishes ──> B dispatched ──> C dispatched             │
└──────────────────────────────────────────────────────────┘

What should happen:

┌──────────────────────────────────────────────────────────┐
│ Serenity Event Loop (with tokio::spawn)                  │
│                                                          │
│  msg A arrives ──> spawn(handle_message) ──> return       │
│  msg B arrives ──> spawn(handle_message) ──> return       │
│  msg C arrives ──> spawn(handle_message) ──> return       │
│                                                          │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐               │
│  │ Task A   │  │ Task B   │  │ Task C   │  (concurrent)  │
│  │streaming │  │streaming │  │streaming │               │
│  └──────────┘  └──────────┘  └──────────┘               │
└──────────────────────────────────────────────────────────┘

Proposed Fix

Spawn handle_message as a separate tokio task so the event handler returns immediately:

async fn message(&self, ctx: Context, msg: Message) {
    // ... filtering ...
    let router = self.router.clone();
    tokio::spawn(async move {
        if let Err(e) = router.handle_message(...).await {
            error!("handle_message error: {e}");
        }
    });
}

Impact

Related

Metadata

Metadata

Assignees

No one assigned

    Labels

    bugSomething isn't workingdiscordp1High — address this sprintsession

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions