feat(react): concurrent-thread streaming via concurrentThreads prop

[Gourav-InfoTech]

Concurrent-thread streaming — build notes

Adds a new opt-in capability to CopilotProvider: users can switch away
from a generating thread, start a second thread, send a prompt in it — and
the first thread keeps streaming in the background. Both threads show a
spinner in the picker; switching back to either one shows its live state.

Previously the SDK held exactly one ReactChatWithTools instance per
CopilotProvider, and the thread picker disabled itself the moment
isBusy went true. The feature request was to remove that limit.

This doc captures everything that changed to make that work, the bugs we
hit along the way, and what needs to be verified before shipping.

What was built

• A registry of chat instances keyed by thread id inside CopilotProvider,
  gated behind a new concurrentThreads prop (default false — so every
  existing consumer keeps running on the old single-instance code path
  byte-for-byte).
• A stable useSyncExternalStore wrapper so swapping the active instance
  doesn't tear React state.
• Per-instance dispose() so deleting a thread aborts its in-flight request
  cleanly.
• A busyThreadIds: ReadonlySet<string> surface for the picker to render
  per-thread spinners.
• Hooked all this through useInternalThreadManager so persisted threads
  are restored with their messages hydrated into the right instance, and
  the thread picker lights up immediately even while the server is still
  streaming the first reply.
• Extension-side pieces: CustomThreadPicker no longer locks during
  streaming, ToolsSetup pins browser tabs per-thread so Session 1 on Tab A
  and Session 2 on Tab B can run concurrently.

Files changed in the SDK

File	What it got
packages/copilot-sdk/src/chat/classes/AbstractChat.ts	dispose() / revive() for StrictMode + per-instance cleanup; public get threadId() getter.
packages/copilot-sdk/src/chat/AbstractAgentLoop.ts	Tool handler context now carries threadId (for per-thread state like browser-tab pinning).
packages/copilot-sdk/src/chat/types/tool.ts	getThreadId? added to AgentLoopConfig.
packages/copilot-sdk/src/chat/ChatWithTools.ts	Wires the getThreadId callback; added an overrideable getThreadId option on ChatWithToolsConfig so the provider can supply a stable registry key.
packages/copilot-sdk/src/react/provider/CopilotProvider.tsx	The bulk: instance registry, concurrentThreads prop, busyThreadIds, assignLocalThreadId, disposeThreadInstance, handleInstanceThreadAssigned, provider-level shared tools/skills/context.
packages/copilot-sdk/src/ui/hooks/useInternalThreadManager.ts	Multi-thread-aware phase transitions (early thread creation when a session creator isn't available, delegates hydration to the provider).
packages/copilot-sdk/src/ui/components/composed/connected-chat.tsx	Propagates busyThreadIds, calls disposeThreadInstance on delete.
packages/copilot-sdk/src/ui/components/composed/chat/chat.tsx + types.ts	busyThreadIds plumbed into the chat internals.

Issues we hit building this, and how each got fixed

1. Tool handlers didn't know which tab they were running for

The extension's tools needed per-thread state (which browser tab a given
conversation is "attached to"). There was no id in the tool handler's
context.

Fix: added threadId to ToolContext, fed from AgentLoopConfig.getThreadId.
ChatWithTools initially wired this as () => this.chat.threadId.

2. Stream UI froze under React StrictMode

ReactChatState.dispose() clears its subscribers. revive() didn't re-add
them, so after StrictMode's mount → unmount → remount cycle the provider's
notifyStateChange was detached. Streams kept running in the background
but the UI only repainted when something else (like clicking Stop) forced
a render.

Fix: in the provider's revive branch, explicitly call
inst.subscribe(notifyStateChange) for each revived instance.

3. Picker spinner sometimes didn't show on send

When the active instance started out as an empty pending slot (typical for
a brand-new send) and auto-restore raced in at the same time, the switch
logic would create a second instance and the pending one never made it
into busyThreadIds.

Fix: in switchActiveInstance, promote the current empty internal slot
to the target key instead of spinning up a parallel instance. Auto-restore
now goes through setActiveThread(id, {hydrateMessages}) so the restored
thread ends up on the same instance the provider already created.

4. Tools said "failed" even though the server-side tool succeeded

Provider callbacks that gated on key === activeInstanceKeyRef.current
captured the key string in a closure. When handleInstanceThreadAssigned
later re-keyed the instance from __pending_1__ → the real id, the
captured string went stale and the callback thought it wasn't active
anymore — so tool executions got dropped.

Fix: gate callbacks on instance identity
(inst === chatRef.current) instead of captured strings.

5. Tools not used at all — plain text responses on new threads

Each chat instance owned its own tool registry. When the provider minted a
new pending instance for a second thread, the instance started with zero
tools, so the LLM had nothing to call.

Fix: hoisted tool / skill / system-context registries to the provider
level (sharedToolsRef, sharedSkillsRef, sharedSystemContextRef). New
instances are seeded from these on creation, and registerTool /
unregisterTool / setInlineSkills fan out to every live instance.

6. Pinning broke when user switched browser tabs

Old pin logic pinned the active browser tab while isLoading was true.
If the user switched browser tabs mid-stream, the pin moved — so tools
that started on Tab A finished on Tab B.

Fix: pin per-thread, keyed by thread id. Set when a thread enters
busyThreadIds (captures the current tab at that moment), cleared when it
leaves. Tools look up their tab via context.threadId. A legacy
fallback pin covered the gap before the thread id was assigned.

7. Tools stopped executing when user switched sessions

Switching from Session 1 (streaming, running tools on Tab A) to Session 2
would stop Session 1's tools mid-flow.

Root cause: useAIContext in the extension derives its value from
pageContextData (the current browser tab). Every time the user switched
tabs, pageContextData changed → addContext/removeContext ran in the
provider → the provider fanned that update out to EVERY chat instance. So
Session 1's system prompt got rewritten to reflect Tab B's page, the AI
lost the thread of what it was doing on Tab A, and abandoned the task.

Fix: scope addContext/removeContext to the active instance only.
Background threads keep the context snapshot that was in place when their
instance was created (seeded via sharedSystemContextRef). New instances
created later still get the latest context.

8. New thread didn't appear in the picker until generation finished

The picker is driven by threadManager.threads, and useInternalThreadManager
was waiting for sdkThreadId (server-assigned session id) before calling
createThread(). Backends that only emit the id in the final done chunk
— including the one the extension talks to — meant the picker row didn't
appear at all until after the message was done streaming.

Fix:

• New assignLocalThreadId helper on the provider that re-keys the active
  pending instance to a caller-supplied local id, without touching
  chat.config.threadId (so the server still creates its own session).
• In multi-thread mode, the hook dispatches FIRST_RESPONSE_COMPLETE as
  soon as streaming starts, mints a local id via threadManager.createThread(),
  and hands it to assignLocalThreadId.
• When the server later emits its real id, AbstractChat updates
  chat.config.threadId internally for backend communication, but the
  registry key and UI thread id stay on the local id (a "non-internal-key
  skip" in handleInstanceThreadAssigned).

9. Session 2's tools ran on Session 1's tab, even with per-thread pinning

Once (8) landed, the picker looked right — but tools in Session 2 were
still targeting Tab A. Investigation showed the tool handler's
context.threadId came from chat.config.threadId, which is undefined
for a fresh send (until the server emits its id). Session 2's tools fell
through to the legacy pin, which was captured when Session 1 started
streaming → Tab A.

Fix:

• Added a getThreadId? override to ChatWithToolsConfig. The provider
  wires it to return the instance's registry key (which becomes a
  stable UI id as soon as assignLocalThreadId runs), not
  chat.config.threadId.
• Overlay helpers (showPageOverlay, hidePageOverlay, hidePagePill) in
  the extension now accept a { threadId } context, so the page overlay
  appears on the right session's pinned tab.
• Removed the legacy global pin (it was the source of the cross-session
  bleed) and kept only per-thread pins, plus a lastActivePinRef so the
  settle-all cleanup can hide the overlay on the correct tab.

10. R1 — single-thread renewSession() silently broken

The "non-internal-key skip" from (8) was applied in both modes. In
single-thread mode, calling renewSession() then sending a message meant
the server's new id never propagated to useCopilot().threadId or the
onThreadChange prop (the instance's oldKey was the previous server id,
which is "non-internal", so the skip kicked in).

Fix: gated the skip on concurrentThreads === true:

const shouldRekey = !concurrentThreads || isInternalKey;

Single-thread mode now always re-keys and propagates, matching the
pre-refactor behavior byte-for-byte.

11. R2 — yourgptConfig / onCreateSession decoupled from the thread id in multi-thread mode

Also a consequence of (8). The hook dispatched FIRST_RESPONSE_COMPLETE
the instant streaming started, even when the sessionInitPromise was
~50ms from resolving with the real session id. yourgptConfig consumers
ended up with a locally-generated id in useCopilot().threadId instead of
the session_uid, breaking anyone storing the id server-side.

Fix: added a sessionStatus === "creating" guard. If a pre-stream
session creator is in flight, defer dispatch until either sdkThreadId
arrives (use it) or the creator finishes some other way (proceed with
local-id path as a graceful fallback).

Behavior by mode

Single-thread mode (concurrentThreads={false}, the default)

No behavior changes from pre-feature main. Every existing consumer
keeps running exactly as before. The registry holds one instance under an
internal key; the thread-id assignment path is unchanged.

Multi-thread mode + yourgptConfig / onCreateSession

1. sendMessage → sessionStatus = "creating".
2. Hook effect sees sessionStatus === "creating" — defers.
3. Creator resolves → onThreadChange(sessionId) → sdkThreadId = sessionId.
4. Hook uses sessionId as the thread id. useCopilot().threadId === sessionId.
5. Picker row shows the session id. Reload persists by session id.
6. Same UI/backend id binding as the pre-multi-thread contract.

Multi-thread mode + server-managed sessions (e.g. the extension)

1. sendMessage → sessionStatus stays "idle".
2. Hook dispatches immediately, mints a local id, calls assignLocalThreadId.
3. Picker row appears with a spinner while streaming.
4. Tool handlers get context.threadId = localId (stable).
5. Server emits real id in thread:created/done → chat.config.threadId
   updates internally, registry + picker + useCopilot().threadId keep
   the local id.

What to test

Regression suite — existing single-thread consumers

• examples/playground, examples/togetherai-demo, examples/skills-demo
  behave identically to main: send, stream, stop, clear, reload.
• renewSession() flow: after calling it, send a message; verify
  useCopilot().threadId updates to the new server id AND the
  onThreadChange prop fires with that id.
• yourgptConfig single-thread: first send creates session,
  useCopilot().threadId === session_uid, subsequent sends reuse it.

Multi-thread mode — happy path

• Start Thread A with a long prompt. While it's streaming, verify the
  picker is enabled and shows Thread A with a spinner.
• Open Thread B, send a different prompt. Both show spinners. Switch
  between them — each shows its own live state. No cross-bleed.
• Both complete; reload. Messages for both threads persist correctly.
• Stop scope: start streams in A and B, switch to A, click stop. Only
  A stops; B keeps going.

Multi-thread mode + yourgptConfig

• First send resolves useCopilot().threadId to the YourGPT
  session_uid, NOT a local uuid.
• Picker row id equals the session_uid.
• onThreadChange prop fires once with the session_uid.
• Reload restores by session_uid; follow-up sends continue the same
  backend session.

Extension-specific (concurrentThreads + no session creator)

• Send on a fresh thread → picker row appears with a spinner
  immediately (not after generation finishes).
• While Session 1 is streaming on Tab A, switch to Tab B, create
  Session 2 there, send a prompt. Session 2's tools run on Tab B;
  Session 1 keeps running on Tab A.
• Page overlay appears on the session's pinned tab, not on whichever
  tab the user is currently viewing.
• Stop Session 1; Session 2 continues.
• Delete a streaming thread: its request aborts (DevTools Network
  shows cancelled), registry entry gone, no console errors.
• Reload mid-stream: stream drops, pre-stream persisted messages
  remain, no error banner, no orphan placeholder messages.

StrictMode (dev)

• Load the extension with <React.StrictMode> enabled; verify no
  duplicate instances in the registry, tools fire once per call,
  streaming survives the initial mount→unmount→remount.

Memory

• Open 10+ threads; verify memory stays bounded (LRU keeps ≤ 10 live
  instances).

Known limitations

Reload loses backend session continuity (multi-thread + local-id mode only)

When concurrentThreads={true} AND no session creator is configured, the
UI thread id is locally generated. On reload, the thread manager restores
it, but the server's real session id is NOT persisted. A follow-up send
after reload will send the local id to the server, which will typically
create a new session rather than resume the old one. Persisted messages
still display; only the backend session chain breaks.

Workaround: use yourgptConfig or onCreateSession — then the backend
id IS the thread id and persistence round-trips cleanly.

Proper fix (deferred): add sessionId?: string to ThreadData, persist
the server id alongside the UI id, use it to initialize the chat on
restore.

onThreadChange prop in multi-thread + local-id mode fires with local id only

In multi-thread mode with no session creator, onThreadChange fires once
with the local id. It does NOT fire again when the server later emits its
real session id. Consumers who need the backend id for their own API
calls have no way to get it today.

Proper fix (deferred): add a separate onServerSessionAssigned(uiThreadId, sessionId)
callback.

Background-streaming threads don't recompact

useMessageHistory compaction state only updates for the active thread.
A thread generating in the background while the user views another thread
won't recompact mid-stream. Accepted as a v1 limitation — documented in
the original plan.

Tool approval UI is global in multi-thread mode

If a background-streaming thread calls a tool that requires approval
while the user is viewing another thread, the approval dialog still
surfaces (routing by toolExecutionId, not by thread), so it resolves
correctly regardless of which thread is active. No functional gap, but
worth noting.

Build & typecheck

Both passed clean at commit time:

cd copilot-sdk/packages/copilot-sdk && pnpm build
cd Custom-AI-Knowledgebase-Extension && pnpm tsc --noEmit

[vercel]

@Gourav-InfoTech is attempting to deploy a commit to the Delta4 Infotech Team on Vercel.

A member of the Team first needs to authorize it.