-
-
Notifications
You must be signed in to change notification settings - Fork 254
fix(core-backend): improve traces #7101
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
base: main
Are you sure you want to change the base?
Conversation
9f2a6b4 to
0894c21
Compare
249675d to
9f663a7
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Bug: Disconnect Undoes Scheduled Reconnection Timer
forceReconnection() fails to reconnect because the scheduled reconnection timer is cancelled. It calls disconnect() which triggers ws.close() with MANUAL_DISCONNECT_CODE, then immediately calls #scheduleReconnect() to schedule a reconnection timer. However, when ws.onclose fires asynchronously, it detects the manual disconnect code and calls #clearTimers(), which cancels the reconnection timer that was just scheduled. This leaves the WebSocket disconnected with no reconnection attempt, breaking the intended disconnect-then-reconnect behavior.
packages/core-backend/src/BackendWebSocketService.ts#L519-L534
core/packages/core-backend/src/BackendWebSocketService.ts
Lines 519 to 534 in 9f663a7
| if (this.#reconnectTimer) { | |
| log('Reconnect already scheduled, skipping force reconnection'); | |
| return; | |
| } | |
| log('Forcing WebSocket reconnection to clean up subscription state'); | |
| // Perform controlled disconnect | |
| this.disconnect(); | |
| // Schedule reconnection with exponential backoff | |
| this.#scheduleReconnect(); | |
| } | |
| /** | |
| * Sends a message through the WebSocket (fire-and-forget, no response expected) |
Bug: Forced Reconnection Silently Cancelled
forceReconnection() schedules a reconnect timer that gets immediately cleared when ws.onclose fires. The method calls disconnect() then #scheduleReconnect() synchronously, but disconnect() triggers an asynchronous onclose event. When onclose fires with the manual disconnect code, it calls #clearTimers() which clears the reconnect timer that was just scheduled, and sets #reconnectAttempts to 0 without rescheduling. The forced reconnection never occurs, breaking the documented behavior that states it "schedules reconnection with exponential backoff".
packages/core-backend/src/BackendWebSocketService.ts#L519-L534
core/packages/core-backend/src/BackendWebSocketService.ts
Lines 519 to 534 in 9f663a7
| if (this.#reconnectTimer) { | |
| log('Reconnect already scheduled, skipping force reconnection'); | |
| return; | |
| } | |
| log('Forcing WebSocket reconnection to clean up subscription state'); | |
| // Perform controlled disconnect | |
| this.disconnect(); | |
| // Schedule reconnection with exponential backoff | |
| this.#scheduleReconnect(); | |
| } | |
| /** | |
| * Sends a message through the WebSocket (fire-and-forget, no response expected) |
9f663a7 to
163dffa
Compare
163dffa to
03b0cc2
Compare
03b0cc2 to
b2d1def
Compare
Explanation
While this PR is focusing on improve traces, I took the opportunity to clean the code around disconnection. This PR:
Changed
BackendWebSocketService(#7101)connectionDuration_msfrom disconnection traces when connection never established (onClose without onOpen)BackendWebSocketServicedefault exponential backoff options for reconnection (#7101)reconnectDelayfrom 500 milliseconds to 10 secondsmaxReconnectDelayfrom 30 seconds to 60 secondsBackendWebSocketService(#7101)ws.onclosehandler for single source of truth#establishConnectionmethod - state transitions only occur inonopen(CONNECTING → CONNECTED) andonclose(any state → DISCONNECTED)MANUAL_DISCONNECT_CODE(4999) andMANUAL_DISCONNECT_REASONconstants to distinguish manual from unexpected disconnectsRemoved
BackendWebSocketService Channel Messagetrace as it provided no useful performance insights (#7101)Before:

After:

References
Checklist
Note
Refactors WebSocket lifecycle to centralize disconnection handling with improved tracing and less aggressive reconnection backoff, updating AccountActivityService, tests, and docs accordingly.
BackendWebSocketService):ws.onclose; add disconnection trace with optionalconnectionDuration_ms; remove channel message trace.CONNECTING|CONNECTED|DISCONNECTED; addMANUAL_DISCONNECT_CODE(4999) andFORCE_RECONNECT_CODE(4998); disconnect errors now include close code/reason.reconnectDelay→10000ms,maxReconnectDelay→60000ms; ensure idempotent scheduling; clear timers on destroy/disable.#establishConnection; resolve connect on manual close during CONNECTING; timeout drivesonclosepath; remove manual error state.disconnect()uses manual close;forceReconnection()uses force code; cleanup pending requests/subscriptions on close; adjust notification tracing.DISCONNECTEDas down signal; publish tracked chains asdownon disconnect; add concise status-change logging.Written by Cursor Bugbot for commit b2d1def. This will update automatically on new commits. Configure here.