Search results

Continue close monitoring of dowhizprod1 bottlenecks and server faults

P0

Status: Open.

#1571 In KnoWhiz/DoWhiz;

· dtang04 opened on Apr 23, 2026

[P0] devops scan 2026-04-23 01:10 UTC: prod fully degraded — c1792f8 not deployed, 0 success in 2h; 2 new stuck staging tasks

bugSomething isn't workingSomething isn't working

Status: Open.

#1559 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 23, 2026

[P2] list_aci_containers() fails on Cosmos (400 BadValue) — order-by on non-indexed created_at breaks orphan recovery

bugSomething isn't workingSomething isn't working

Status: Open.

#1555 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 23, 2026

[P1] devops scan 2026-04-22 23:00 UTC: 2 new stuck task_executions (prod + staging) — 1 Discord reply sent manually, 1 email retry pending

bugSomething isn't workingSomething isn't working

Status: Open.

#1552 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 22, 2026

[P2] staging worker: PoolManager::replenish panics in non-tokio context (deployed binary stale)

bugSomething isn't workingSomething isn't working

Status: Open.

#1551 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 22, 2026

[P1] prod incident 2026-04-22 19:11 UTC: 27 task_executions stuck in 'running' — users missing scheduled replies

bugSomething isn't workingSomething isn't working

Status: Open.

#1550 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 22, 2026

prod: task_index has 11 cross-user orphan rows pointing to wrong user_id

bugSomething isn't workingSomething isn't working

Status: Open.

#1545 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 22, 2026

ACI delete-request timeout too short (20s) causing cleanup failures in hot path

Status: Open.

#1543 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 22, 2026

User-facing: tasks disabled after max retries only write local file — user gets no reply, no notification

Status: Open.

#1541 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 22, 2026

Scheduler hot-loop on Cron tasks when workspace thread is busy (200% CPU on prod worker)

Status: Open.

#1539 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 22, 2026

[P1] prod: root disk filled by ~/.azcopy logs (9.6GB) → Azure CLI fails → ACI task execution errors

bugSomething isn't workingSomething isn't working

Status: Open.

#1537 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 22, 2026

[P1] prod worker: Azure File SAS tokens generated with expiry ≤ start time (1h window collapses to seconds)

bugSomething isn't workingSomething isn't working

P1

production

Status: Open.

#1533 In KnoWhiz/DoWhiz;

· bingran-you opened on Apr 22, 2026