Critical: App exhausts macOS memory (96GB+) causing system crash

[hyper-charlie]

Description

Char exhausts system memory and crashes the macOS system. The app shows using up to 96GB of memory in Force Quit Applications menu, causing all apps to enter a paused state. Only recovery is force quitting all apps and rebooting.

Frequency

4-5 occurrences in the last 2 weeks

Steps to Reproduce

Unknown - occurs during normal use

Environment

• Device: MacBook Pro, Apple M5
• RAM: 26GB
• macOS Version: Tahoe 26.2
• App Name: Char
• App Version: 1.0.18
• Build SHA: e0bd093
• Platform: macOS (aarch64)
• User ID: 0a22d2ed-38a3-4cca-b3f4-c7e892bfe689

Expected Behavior

App should use reasonable memory and remain stable

Actual Behavior

Memory usage grows uncontrollably, reaching 96GB+, causing system-wide performance degradation and requiring system restart

Severity

Critical - system stability impact

[devin-ai-integration]

Codebase Analysis: Memory Exhaustion Investigation

I explored the codebase at build SHA e0bd093a1f to identify potential sources of the 96GB+ memory exhaustion. Here's a summary of the relevant components and findings.

---

1. Cactus FFI / Native Memory (Highest Suspicion)

Files: crates/cactus/src/model.rs, crates/cactus/src/stt/transcriber.rs, crates/cactus/src/llm/complete.rs

The app uses cactus-sys (rev a5acad3 from cactus-compute/cactus) for on-device STT inference via Apple Neural Engine. This is a C++ FFI layer and is the most likely source of unbounded memory growth on Apple Silicon:

• Model lifecycle: cactus_init allocates native memory; cactus_destroy frees it (called in Model::drop). The ModelManager (crates/model-manager/src/manager.rs) holds the model via Arc<M> and has an inactivity timeout that sets active = None to release it — but if any other Arc clone keeps the model alive, the native memory won't be freed.
• Streaming transcription: Each cactus_stream_transcribe_start allocates a native transcription context. The Transcriber::drop calls cactus_stream_transcribe_stop to free it. However, if a Transcriber is leaked (e.g., panic in the worker thread before drop runs, or the thread is detached without joining), the native context would leak.
• 64KB response buffers: Every process() and stop() call allocates a vec![0u8; 64 * 1024] buffer (RESPONSE_BUF_SIZE). These are per-call and dropped after use, but under continuous streaming they create significant allocation churn.
• Worker thread: transcribe_stream() spawns a dedicated std::thread per transcription session. The TranscriptionSession::drop spawns another thread just to join the worker. If drop never runs (e.g., the session is leaked), both the worker thread and its Arc<Model> reference survive indefinitely.

Key concern: The C++ side (cactus-sys) may have its own internal buffers or caches that grow over time, especially when processing continuous audio streams on Apple Silicon with ANE. This is opaque from the Rust side.

2. Audio Pipeline Buffers (Bounded — Lower Suspicion)

Files: crates/audio-actual/src/capture/joiner.rs, crates/listener-core/src/actors/source/pipeline.rs

These have explicit bounds:

• Joiner: MAX_QUEUE_SIZE = 30 with overflow pop
• AudioBuffer (pipeline): MAX_BUFFER_CHUNKS = 150 with overflow pop
• Audio data uses Arc<[f32]> for shared ownership — proper cleanup depends on all Arc references being dropped

These are unlikely to cause 96GB growth given the bounds, but if a downstream consumer (e.g., the listener actor) stops processing while the source keeps producing, Arc references could accumulate.

3. In-Memory Recording (Bounded)

Files: crates/listener-core/src/actors/recorder/memory/retained_audio.rs, crates/listener-core/src/actors/recorder/memory/mod.rs

• RetainedAudio uses KeepLast(Duration::from_secs(3600)) (1 hour) retention with proper enforcement in enforce_retention().
• This is bounded and should not exceed ~500MB for 1 hour of encoded audio.

4. LiveTranscriptEngine (Unbounded Growth Pattern)

File: crates/listener-core/src/live_transcript.rs

The RenderedSegmentState maintains:

• words: BTreeMap<String, FinalizedWord> — grows with every finalized word, never trimmed during a session
• segments: BTreeMap<String, LiveTranscriptSegment> — rebuilt on every delta

For very long sessions (hours), the words map will grow continuously. While unlikely to reach 96GB alone, it's an unbounded accumulation pattern.

5. Actor System Lifecycle

Files: plugins/local-stt/src/server/supervisor.rs, crates/listener-core/src/actors/session/supervisor.rs

• STT supervisor has max_restarts: 100 within a 3-minute window. If the cactus server crashes and restarts repeatedly, each restart could load a new model instance into memory while the old one is still being cleaned up.
• The RestartPolicy::Transient with 500ms backoff means rapid restarts are possible.
• The session supervisor tracks source_restarts and recorder_restarts — these could accumulate if actors keep failing.

6. Recent Changes Near Build SHA

Three commits touched critical memory-related code shortly before the build:

1. 572f66415 — "various local server refactor": Refactored Internal2STTActor (Cactus server) to use TranscribeService::into_router(). Changed health check to make HTTP requests. This is the most relevant change.
2. 2f4233b2a — "add model-manager in transcribe-cactus": Introduced ModelManager with Arc<M> model caching and inactivity-based eviction. A new shared model lifecycle layer.
3. c34e6dc4e — "better model download interface": Model download changes.

The model-manager introduction (2f4233b2a) is notable — it added a new layer of Arc<Model> management. The ModelManager::get() method stores Arc::clone(&model) in active and also returns a clone to the caller. If the TranscribeService holds onto model references beyond what the manager tracks, models may not be evicted as expected.

---

Suggested Investigation Approaches

1. Profile cactus-sys native memory: The most likely culprit is the C++ layer. Use leaks, vmmap, or Instruments on macOS to check if native allocations from the cactus framework are growing unbounded during streaming sessions.

2. Check for leaked TranscriptionSession / Transcriber instances: Add logging or metrics around Transcriber::drop and TranscriptionSession::drop to verify they're being called when sessions end.

3. Monitor Arc<Model> strong count: Log Arc::strong_count on the cactus model to see if references accumulate over time. If the count grows beyond 2-3, something is holding onto model references.

4. Check actor restart patterns: If the STT server actor is crash-looping, each restart loads the model. Look at logs for "starting internal2 STT server" frequency.

5. Reproduce with Instruments: Since this is macOS/M5 specific, running with Instruments' Allocations tool would pinpoint the exact allocation site. The cactus_stream_transcribe_* FFI calls would be the first place to look.

6. Long session transcript accumulation: For users with long listening sessions, the RenderedSegmentState.words BTreeMap grows without bound. Consider adding a sliding window or periodic compaction.

[yujonglee]

---

• This should not happen, if using the new Recommended Section, instead of Deprecated section.
• Will close this, once we fully remove the Deprecated inference engine.

[migavel508]

hi @yujonglee , me also facing the same it crashes down and UI not even respond anything and I can't be able to click anything , once when the meeting is going too long transcription pipeline also breaks