Context
The current Android build uses a "Logic Engine" (mock) because ort (ONNX Runtime) failed to compile for aarch64-linux-android due to missing precompiled binaries/pkg-config issues.
Task
Enable real local inference on Android.
Options
- Fix ORT: Configure
ort to download/compile correctly for Android NDK.
- Switch to Candle: Use Hugging Face's
candle crate which is pure Rust and easier to compile for Android.
- Switch to Llama.cpp: Use
llama.cpp bindings.
Requirements
- Must run
Phi-3-mini or Gemma-2b (quantized).
- Must not crash the app on load.
- Must stream tokens to the UI.
Reference
crates/synapse-cognition/src/lib.rs
apps/desktop/src-tauri/Cargo.toml