Skip to content

Commit 29f233a

Browse files
committed
Add Run Android LlamaDemo with QNN backend
1 parent 12d17ef commit 29f233a

File tree

1 file changed

+131
-0
lines changed

1 file changed

+131
-0
lines changed

docs/source/backends-qualcomm.md

Lines changed: 131 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -288,6 +288,137 @@ After the above command, pre-processed inputs and outputs are put in `$EXECUTORC
288288
The command-line arguments are written in [utils.py](https://github.com/pytorch/executorch/blob/main/examples/qualcomm/utils.py#L139).
289289
The model, inputs, and output location are passed to `qnn_executorch_runner` by `--model_path`, `--input_list_path`, and `--output_folder_path`.
290290

291+
### Run [Android LlamaDemo](https://github.com/meta-pytorch/executorch-examples/tree/main/llm/android/LlamaDemo) with QNN backend
292+
293+
`$DEMO_APP` refers to the root of executorch android demo root, i.e., the directory containing `build.gradle.kts`.
294+
295+
***Step 1***: Rebuild ExecuTorch AAR
296+
297+
```bash
298+
# Build the AAR
299+
cd $EXECUTORCH_ROOT
300+
export BUILD_AAR_DIR=$EXECUTORCH_ROOT/aar-out
301+
./scripts/build_android_library.sh
302+
```
303+
304+
***Step 2***: Copy AAR to Android Project
305+
306+
```bash
307+
cp $EXECUTORCH_ROOT/aar-out/executorch.aar \
308+
$DEMO_APP/app/libs/executorch.aar
309+
```
310+
311+
***Step 3***: Build Android APK
312+
313+
```bash
314+
cd $DEMO_APP
315+
./gradlew clean assembleDebug -PuseLocalAar=true
316+
```
317+
318+
***Step 4***: Install on Device
319+
320+
```bash
321+
adb install -r app/build/outputs/apk/debug/app-debug.apk
322+
```
323+
324+
***Step 5***: Push model
325+
326+
```bash
327+
adb shell mkdir -p /data/local/tmp/llama
328+
adb push model.pte /data/local/tmp/llama
329+
adb push tokenizer.bin /data/local/tmp/llama
330+
```
331+
332+
***Step 5***: Run the Llama Demo
333+
334+
- Open the App on Android
335+
- Select `QUALCOMM` backend
336+
- Select `model.pte` Model
337+
- Select `tokenizer.bin` Tokenizer
338+
- Select Model Type
339+
- Click LOAD MODEL
340+
- It should show `Successfully loaded model.`
341+
342+
343+
#### Verification Steps
344+
345+
***Step 1***. Verify AAR Contains Your Changes
346+
347+
```bash
348+
# Check for debug strings in the AAR
349+
unzip -p $DEMO_APP/app/libs/executorch.aar jni/arm64-v8a/libexecutorch.so | \
350+
strings | grep "YOUR DEBUG INFO"
351+
```
352+
353+
If found, your changes are in the AAR.
354+
355+
***Step 2***. Verify APK Contains Correct Libraries
356+
357+
```bash
358+
# Check QNN library version in APK
359+
cd $DEMO_APP
360+
unzip -l app/build/outputs/apk/debug/app-debug.apk | grep "libQnnHtp.so"
361+
```
362+
363+
Expected size for QNN 2.37.0: ~2,465,440 bytes
364+
365+
***Step 3***. Monitor Logs During Model Loading
366+
367+
```bash
368+
adb logcat -c
369+
adb logcat | grep -E "ExecuTorch"
370+
```
371+
372+
#### Common Issues and Solutions
373+
374+
##### Issue 1: Error 18 (InvalidArgument)
375+
376+
**Cause**: Wrong parameter order in Runner constructor or missing QNN config
377+
378+
**Solution**: Check `$EXECUTORCH_ROOT/examples/qualcomm/oss_scripts/llama/runner/runner.h` for the correct constructor signature.
379+
380+
##### Issue 2: Error 1 (Internal) with QNN API Version Mismatch
381+
382+
**Symptoms**:
383+
```
384+
W [Qnn ExecuTorch]: Qnn API version X.XX.X is mismatched
385+
E [Qnn ExecuTorch]: Using newer context binary on old SDK
386+
E [Qnn ExecuTorch]: Can't create context from binary. Error 5000
387+
```
388+
389+
**Cause**: Model compiled with QNN SDK version X but APK uses QNN runtime version Y
390+
391+
**Solution**:
392+
1. Update `build.gradle.kts` with matching QNN runtime version
393+
394+
**Before**:
395+
```kotlin
396+
implementation("com.qualcomm.qti:qnn-runtime:2.33.0")
397+
```
398+
399+
**After**:
400+
```kotlin
401+
implementation("com.qualcomm.qti:qnn-runtime:2.37.0")
402+
```
403+
404+
2. Or recompile model with matching QNN SDK version
405+
406+
##### Issue 3: Native Code Changes Not Applied
407+
408+
**Symptoms**: Debug logs don't appear, behavior doesn't change
409+
410+
**Cause**: Gradle using Maven dependency instead of local AAR
411+
412+
**Solution**: Always build with `-PuseLocalAar=true` flag
413+
414+
##### Issue 4: Logs Not Appearing
415+
416+
**Cause**: Wrong logging tag filter
417+
418+
**Solution**: QNN uses "ExecuTorch" tag:
419+
```bash
420+
adb logcat | grep "ExecuTorch"
421+
```
291422

292423
## Supported model list
293424

0 commit comments

Comments
 (0)