Skip to content
Open
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
132 changes: 132 additions & 0 deletions docs/source/backends-qualcomm.md
Original file line number Diff line number Diff line change
Expand Up @@ -288,6 +288,138 @@ After the above command, pre-processed inputs and outputs are put in `$EXECUTORC
The command-line arguments are written in [utils.py](https://github.com/pytorch/executorch/blob/main/examples/qualcomm/utils.py#L139).
The model, inputs, and output location are passed to `qnn_executorch_runner` by `--model_path`, `--input_list_path`, and `--output_folder_path`.

### Run [Android LlamaDemo](https://github.com/meta-pytorch/executorch-examples/tree/main/llm/android/LlamaDemo) with QNN backend

Comment on lines +291 to +292
Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The external link points to 'meta-pytorch/executorch-examples' but the PR is in the 'pytorch/executorch' repository. Verify this is the correct repository reference for the LlamaDemo, as it may confuse users if the link is incorrect or if the demo has moved.

Suggested change
### Run [Android LlamaDemo](https://github.com/meta-pytorch/executorch-examples/tree/main/llm/android/LlamaDemo) with QNN backend
### Run [Android LlamaDemo](https://github.com/pytorch/executorch/tree/main/examples/llm/android/LlamaDemo) with QNN backend
> **Note:** If the LlamaDemo is not present in the `pytorch/executorch` repository, you can find it in the [meta-pytorch/executorch-examples](https://github.com/meta-pytorch/executorch-examples/tree/main/llm/android/LlamaDemo) repository.

Copilot uses AI. Check for mistakes.
`$DEMO_APP` refers to the root of the executorch android demo, i.e., the directory containing `build.gradle.kts`.

***Step 1***: Rebuild ExecuTorch AAR

```bash
# Build the AAR
cd $EXECUTORCH_ROOT
export BUILD_AAR_DIR=$EXECUTORCH_ROOT/aar-out
./scripts/build_android_library.sh
Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script invocation lacks context about required environment variables or prerequisites. Consider adding a note about any required environment setup (e.g., Android NDK path, QNN SDK path) before running this script, or reference existing documentation that covers these prerequisites.

Copilot uses AI. Check for mistakes.
```

***Step 2***: Copy AAR to Android Project

```bash
cp $EXECUTORCH_ROOT/aar-out/executorch.aar \
$DEMO_APP/app/libs/executorch.aar
```

***Step 3***: Build Android APK

```bash
cd $DEMO_APP
./gradlew clean assembleDebug -PuseLocalAar=true
```

***Step 4***: Install on Device

```bash
adb install -r app/build/outputs/apk/debug/app-debug.apk
```

***Step 5***: Push model

```bash
adb shell mkdir -p /data/local/tmp/llama
adb push model.pte /data/local/tmp/llama
adb push tokenizer.bin /data/local/tmp/llama
Comment on lines +328 to +329
Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The documentation doesn't specify where to obtain 'model.pte' and 'tokenizer.bin'. Add a reference to instructions for generating or downloading these required files, especially since the QNN backend requires specific model compilation steps.

Copilot uses AI. Check for mistakes.
```

***Step 6***: Run the Llama Demo

- Open the App on Android
- Select `QUALCOMM` backend
- Select `model.pte` Model
- Select `tokenizer.bin` Tokenizer
- Select Model Type
- Click LOAD MODEL
- It should show `Successfully loaded model.`


#### Verification Steps

***Step 1***. Verify AAR Contains Your Changes

```bash
# Check for debug strings in the AAR
unzip -p $DEMO_APP/app/libs/executorch.aar jni/arm64-v8a/libexecutorch.so | \
strings | grep "QNN" # Replace "QNN" with your actual debug string if needed
```

If found, your changes are in the AAR.

***Step 2***. Verify APK Contains Correct Libraries

```bash
# Check QNN library version in APK
cd $DEMO_APP
unzip -l app/build/outputs/apk/debug/app-debug.apk | grep "libQnnHtp.so"
```

Expected size for QNN 2.37.0: ~2,465,440 bytes
Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The expected file size is overly specific and may vary slightly between builds or minor version updates. Consider changing this to a range (e.g., '~2.4-2.5 MB') to make the verification more resilient to minor variations.

Suggested change
Expected size for QNN 2.37.0: ~2,465,440 bytes
Expected size for QNN 2.37.0: ~2.4–2.5 MB (2,400,000–2,500,000 bytes)

Copilot uses AI. Check for mistakes.

***Step 3***. Monitor Logs During Model Loading

```bash
adb logcat -c
adb logcat | grep -E "ExecuTorch"
```

#### Common Issues and Solutions

##### Issue 1: Error 18 (InvalidArgument)

**Cause**: Wrong parameter order in Runner constructor or missing QNN config

**Solution**: Check `$EXECUTORCH_ROOT/examples/qualcomm/oss_scripts/llama/runner/runner.h` for the correct constructor signature.

##### Issue 2: Error 1 (Internal) with QNN API Version Mismatch

**Symptoms**:
```
W [Qnn ExecuTorch]: Qnn API version 2.33.0 is mismatched
E [Qnn ExecuTorch]: Using newer context binary on old SDK
E [Qnn ExecuTorch]: Can't create context from binary. Error 5000
```

**Cause**: Model compiled with QNN SDK version X but APK uses QNN runtime version Y

**Solution**:
1. Update `build.gradle.kts` with matching QNN runtime version

Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The note appears after the 'Before' label but should appear before showing any code examples. Moving it above line 395 would make the documentation flow more clearly and prevent confusion about which versions to use.

Suggested change

Copilot uses AI. Check for mistakes.
> **Note:** The version numbers below (`2.33.0` and `2.37.0`) are examples only. Please check for the latest compatible QNN runtime version or match your QNN SDK version to avoid API mismatches.
Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

[nitpick] The note about version numbers being examples appears after mentioning 'version X' and 'version Y' in the previous text but before showing the actual version numbers. This note would be more effective if placed immediately after the code examples (after line 403) where the specific versions are shown, or reference should be made to it in the text above where X and Y are mentioned.

Copilot uses AI. Check for mistakes.
**Before**:
```kotlin
implementation("com.qualcomm.qti:qnn-runtime:2.33.0")
```

**After**:
```kotlin
implementation("com.qualcomm.qti:qnn-runtime:2.37.0")
```
Comment on lines +394 to +403
Copy link

Copilot AI Dec 1, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The numbered list item '1.' is followed by a note and code examples, but there's no item '2.' at the same indentation level until line 405. The note and code examples should be indented or formatted as sub-items of item 1 to maintain proper list structure.

Suggested change
> **Note:** The version numbers below (`2.33.0` and `2.37.0`) are examples only. Please check for the latest compatible QNN runtime version or match your QNN SDK version to avoid API mismatches.
**Before**:
```kotlin
implementation("com.qualcomm.qti:qnn-runtime:2.33.0")
```
**After**:
```kotlin
implementation("com.qualcomm.qti:qnn-runtime:2.37.0")
```
> **Note:** The version numbers below (`2.33.0` and `2.37.0`) are examples only. Please check for the latest compatible QNN runtime version or match your QNN SDK version to avoid API mismatches.
**Before**:
```kotlin
implementation("com.qualcomm.qti:qnn-runtime:2.33.0")
```
**After**:
```kotlin
implementation("com.qualcomm.qti:qnn-runtime:2.37.0")
```

Copilot uses AI. Check for mistakes.

2. Or recompile model with matching QNN SDK version

##### Issue 3: Native Code Changes Not Applied

**Symptoms**: Debug logs don't appear, behavior doesn't change

**Cause**: Gradle using Maven dependency instead of local AAR

**Solution**: Always build with `-PuseLocalAar=true` flag

##### Issue 4: Logs Not Appearing

**Cause**: Wrong logging tag filter

**Solution**: QNN uses "ExecuTorch" tag:
```bash
adb logcat | grep "ExecuTorch"
```

## Supported model list

Expand Down