Skip to content

fix(qualcomm): Qnn AOT refactor #578

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
chenghuaWang merged 8 commits into from
Jan 5, 2026
Merged

fix(qualcomm): Qnn AOT refactor #578

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
8 commits
Select commit Hold shift + click to select a range
827716a
refactor(qnn): change legacy QNN_SDK_ROOT to QAIRT_SDK_ROOT
oreomaker Dec 23, 2025
ed68fa6
Merge branch 'main' into qnn-aot
oreomaker Dec 23, 2025
e15db31
Merge branch 'main' into qnn-aot
oreomaker Dec 29, 2025
d380bfe
Merge branch 'main' into qnn-aot
oreomaker Dec 30, 2025
5174606
refactor(qnn aot): add QNN AOT support and improve QNN backend integr…
oreomaker Jan 1, 2026
3f63883
Merge branch 'main' into qnn-aot
oreomaker Jan 1, 2026
0d08209
refactor(qnn): restore QNN allocator implementation
oreomaker Jan 1, 2026
550380c
Merge branch 'main' into qnn-aot
chenghuaWang Jan 5, 2026
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
6 changes: 3 additions & 3 deletions docs/qnn_backend/setup_env.rst
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -31,7 +31,7 @@ QNN SDK Installation

1. Download the QNN SDK from the `official Qualcomm website <https://www.qualcomm.com/developer/software/qualcomm-ai-engine-direct-sdk>`_
2. Unzip the downloaded file
3. Set the environment variable ``QNN_SDK_ROOT`` to point to the unzipped directory
3. Set the environment variable ``QAIRT_SDK_ROOT`` to point to the unzipped directory

Hexagon SDK Installation
~~~~~~~~~~~~~~~~~~~~~~~~
Expand Down Expand Up @@ -68,7 +68,7 @@ After setting up the environment, verify that the following environment variable

.. code-block:: bash

echo $QNN_SDK_ROOT # Should point to /path/to/your/qnn/sdk
echo $QAIRT_SDK_ROOT # Should point to /path/to/your/qnn/sdk
echo $HEXAGON_SDK_ROOT # Should point to /path/to/your/hexagon/sdk

.. note::
Expand All @@ -84,7 +84,7 @@ Prerequisites for Compilation

Ensure the following environment variables are set:

- ``QNN_SDK_ROOT``
- ``QAIRT_SDK_ROOT``
- ``HEXAGON_SDK_ROOT``
- ``ANDROID_NDK_ROOT``

Expand Down
7 changes: 1 addition & 6 deletions examples/qwen3_qnn_aot/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
@@ -1,8 +1,3 @@
add_executable(mllm-qwen3-aot-c compile.cpp)
target_link_libraries(mllm-qwen3-aot-c PRIVATE MllmRT MllmCPUBackend)
target_link_libraries(mllm-qwen3-aot-c PRIVATE MllmRT MllmCPUBackend MllmQNNBackend)
target_include_directories(mllm-qwen3-aot-c PRIVATE ${MLLM_INCLUDE_DIR})

target_include_directories(mllm-qwen3-aot-c PRIVATE
$ENV{QAIRT_SDK_ROOT}/include # QNN SDK include
$ENV{QAIRT_SDK_ROOT}/include/QNN # QNN SDK include
)
13 changes: 4 additions & 9 deletions mllm/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -8,9 +8,6 @@ file(GLOB_RECURSE MLLM_RT_PREPROCESSOR_SRC ${CMAKE_CURRENT_LIST_DIR}/preprocesso
if(MLLM_BUILD_EXPERIMENTS)
file(GLOB_RECURSE MLLM_RT_AUTO_TUNE_SRC ${CMAKE_CURRENT_LIST_DIR}/experiments/auto_tune/*.cpp)
endif()
if(MLLM_QUALCOMM_QNN_AOT_ON_X86_ENABLE)
file(GLOB_RECURSE MLLM_QUALCOMM_AOT_SRC ${CMAKE_CURRENT_LIST_DIR}/backends/qnn/aot/*.cpp)
endif()
file(GLOB WENET_AUDIO_SOURCES ${PROJECT_SOURCE_DIR}/third_party/wenet_audio/*)

add_library(
Expand Down Expand Up @@ -117,19 +114,17 @@ if(MLLM_BUILD_OPENCL_BACKEND)
)
endif()

if(MLLM_QUALCOMM_QNN_AOT_ON_X86_ENABLE OR MLLM_BUILD_QNN_BACKEND)
add_subdirectory(backends/qnn)
endif()

if(MLLM_QUALCOMM_QNN_AOT_ON_X86_ENABLE)
# Build
target_include_directories(MllmRT PRIVATE
$ENV{QAIRT_SDK_ROOT}/include # QNN SDK include
$ENV{QAIRT_SDK_ROOT}/include/QNN # QNN SDK include
)
add_compile_definitions(
MLLM_QUALCOMM_QNN_AOT_ON_X86_ENABLE
)
endif()

if(MLLM_BUILD_QNN_BACKEND)
add_subdirectory(backends/qnn)
add_compile_definitions(
MLLM_QNN_BACKEND
)
Expand Down
12 changes: 10 additions & 2 deletions mllm/backends/qnn/CMakeLists.txt
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -13,18 +13,26 @@ file(GLOB MLLM_QNN_SRC

)

if(MLLM_QUALCOMM_QNN_AOT_ON_X86_ENABLE)
file(GLOB_RECURSE MLLM_QUALCOMM_AOT_SRC
${CMAKE_CURRENT_LIST_DIR}/aot/*.hpp
${CMAKE_CURRENT_LIST_DIR}/aot/*.cpp
)
list(APPEND MLLM_QNN_SRC ${MLLM_QUALCOMM_AOT_SRC})
endif()

add_library(
MllmQNNBackend
SHARED
${MLLM_QNN_SRC}
)

target_include_directories(MllmQNNBackend PUBLIC
$ENV{QNN_SDK_ROOT}/include/QNN # QNN SDK include
$ENV{QAIRT_SDK_ROOT}/include/QNN # QNN SDK include
Copy link
Contributor

@coderabbitai coderabbitai bot Jan 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

⚠️ Potential issue | 🟠 Major

🧩 Analysis chain

🏁 Script executed:

# First, check the CMakeLists.txt file around line 31
cat -n mllm/backends/qnn/CMakeLists.txt | head -50

Repository: UbiquitousLearning/mllm

Length of output: 1379

🏁 Script executed:

# Search for all QAIRT_SDK_ROOT usage in CMake files
rg -n "QAIRT_SDK_ROOT" --type cmake -B 2 -A 2

Repository: UbiquitousLearning/mllm

Length of output: 1040

🏁 Script executed:

# Search for validation patterns of environment variables in CMake files
rg -n "if.*NOT.*DEFINED.*ENV" --type cmake -B 1 -A 3

Repository: UbiquitousLearning/mllm

Length of output: 399

🏁 Script executed:

# Search for Makefile references mentioned in the review
fd -e Makefile | head -20

Repository: UbiquitousLearning/mllm

Length of output: 49

🏁 Script executed:

# Check for QNN_SDK_ROOT references to understand migration context
rg -n "QNN_SDK_ROOT" -B 1 -A 1

Repository: UbiquitousLearning/mllm

Length of output: 49

🏁 Script executed:

# Search for how QAIRT_SDK_ROOT is set or documented
rg -n "QAIRT_SDK_ROOT" -B 3 -A 3

Repository: UbiquitousLearning/mllm

Length of output: 4854

🏁 Script executed:

# Check if there's a custom-op-package directory and Makefile referenced in review
find . -type d -name "*custom*op*" | head -10
find . -name "Makefile" | head -20

Repository: UbiquitousLearning/mllm

Length of output: 180

🏁 Script executed:

# Check if there's any validation or setup script for QAIRT_SDK_ROOT
fd -e "\.sh$|\.md$" | xargs rg -l "QAIRT_SDK_ROOT" 2>/dev/null | head -10

Repository: UbiquitousLearning/mllm

Length of output: 243

Add validation for QAIRT_SDK_ROOT environment variable before use.

The CMakeLists.txt uses $ENV{QAIRT_SDK_ROOT} without checking if the environment variable is defined. While the associated Makefile in custom-op-package/LLaMAPackage/ validates this with error handling, the CMake build files do not. If QAIRT_SDK_ROOT is unset, the include path becomes /include/QNN, causing a confusing compile failure rather than a clear configuration error.

Add validation similar to other backends in the codebase:

if(NOT DEFINED ENV{QAIRT_SDK_ROOT})
  message(FATAL_ERROR "QAIRT_SDK_ROOT environment variable is not set. Please set it to point to the Qualcomm AI SDK root directory.")
endif()

This check should be added early in the CMakeLists.txt files (both mllm/backends/qnn/CMakeLists.txt and mllm/ffi/CMakeLists.txt) before the environment variable is referenced.

${MLLM_INCLUDE_DIR}
)

message(STATUS "QNN SDK root: $ENV{QNN_SDK_ROOT}")
message(STATUS "QNN SDK root: $ENV{QAIRT_SDK_ROOT}")

get_property(current_includes DIRECTORY ${CMAKE_CURRENT_SOURCE_DIR} PROPERTY INCLUDE_DIRECTORIES)
message(STATUS "MLLM_QNN INCLUDES: ${current_includes}") #print include directories
Expand Down
Loading