Qualcomm AI Engine Direct - Improve GA Static Phi-4-mini accuracy

[shewu-quic]

Summary:

• Refactor custom annotation for R3
• Fix warning message in quantization
• Add phi-4-mini setting into README
• Fixed segmemtation fault when run the model with sharding
• Add a test case for phi-4 in test_qnn_delegate.py
• Add new parameter "group_size" in llama.py to set block size in block quantization

Sample Script

python examples/qualcomm/oss_scripts/llama/llama.py -b build-android -s ${SERIAL_NUM} -m ${SOC_MODEL} \ 
--ptq 16a4w_block --group_size 16 --checkpoint consolidated.00.pth --params params.json --num_sharding 4 \
--tokenizer_model tokenizer.model --decoder_model phi_4_mini --model_mode hybrid --prefill_ar_len 128 \ 
--max_seq_len 1024 --prompt "I would like to learn python, could you teach me with a simple example?"

Result

Stats with QNN2.37.0 on SM8750
Accuracy: 10.82
Token Rate: 22.727273
Results:
--prompt "I would like to learn python, could you teach me with a simple example?"

<|user|>I would like to learn python, could you teach me with one simple program?<|end|><|assistant|>Of course! Let's get started with a simple Python program. We'll create a simple program that asks for your name and then greets you.

```python
# Ask for the user's name
name = input("Please enter your name: ")

# Greet the user
print(f"Hello, {name}! Welcome to the world of Python.")

To run this program, you would need to copy the code into a Python environment (like an IDE or a Python interpreter). When you run the program, it will prompt you to enter your name, and then it will greet you by name. Enjoy learning Python!<|end|>

## Test plan
Added E2E test to test_qnn_delegate.py

cc: @haowhsu-quic 

[pytorch-bot]

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/13573

• 📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 2 New Failures

As of commit fc31b75 with merge base 3dac421 ():

NEW FAILURES - The following jobs have failed:

• pull / unittest-arm-backend-with-no-fvp (test_pytest_models) / linux-job (gh)
  RuntimeError: Command docker exec -t 1b33f25ec56755a5199d139214a3f850aadbd9082fcd31ee1ab028ceb3fc3b15 /exec failed with exit code 1
• pull / unittest-arm-backend-with-no-fvp (test_pytest_ops) / linux-job (gh)
  RuntimeError: Command docker exec -t 03f709aa965df3db73e16306e7b725bf13696ad6074d65ab2da5097c59565411 /exec failed with exit code 1

This comment was automatically generated by Dr. CI and updates every 15 minutes.

[github-actions]

This PR needs a release notes: label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

[shewu-quic]

Hi @cccclai ,
This PR is to add phi-4-mini setting into README and fix some minor issue.
Could you please help to take a look? Thanks!

[facebook-github-bot]

@cccclai has imported this pull request. If you are a Meta employee, you can view this in D80711106.

[cccclai]

Looks good, thank you!