[Maca] Fix qwenvl and internvl using mcoplib #279
Conversation
There was a problem hiding this comment.
Pull request overview
Copilot reviewed 3 out of 6 changed files in this pull request and generated 2 comments.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
CMakeLists.txt
Outdated
| add_subdirectory(dlinfer/vendor/${DEVICE}) | ||
| add_subdirectory(dlinfer/graph/dicp/vendor) | ||
|
|
||
| install(CODE "message(STATUS \"Install completed for device: ${DEVICE}\")") |
There was a problem hiding this comment.
Trailing whitespace at end of line. Remove the trailing spaces after the closing parenthesis and double quote.
| install(CODE "message(STATUS \"Install completed for device: ${DEVICE}\")") | |
| install(CODE "message(STATUS \"Install completed for device: ${DEVICE}\")") |
| import dlinfer.framework.transformers_ext | ||
| import dlinfer.framework.torch_npu_ext | ||
|
|
||
| # import dlinfer.framework.torch_npu_ext |
There was a problem hiding this comment.
Commented out import without explanation. The import of torch_npu_ext has been commented out. If this is intentional for this PR (e.g., MACA-specific changes that don't need NPU support), consider removing the line entirely or adding a comment explaining why it's disabled. If it's temporary, add a TODO comment.
| # import dlinfer.framework.torch_npu_ext |
| @@ -1,6 +1,6 @@ | |||
| # Copyright (c) 2024, DeepLink. All rights reserved. | |||
| import dlinfer.framework.transformers_ext | |||
| import dlinfer.framework.torch_npu_ext | |||
There was a problem hiding this comment.
目前main分支这一行应该不会影响maca，可以rebase main后试一下
There was a problem hiding this comment.
经测试，在最新main分支上，不会影响maca
dlinfer/vendor/maca/fused_moe.py
Outdated
| maca_ext_ops.moe_align_block_size( | ||
| topk_ids, num_experts, block_size, sorted_ids, expert_ids, num_tokens_post_pad | ||
| ) | ||
| if USE_MCOPLIB_OPS: |
There was a problem hiding this comment.
no need this "if", USE_MCOPLIB_OPS is a MUST
3 participants
feature adapter for mcoplib, it is a metax office ops library
support model list:
Qwen3 8B
Qwen3 32B
Qwen3 30BA3B(only tp)
Qwen2.5VL 8B
Qwen3VL 8B
InternVL2.5 8B
InternVL3.5 8B
InternVL3.5 38B(only tp)
InternS1 mini
multi round chat and multi batch is ok