[Bug] mindIE2.3.0版本测试textvqa报错 #135
Description
操作系统及版本
openEuler 24.03 (LTS)
安装工具的python环境
docker容器中的python环境
python版本
3.11
AISBench工具版本
3.1.20260119
AISBench执行命令
ais_bench --models mindie_stream_api_general --datasets textvqa_gen --mode perf --debug --num-prompts 4
模型配置文件或自定义配置文件内容
from ais_bench.benchmark.models import MindieStreamApi
MULTI_FLAG = True
#PATH = "/home/model/Qwen2.5-14B-Instruct"
PATH = "/home/model/Qwen2.5-VL-7B-Instruct"
MODEL_NAME = "qwen"
HOST_IP = "localhost"
HOST_PORT = 1025
models = [
dict(
attr="service",
type=MindieStreamApi,
abbr="mindie-stream-api",
path=PATH,
request_rate=0,
use_timestamp=False,
stream=True,
retry=2,
host_ip=HOST_IP,
host_port=HOST_PORT,
url="",
max_out_len=512,
batch_size=4,
trust_remote_code=False,
generation_kwargs=dict(
temperature=0.01,
ignore_eos=False,
),
)
]
#textvqa_gen.py
from ais_bench.benchmark.openicl.icl_prompt_template.icl_prompt_template_mm import MMPromptTemplate
from ais_bench.benchmark.openicl.icl_retriever import ZeroRetriever
from ais_bench.benchmark.openicl.icl_inferencer import GenInferencer
from ais_bench.benchmark.datasets import TEXTVQADataset, TEXTEvaluator
textvqa_reader_cfg = dict(
input_columns=['question', 'image'],
output_column='answer'
)
textvqa_infer_cfg = dict(
prompt_template=dict(
type=MMPromptTemplate,
template=dict(
round=[
dict(role="HUMAN", prompt_mm={
"text": {"type": "text", "text": "{question} Answer the question using a single word or phrase."},
"image": {"type": "image_url", "image_url": {"url": "file://{image}"}},
})
]
)
),
retriever=dict(type=ZeroRetriever),
inferencer=dict(type=GenInferencer)
)
textvqa_eval_cfg = dict(
evaluator=dict(type=TEXTEvaluator)
)
textvqa_datasets = [
dict(
abbr='textvqa',
type=TEXTVQADataset,
# path='ais_bench/datasets/textvqa/textvqa_json/textvqa_val.jsonl', # 数据集路径，使用相对路径时相对于源码根路径，支持绝对路径
path='/home/datasets/textvqa/textvqa_json/textvqa_val.jsonl',
image_type="image_path",
reader_cfg=textvqa_reader_cfg,
infer_cfg=textvqa_infer_cfg,
eval_cfg=textvqa_eval_cfg
)
]
预期行为
success
实际行为
[2026-02-05 11:42:42,634] [ais_bench] [INFO] Loading mindie_stream_api_general: /home/l00838764/benchmarkgithub/ais_bench/benchmark/configs/./models/mindie_api/mindie_stream_api_general.py
[2026-02-05 11:42:42,642] [ais_bench] [INFO] Loading textvqa_gen: /home/l00838764/benchmarkgithub/ais_bench/benchmark/configs/./datasets/textvqa/textvqa_gen.py
[2026-02-05 11:42:42,646] [ais_bench] [INFO] Loading example: /home/l00838764/benchmarkgithub/ais_bench/benchmark/configs/./summarizers/example.py
[2026-02-05 11:42:42,696] [ais_bench] [INFO] Current exp folder: outputs/default/20260205_114227
[2026-02-05 11:42:42,696] [ais_bench] [INFO] Keeping the first 4 prompts for dataset [textvqa]
[2026-02-05 11:42:42,709] [ais_bench] [INFO] Starting inference tasks...
[2026-02-05 11:42:42,710] [ais_bench] [WARNING] Multimodal dataset 'ais_bench.benchmark.datasets.TEXTVQADataset' can only be used for VLLMCustomAPIChat API, but API='ais_bench.benchmark.models.MindieStreamApi', so this task is ignored!
Traceback (most recent call last):
File "/usr/local/bin/ais_bench", line 8, in
sys.exit(main())
^^^^^^
File "/homebenchmarkgithub/ais_bench/benchmark/cli/main.py", line 6, in main
task_manager.run()
File "/home/benchmarkgithub/ais_bench/benchmark/cli/task_manager.py", line 39, in run
workflow_executor.execute()
File "/home//benchmarkgithub/ais_bench/benchmark/cli/workers.py", line 252, in execute
worker.do_work(self.cfg)
File "/home//benchmarkgithub/ais_bench/benchmark/cli/workers.py", line 66, in do_work
tasks = partitioner(cfg)
^^^^^^^^^^^^^^^^
File "/home/l/benchmarkgithub/ais_bench/benchmark/partitioners/base.py", line 92, in call
tasks = self._check_task_cfg(tasks)
^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/l/benchmarkgithub/ais_bench/benchmark/partitioners/base.py", line 129, in _check_task_cfg
raise ValueError("No executable task found; please check the configuration!")
ValueError: No executable task found; please check the configuration!
[ERROR] 2026-02-05-11:42:42 (PID:24633, Device:-1, RankID:-1) ERR99999 UNKNOWN application exception
前置检查
- 我已读懂主页文档的快速入门，无法解决问题
- 我已检索过FAQ，无重复问题
- 我已搜索过现有Issue，无重复问题
- 我已更新到最新版本，问题仍存在