I used qwen model of you hgingface with 85.3 pdms in title, but I only get 69 in my tests. Did I mistake any inference parameters? #16
Description
results:
12146,average,True,0.9244607278116252,0.8140128437345628,0.6422945337259997,0.853779021900214,1.0,0.9092293759262309,0.6900572286162361
`
QWEN_HUB=./DriveVLA-W0/logs/train_qwen_vla_ross_dual_vit
OUTPUT_DIR=./DriveVLA-W0/logs/train_qwen_vla_ross_dual_vit/json_output_trainedvlm_frozenrawvit
TEST_PKL=./pretrained_models/DriveVLA-W0/navsim_emu_vla_256_144_test_pre_1s.pkl
ACTION_TOKENIZER=./pretrained_models/fast
TOKEN_YAML=./DriveVLA-W0/inference/navsim/navsim/navsim/planning/script/config/common/train_test_split/scene_filter/navtest.yaml
NORM_STATS=./DriveVLA-W0/configs/normalizer_navsim_test/norm_stats.json
Inference knobs (modify these directly as needed)
CUR_IDX=3
FUTURE_NUMS=8
ACTION_DIM=3
MODEL_MAX_LEN=1400
USE_PREV=1
BF16=1
SAVE_GT=1
Optional path remap for absolute paths inside dataset, our pkl has abs path, so no need to replace. if pkl has correct rel path, no need to replace.
PATH_REPLACE_FROM=''
PATH_REPLACE_TO=''
Optional camera subdir to insert before image basename
IMAGE_CAM_SUBDIR=CAM_F0
RAW_IMG_ROOT='./openscene-v1.1_navsim/dataset/sensor_blobs/test'
mkdir -p "${OUTPUT_DIR}"
CMD=(
torchrun --nproc_per_node="${GPUS}" inference/qwen/inference_action_navsim_vava.py
--qwen_hub "${QWEN_HUB}"
--output_dir "${OUTPUT_DIR}"
--test_data_pkl "${TEST_PKL}"
--action_tokenizer_path "${ACTION_TOKENIZER}"
--token_yaml "${TOKEN_YAML}"
--norm_stats_path "${NORM_STATS}"
--cur_frame_idx "${CUR_IDX}"
--future_nums "${FUTURE_NUMS}"
--action_dim "${ACTION_DIM}"
--model_max_length "${MODEL_MAX_LEN}"
--path_replace_from "${PATH_REPLACE_FROM}"
--path_replace_to "${PATH_REPLACE_TO}"
--image_cam_subdir "${IMAGE_CAM_SUBDIR}"
--raw_img_root "${RAW_IMG_ROOT}"
)
if [[ "${USE_PREV}" == "1" ]]; then
CMD+=(--use_previous_actions)
fi
if [[ "${BF16}" == "1" ]]; then
CMD+=(--bf16)
fi
if [[ "${SAVE_GT}" == "1" ]]; then
CMD+=(--save_gt)
fi
echo "Running: ${CMD[*]}"
"${CMD[@]}"
`
I used inference_2va.sh with navsim_emu_vla_256_144_test_pre_1s.pkl