Skip to content

Conversation

@yongming-qin
Copy link
Contributor

@yongming-qin yongming-qin commented Nov 29, 2025

Purpose

Introduce lazy-loaded model config registry for vllm.transformers_utils.configs, replacing the previous eager imports from many files, which is slow.
This significantly reduces import overhead and avoids loading all config submodules during startup.
Also preserves full backward compatibility with existing configs.<ConfigClass> usage.

  • Resolves: (link your issue / discussion)
  • Adds lazy loading via __getattr__ and a central _CLASS_TO_MODULE mapping
  • Handles special case: DeepseekV3Config from transformers
  • No behavior change to individual configs; only import mechanism changes

Test Plan

  1. Verified dynamic import for each config class listed in __all__
  2. Ran basic vLLM startup to ensure no breakage
  3. Loaded several models that rely on overridden configs (e.g., Deepseek-VL2, Kimi, RW/Falcon)
  4. Confirmed that undefined attributes raise correct AttributeError

Command examples:

python - <<'EOF'
from vllm.transformers_utils import configs
print(configs.AfmoeConfig)
print(configs.DeepseekV3Config)
EOF

Test Result

  • All config classes load on demand without errors
  • CI/lint pass locally
  • Import time reduced (lazy import ensures submodules are not loaded upfront)
  • No differences in model loading behavior observed

@yongming-qin
Copy link
Contributor Author

NOTE: Currently this is a draft PR. I am still completing the full test results. Feedback on the core approach is welcome.

Question for Reviewers:
What are the cases when this init.py is used? Serving a model that is customized built will always trigger this init.py, correct? If so, the lazy loading will in total save a lot of time.

Signed-off-by: yongming-qin <yq0536@gmail.com>
@yongming-qin yongming-qin force-pushed the lazy-config-loading-v2 branch from 14d53ab to 52fbcb0 Compare November 30, 2025 01:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant