Stable Diffusion WebUI Forge - Nunchaku

⚠️ This project is in beta testing and is not ready for general users.

⚠️ This project requires Python 3.13 exclusively.

About

This program is a fork that integrates Nunchaku support into Stable Diffusion WebUI Forge, including LoRA and ControlNet support for Nunchaku models.

Features

🎯 Key Features

• LoRA Support for Nunchaku Qwen Image and Z-Image

  • ✅ Full LoRA support for Nunchaku Qwen Image (QI) models
  • ✅ Full LoRA support for Nunchaku Z-Image (ZIT) models
  • Completely separated implementations for Qwen Image and Z-Image
  • Support for standard LoRA formats (lora_A/lora_B, lora_up/lora_down)
  • Comprehensive logging with format detection for all LoRAs
  • Robust change detection to handle model reloads correctly
  • AWQ quantization layer handling with safety switch

• Union ControlNet for Flux1, Nunchaku Flux1, and Nunchaku Qwen Image

  • ✅ Union ControlNet support for Flux1 and Nunchaku Flux1 models
  • ✅ Union ControlNet support for Nunchaku Qwen Image (QI) models
  • Multiple ControlNet models can be used simultaneously (Union ControlNet)
  • Supports Flux Union ControlNet models:
    • flux_shakker_labs_union_pro-2-fp8.safetensors
  • Supports Qwen Image Union ControlNet models:
    • Qwen-Image-InstantX-ControlNet-Union.safetensors (InstantX)
    • Qwen-Image-2512-Fun-Controlnet-Union-2602.safetensors (Fun)
  • Automatic model detection and loading:
    • Flux models: via controlnet_x_embedder.weight key detection
    • Qwen Image models:
      • InstantX: via transformer_blocks.0.img_mlp.net.0.proj.weight key detection
      • Fun: via control_img_in.weight key detection
  • VAE wrapper for seamless Forge VAE integration with ComfyUI ControlNet interface
  • Strict model type checking to ensure compatibility with correct model types
  • Complete and independent implementation for each model type
  

  Flux1 Union ControlNet workflow example

• Diffsynth Union ControlNet for Z-Image Turbo (ZIT) - Standard and Nunchaku

  • ✅ Diffsynth Union ControlNet support for standard Z-Image Turbo (ZIT) and Nunchaku Z-Image Turbo (ZIT) models
  • Multiple ControlNet models can be used simultaneously (Union ControlNet)
  • Supports ZIT ControlNet models (e.g., z-image-turbo-controlnet.safetensors)
  • Note: ZIT Diffsynth ControlNet uses a different mechanism than standard ControlNet
  • ZIT ControlNet works as a "model patch" rather than a traditional ControlNet
  • Automatic model detection for ZIT models via NextDiT model type
  • Strict model type checking to ensure compatibility only with ZIT models
  • VAE wrapper for seamless Forge VAE integration with ComfyUI ControlNet interface
  • Complete implementation based on ComfyUI's nodes_model_patch.py
  • Fixed double patching and stale patches issues causing RecursionError

• Built-in ADetailer

  • Python 3.13 compatible face detection and enhancement
  • YOLOv8, YOLOv11, and InsightFace hybrid detection system
  • Enhanced face detection accuracy with complementary detection
  • Automatic model download and management
  • SDXL/Pony optimized detection thresholds

• Python 3.13 Exclusive

  • Latest Python features and performance improvements
  • Optimized for modern hardware and workflows
  • Future-proof architecture

• RES4LYF Sampler Support

  • ✅ Full support for RES4LYF (RES4) samplers
  • Support for all model types including Nunchaku Qwen Image, Nunchaku Flux1, Nunchaku SDXL, standard SDXL, and standard Flux1
  • Comprehensive sampler collection with multistep and exponential variants
  • ODE version support for non-implicit samplers
  • Robust model detection and handling for both Forge and ComfyUI model structures
  • Automatic CONST and EPS model type detection for proper sampling behavior

Known Limitations

LoRA Format Support for Nunchaku Models

⚠️ Important: The following limitations apply only to Nunchaku quantization models (Nunchaku Qwen Image, Nunchaku Z-Image, Nunchaku SDXL). Standard (non-quantized) models may support additional LoRA formats.

LoKR (Lycoris) LoRA Support

Status: ❌ Not Supported for Nunchaku Models

Issue: LoRAs in LoKR format (created by Lycoris) are not supported for Nunchaku quantization models.

Note: LoKR format LoRAs may work with standard (non-quantized) Qwen Image, Z-Image, or SDXL models, but this implementation is designed specifically for Nunchaku quantization models.

⚠️ Development History: We have spent considerable time analyzing the internal structure of LoKR format and performing extensive mapping tests. Despite these efforts, we have not been able to find a successful way to apply LoKR weights to Nunchaku quantization models. The experimental conversion code has been tested but ultimately disabled due to incompatibility issues.

• LoKR weights are automatically skipped when detected for Nunchaku models (experimental conversion code is disabled).
• Converting to Standard LoRA using SVD approximation (via external tools or scripts) has also been tested and found to result in noise/artifacts when applied to Nunchaku quantization models.

Conclusion: At this time, we have not found a way to successfully apply LoKR weights to Nunchaku models. Please use Standard LoRA formats for Nunchaku models.

Supported LoRA Formats for Nunchaku Models

✅ Standard LoRA (Rank-Decomposed):

Supported weight keys:

• lora_up.weight / lora_down.weight
• lora.up.weight / lora.down.weight
• lora_A.weight / lora_B.weight
• lora.A.weight / lora.B.weight

These are the standard formats produced by Kohya-ss, Diffusers, and most training scripts.

❌ LoKR (Lycoris): Not supported for Nunchaku models (Keys like lokr_w1, lokr_w2)

❌ LoHa: Not supported for Nunchaku models (Keys like hada_w1, hada_w2)

❌ IA3: Not supported for Nunchaku models

Non-Quantized Qwen Image (QI) Models

Status: ❌ LoRA and ControlNet Not Implemented

Issue: Non-quantized Qwen Image (QI) models do not have LoRA or ControlNet support implemented. The performance is too heavy for development and testing in our environment.

Note: This implementation is designed specifically for Nunchaku quantization models. For non-quantized Qwen Image models, please use alternative implementations or consider using Nunchaku quantized models instead.

Changelog

For full changelog, see CHANGELOG.md.

Base Repositories

This project is built upon the following repositories:

• stable-diffusion-webui-forge
  Original Forge implementation by @lllyasviel

• sd-webui-forge-classic (neo branch)
  Classic Forge implementation by @Haoming02

• ComfyUI
  The most powerful and modular diffusion model GUI, API and backend with a graph/nodes interface by @Comfy-Org

• ComfyUI-nunchaku-unofficial-loader
  Nunchaku integration reference implementation by @ussoewwin

• ComfyUI-QwenImageLoraLoader
  LoRA Loader & Diffsynth ControlNet Loader for Nunchaku Qwen Image & Nunchaku Z-Image Turbo by @ussoewwin

• ADetailer_without_mediapipe
  ADetailer with Python 3.13 support using InsightFace instead of MediaPipe by @ussoewwin

• Config-Presets
  Extension for Automatic1111 - configurable dropdown to change settings in txt2img and img2img tabs by @Zyin055

• stable-diffusion-webui-wd14-tagger
  Labeling extension for Web UI (DeepDanbooru etc.) by @picobyte

License

This project is licensed under AGPL-3.0 (GNU Affero General Public License v3.0).

Licenses of Base Repositories

This project is built upon the following repositories, each with their respective licenses:

• stable-diffusion-webui-forge - AGPL-3.0
• sd-webui-forge-classic (neo branch) - AGPL-3.0
• ComfyUI - GPL-3.0
• ComfyUI-nunchaku-unofficial-loader - Apache 2.0
• ComfyUI-QwenImageLoraLoader - Apache 2.0
• ADetailer_without_mediapipe - GPL-3.0
• Config-Presets - MIT
• stable-diffusion-webui-wd14-tagger - Public domain, except borrowed parts (e.g. dbimutils.py)

See LICENSE file for details.