Skip to content

Commit 91a151b

Browse files
SamYuan1990SamYuan1990
authored
continue translate document to zh (#12194)
Signed-off-by: SamYuan1990 <yy19902439@126.com>
1 parent 4fcd0bc commit 91a151b

25 files changed

+4709
-0
lines changed

‎docs/source/zh/_toctree.yml

Lines changed: 56 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -15,15 +15,49 @@
1515
- local: using-diffusers/schedulers
1616
title: Load schedulers and models
1717

18+
- title: Inference
19+
isExpanded: false
20+
sections:
21+
- local: training/distributed_inference
22+
title: Distributed inference
23+
1824
- title: Inference optimization
1925
isExpanded: false
2026
sections:
2127
- local: optimization/fp16
2228
title: Accelerate inference
29+
- local: optimization/cache
30+
title: Caching
31+
- local: optimization/memory
32+
title: Reduce memory usage
33+
- local: optimization/speed-memory-optims
34+
title: Compile and offloading quantized models
2335
- title: Community optimizations
2436
sections:
37+
- local: optimization/pruna
38+
title: Pruna
2539
- local: optimization/xformers
2640
title: xFormers
41+
- local: optimization/tome
42+
title: Token merging
43+
- local: optimization/deepcache
44+
title: DeepCache
45+
- local: optimization/tgate
46+
title: TGATE
47+
- local: optimization/xdit
48+
title: xDiT
49+
- local: optimization/para_attn
50+
title: ParaAttention
51+
52+
- title: Hybrid Inference
53+
isExpanded: false
54+
sections:
55+
- local: hybrid_inference/overview
56+
title: Overview
57+
- local: hybrid_inference/vae_encode
58+
title: VAE Encode
59+
- local: hybrid_inference/api_reference
60+
title: API Reference
2761

2862
- title: Modular Diffusers
2963
isExpanded: false
@@ -44,6 +78,10 @@
4478
title: AutoPipelineBlocks
4579
- local: modular_diffusers/modular_pipeline
4680
title: ModularPipeline
81+
- local: modular_diffusers/components_manager
82+
title: ComponentsManager
83+
- local: modular_diffusers/guiders
84+
title: Guiders
4785

4886
- title: Training
4987
isExpanded: false
@@ -56,12 +94,20 @@
5694
sections:
5795
- local: training/text2image
5896
title: Text-to-image
97+
- local: training/kandinsky
98+
title: Kandinsky 2.2
99+
- local: training/wuerstchen
100+
title: Wuerstchen
59101
- local: training/controlnet
60102
title: ControlNet
103+
- local: training/instructpix2pix
104+
title: InstructPix2Pix
61105
- title: Methods
62106
sections:
63107
- local: training/text_inversion
64108
title: Textual Inversion
109+
- local: training/dreambooth
110+
title: DreamBooth
65111
- local: training/lora
66112
title: LoRA
67113

@@ -70,6 +116,16 @@
70116
sections:
71117
- local: optimization/onnx
72118
title: ONNX
119+
- local: optimization/open_vino
120+
title: OpenVINO
121+
- local: optimization/coreml
122+
title: Core ML
123+
- local: optimization/mps
124+
title: Metal Performance Shaders (MPS)
125+
- local: optimization/habana
126+
title: Intel Gaudi
127+
- local: optimization/neuron
128+
title: AWS Neuron
73129

74130
- title: Specific pipeline examples
75131
isExpanded: false

‎docs/source/zh/hybrid_inference/api_reference.md

Lines changed: 9 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,9 @@
1+
# 混合推理 API 参考
2+
3+
## 远程解码
4+
5+
[[autodoc]] utils.remote_utils.remote_decode
6+
7+
## 远程编码
8+
9+
[[autodoc]] utils.remote_utils.remote_encode

‎docs/source/zh/hybrid_inference/overview.md

Lines changed: 55 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,55 @@
1+
<!--版权 2025 HuggingFace 团队。保留所有权利。
2+
3+
根据 Apache 许可证 2.0 版本（"许可证"）授权；除非遵守许可证，否则不得使用此文件。
4+
您可以在以下网址获取许可证副本：
5+
6+
http://www.apache.org/licenses/LICENSE-2.0
7+
8+
除非适用法律要求或书面同意，否则根据许可证分发的软件按"原样"分发，不附带任何明示或暗示的担保或条件。请参阅许可证以了解具体的语言管理权限和限制。
9+
-->
10+
11+
# 混合推理
12+
13+
**通过混合推理赋能本地 AI 构建者**
14+
15+
> [!TIP]
16+
> 混合推理是一项[实验性功能](https://huggingface.co/blog/remote_vae)。
17+
> 可以在此处提供反馈[此处](https://github.com/huggingface/diffusers/issues/new?template=remote-vae-pilot-feedback.yml)。
18+
19+
## 为什么使用混合推理？
20+
21+
混合推理提供了一种快速简单的方式来卸载本地生成需求。
22+
23+
- 🚀 **降低要求：** 无需昂贵硬件即可访问强大模型。
24+
- 💎 **无妥协：** 在不牺牲性能的情况下实现最高质量。
25+
- 💰 **成本效益高：** 它是免费的！🤑
26+
- 🎯 **多样化用例：** 与 Diffusers � 和更广泛的社区完全兼容。
27+
- 🔧 **开发者友好：** 简单请求，快速响应。
28+
29+
---
30+
31+
## 可用模型
32+
33+
* **VAE 解码 🖼️：** 快速将潜在表示解码为高质量图像，不影响性能或工作流速度。
34+
* **VAE 编码 🔢：** 高效将图像编码为潜在表示，用于生成和训练。
35+
* **文本编码器 📃（即将推出）：** 快速准确地计算提示的文本嵌入，确保流畅高质量的工作流。
36+
37+
---
38+
39+
## 集成
40+
41+
* **[SD.Next](https://github.com/vladmandic/sdnext)：** 一体化 UI，直接支持混合推理。
42+
* **[ComfyUI-HFRemoteVae](https://github.com/kijai/ComfyUI-HFRemoteVae)：** 用于混合推理的 ComfyUI 节点。
43+
44+
## 更新日志
45+
46+
- 2025 年 3 月 10 日：添加了 VAE 编码
47+
- 2025 年 3 月 2 日：初始发布，包含 VAE 解码
48+
49+
## 内容
50+
51+
文档分为三个部分：
52+
53+
* **VAE 解码** 学习如何使用混合推理进行 VAE 解码的基础知识。
54+
* **VAE 编码** 学习如何使用混合推理进行 VAE 编码的基础知识。
55+
* **API 参考** 深入了解任务特定设置和参数。

‎docs/source/zh/hybrid_inference/vae_encode.md

Lines changed: 184 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,184 @@
1+
# 入门：使用混合推理进行 VAE 编码
2+
3+
VAE 编码用于训练、图像到图像和图像到视频——将图像或视频转换为潜在表示。
4+
5+
## 内存
6+
7+
这些表格展示了在不同 GPU 上使用 SD v1 和 SD XL 进行 VAE 编码的 VRAM 需求。
8+
9+
对于这些 GPU 中的大多数，内存使用百分比决定了其他模型（文本编码器、UNet/Transformer）必须被卸载，或者必须使用分块编码，这会增加时间并影响质量。
10+
11+
<details><summary>SD v1.5</summary>
12+
13+
| GPU | 分辨率 | 时间（秒） | 内存（%） | 分块时间（秒） | 分块内存（%） |
14+
|:------------------------------|:-------------|-----------------:|-------------:|--------------------:|-------------------:|
15+
| NVIDIA GeForce RTX 4090 | 512x512 | 0.015 | 3.51901 | 0.015 | 3.51901 |
16+
| NVIDIA GeForce RTX 4090 | 256x256 | 0.004 | 1.3154 | 0.005 | 1.3154 |
17+
| NVIDIA GeForce RTX 4090 | 2048x2048 | 0.402 | 47.1852 | 0.496 | 3.51901 |
18+
| NVIDIA GeForce RTX 4090 | 1024x1024 | 0.078 | 12.2658 | 0.094 | 3.51901 |
19+
| NVIDIA GeForce RTX 4080 SUPER | 512x512 | 0.023 | 5.30105 | 0.023 | 5.30105 |
20+
| NVIDIA GeForce RTX 4080 SUPER | 256x256 | 0.006 | 1.98152 | 0.006 | 1.98152 |
21+
| NVIDIA GeForce RTX 4080 SUPER | 2048x2048 | 0.574 | 71.08 | 0.656 | 5.30105 |
22+
| NVIDIA GeForce RTX 4080 SUPER | 1024x1024 | 0.111 | 18.4772 | 0.14 | 5.30105 |
23+
| NVIDIA GeForce RTX 3090 | 512x512 | 0.032 | 3.52782 | 0.032 | 3.52782 |
24+
| NVIDIA GeForce RTX 3090 | 256x256 | 0.01 | 1.31869 | 0.009 | 1.31869 |
25+
| NVIDIA GeForce RTX 3090 | 2048x2048 | 0.742 | 47.3033 | 0.954 | 3.52782 |
26+
| NVIDIA GeForce RTX 3090 | 1024x1024 | 0.136 | 12.2965 | 0.207 | 3.52782 |
27+
| NVIDIA GeForce RTX 3080 | 512x512 | 0.036 | 8.51761 | 0.036 | 8.51761 |
28+
| NVIDIA GeForce RTX 3080 | 256x256 | 0.01 | 3.18387 | 0.01 | 3.18387 |
29+
| NVIDIA GeForce RTX 3080 | 2048x2048 | 0.863 | 86.7424 | 1.191 | 8.51761 |
30+
| NVIDIA GeForce RTX 3080 | 1024x1024 | 0.157 | 29.6888 | 0.227 | 8.51761 |
31+
| NVIDIA GeForce RTX 3070 | 512x512 | 0.051 | 10.6941 | 0.051 | 10.6941 |
32+
| NVIDIA GeForce RTX 3070 | 256x256 | 0.015 |
33+
| 3.99743 | 0.015 | 3.99743 |
34+
| NVIDIA GeForce RTX 3070 | 2048x2048 | 1.217 | 96.054 | 1.482 | 10.6941 |
35+
| NVIDIA GeForce RTX 3070 | 1024x1024 | 0.223 | 37.2751 | 0.327 | 10.6941 |
36+
37+
</details>
38+
39+
<details><summary>SDXL</summary>
40+
41+
| GPU | Resolution | Time (seconds) | Memory Consumed (%) | Tiled Time (seconds) | Tiled Memory (%) |
42+
|:------------------------------|:-------------|-----------------:|----------------------:|-----------------------:|-------------------:|
43+
| NVIDIA GeForce RTX 4090 | 512x512 | 0.029 | 4.95707 | 0.029 | 4.95707 |
44+
| NVIDIA GeForce RTX 4090 | 256x256 | 0.007 | 2.29666 | 0.007 | 2.29666 |
45+
| NVIDIA GeForce RTX 4090 | 2048x2048 | 0.873 | 66.3452 | 0.863 | 15.5649 |
46+
| NVIDIA GeForce RTX 4090 | 1024x1024 | 0.142 | 15.5479 | 0.143 | 15.5479 |
47+
| NVIDIA GeForce RTX 4080 SUPER | 512x512 | 0.044 | 7.46735 | 0.044 | 7.46735 |
48+
| NVIDIA GeForce RTX 4080 SUPER | 256x256 | 0.01 | 3.4597 | 0.01 | 3.4597 |
49+
| NVIDIA GeForce RTX 4080 SUPER | 2048x2048 | 1.317 | 87.1615 | 1.291 | 23.447 |
50+
| NVIDIA GeForce RTX 4080 SUPER | 1024x1024 | 0.213 | 23.4215 | 0.214 | 23.4215 |
51+
| NVIDIA GeForce RTX 3090 | 512x512 | 0.058 | 5.65638 | 0.058 | 5.65638 |
52+
| NVIDIA GeForce RTX 3090 | 256x256 | 0.016 | 2.45081 | 0.016 | 2.45081 |
53+
| NVIDIA GeForce RTX 3090 | 2048x2048 | 1.755 | 77.8239 | 1.614 | 18.4193 |
54+
| NVIDIA GeForce RTX 3090 | 1024x1024 | 0.265 | 18.4023 | 0.265 | 18.4023 |
55+
| NVIDIA GeForce RTX 3080 | 512x512 | 0.064 | 13.6568 | 0.064 | 13.6568 |
56+
| NVIDIA GeForce RTX 3080 | 256x256 | 0.018 | 5.91728 | 0.018 | 5.91728 |
57+
| NVIDIA GeForce RTX 3080 | 2048x2048 | 内存不足 (OOM) | 内存不足 (OOM) | 1.866 | 44.4717 |
58+
| NVIDIA GeForce RTX 3080 | 1024x1024 | 0.302 | 44.4308 | 0.302 | 44.4308 |
59+
| NVIDIA GeForce RTX 3070 | 512x512 | 0.093 | 17.1465 | 0.093 | 17.1465 |
60+
| NVIDIA GeForce R
61+
| NVIDIA GeForce RTX 3070 | 256x256 | 0.025 | 7.42931 | 0.026 | 7.42931 |
62+
| NVIDIA GeForce RTX 3070 | 2048x2048 | OOM | OOM | 2.674 | 55.8355 |
63+
| NVIDIA GeForce RTX 3070 | 1024x1024 | 0.443 | 55.7841 | 0.443 | 55.7841 |
64+
65+
</details>
66+
67+
## 可用 VAE
68+
69+
| | **端点** | **模型** |
70+
|:-:|:-----------:|:--------:|
71+
| **Stable Diffusion v1** | [https://qc6479g0aac6qwy9.us-east-1.aws.endpoints.huggingface.cloud](https://qc6479g0aac6qwy9.us-east-1.aws.endpoints.huggingface.cloud) | [`stabilityai/sd-vae-ft-mse`](https://hf.co/stabilityai/sd-vae-ft-mse) |
72+
| **Stable Diffusion XL** | [https://xjqqhmyn62rog84g.us-east-1.aws.endpoints.huggingface.cloud](https://xjqqhmyn62rog84g.us-east-1.aws.endpoints.huggingface.cloud) | [`madebyollin/sdxl-vae-fp16-fix`](https://hf.co/madebyollin/sdxl-vae-fp16-fix) |
73+
| **Flux** | [https://ptccx55jz97f9zgo.us-east-1.aws.endpoints.huggingface.cloud](https://ptccx55jz97f9zgo.us-east-1.aws.endpoints.huggingface.cloud) | [`black-forest-labs/FLUX.1-schnell`](https://hf.co/black-forest-labs/FLUX.1-schnell) |
74+
75+
76+
> [!TIP]
77+
> 模型支持可以在此处请求：[这里](https://github.com/huggingface/diffusers/issues/new?template=remote-vae-pilot-feedback.yml)。
78+
79+
80+
## 代码
81+
82+
> [!TIP]
83+
> 从 `main` 安装 `diffusers` 以运行代码：`pip install git+https://github.com/huggingface/diffusers@main`
84+
85+
86+
一个辅助方法简化了与混合推理的交互。
87+
88+
```python
89+
from diffusers.utils.remote_utils import remote_encode
90+
```
91+
92+
### 基本示例
93+
94+
让我们编码一张图像，然后解码以演示。
95+
96+
<figure class="image flex flex-col items-center justify-center text-center m-0 w-full">
97+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg"/>
98+
</figure>
99+
100+
<details><summary>代码</summary>
101+
102+
```python
103+
from diffusers.utils import load_image
104+
from diffusers.utils.remote_utils import remote_decode
105+
106+
image = load_image("https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/diffusers/astronaut.jpg?download=true")
107+
108+
latent = remote_encode(
109+
endpoint="https://ptccx55jz97f9zgo.us-east-1.aws.endpoints.huggingface.cloud/",
110+
scaling_factor=0.3611,
111+
shift_factor=0.1159,
112+
)
113+
114+
decoded = remote_decode(
115+
endpoint="https://whhx50ex1aryqvw6.us-east-1.aws.endpoints.huggingface.cloud/",
116+
tensor=latent,
117+
scaling_factor=0.3611,
118+
shift_factor=0.1159,
119+
)
120+
```
121+
122+
</details>
123+
124+
<figure class="image flex flex-col items-center justify-center text-center m-0 w-full">
125+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/remote_vae/decoded.png"/>
126+
</figure>
127+
128+
129+
### 生成
130+
131+
现在让我们看一个生成示例，我们将编码图像，生成，然后远程解码！
132+
133+
<details><summary>代码</summary>
134+
135+
```python
136+
import torch
137+
from diffusers import StableDiffusionImg2ImgPip
138+
from diffusers.utils import load_image
139+
from diffusers.utils.remote_utils import remote_decode, remote_encode
140+
141+
pipe = StableDiffusionImg2ImgPipeline.from_pretrained(
142+
"stable-diffusion-v1-5/stable-diffusion-v1-5",
143+
torch_dtype=torch.float16,
144+
variant="fp16",
145+
vae=None,
146+
).to("cuda")
147+
148+
init_image = load_image(
149+
"https://raw.githubusercontent.com/CompVis/stable-diffusion/main/assets/stable-samples/img2img/sketch-mountains-input.jpg"
150+
)
151+
init_image = init_image.resize((768, 512))
152+
153+
init_latent = remote_encode(
154+
endpoint="https://qc6479g0aac6qwy9.us-east-1.aws.endpoints.huggingface.cloud/",
155+
image=init_image,
156+
scaling_factor=0.18215,
157+
)
158+
159+
prompt = "A fantasy landscape, trending on artstation"
160+
latent = pipe(
161+
prompt=prompt,
162+
image=init_latent,
163+
strength=0.75,
164+
output_type="latent",
165+
).images
166+
167+
image = remote_decode(
168+
endpoint="https://q1bj3bpq6kzilnsu.us-east-1.aws.endpoints.huggingface.cloud/",
169+
tensor=latent,
170+
scaling_factor=0.18215,
171+
)
172+
image.save("fantasy_landscape.jpg")
173+
```
174+
175+
</details>
176+
177+
<figure class="image flex flex-col items-center justify-center text-center m-0 w-full">
178+
<img src="https://huggingface.co/datasets/huggingface/documentation-images/resolve/main/blog/remote_vae/fantasy_landscape.png"/>
179+
</figure>
180+
181+
## 集成
182+
183+
* **[SD.Next](https://github.com/vladmandic/sdnext):** 具有直接支持混合推理功能的一体化用户界面。
184+
* **[ComfyUI-HFRemoteVae](https://github.com/kijai/ComfyUI-HFRemoteVae):** 用于混合推理的 ComfyUI 节点。

0 commit comments

Comments
 (0)