◀ 上一页
1 / 5
下一页 ▶
SD3 ControlNet Inpainting 宏观架构
从条件编码、完整 denoiser 到 VAE 解码；关键 shape 均取自 SD3-CONTROL-INPAINT-FULL 的实测记录。
1. SD3 ControlNet Inpainting 宏观架构
从条件编码、完整 denoiser 到 VAE 解码；关键 shape 均取自 SD3-CONTROL-INPAINT-FULL 的实测记录。
SD3-CONTROL-INPAINT-FULL
Prompt
token IDs
CLIP-L
OpenCLIP bigG
T5-XXL
完整文本编码器
[1, 333, 4096]
Noisy latent / tokens
batch = 1
[1, 16, 128, 128]
Inpaint Condition
16-channel latent + 1-channel mask
[1, 17, 128, 128]
SD3Transformer2DModel
17-channel condition + residual
完整生产层数，未折叠
[1, 16, 128, 128]
VAE Decoder
latent → pixels
[1, 3, 1024, 1024]
Final image
[1, 3, 1024, 1024]
ACTUAL-CUDA-FULL-TOPOLOGY
BF16 · NVIDIA A100-SXM4-40GB · native 1024² · 1 shape-equivalent denoiser timestep
条件编码与分支组装
保留所有实际执行的文本与图像条件分支；输出 shape 与 merge 方式分别标注。
2. 条件编码与分支组装
保留所有实际执行的文本与图像条件分支；输出 shape 与 merge 方式分别标注。
SD3-CONTROL-INPAINT-FULL
CLIP-L
Layers ×12
hidden 768 · heads 12
[1, 77, 768]
OpenCLIP bigG
Layers ×32
hidden 1280 · heads 20
[1, 77, 1280]
T5-XXL
Layers ×24
hidden 4096 · heads 64
[1, 256, 4096]
CLIP-L + CLIP-G + T5 merge
CLIP padded to 4096；与 T5 sequence 拼接
[1, 333, 4096]
Inpaint Condition
16-channel latent + 1-channel mask
[1, 17, 128, 128]
ACTUAL-CUDA-FULL-TOPOLOGY
BF16 · NVIDIA A100-SXM4-40GB · native 1024² · 1 shape-equivalent denoiser timestep
完整 MMDiT / MMDiT-X 拓扑
图像 token 与文本 context 在每个 joint block 中共同更新；ControlNet 分支按实际运行单列。
3. 完整 MMDiT / MMDiT-X 拓扑
图像 token 与文本 context 在每个 joint block 中共同更新；ControlNet 分支按实际运行单列。
SD3-CONTROL-INPAINT-FULL
Patch Embed
patch 2×2
4096 image tokens
[1, 16, 128, 128]
Text Context
joint dim 4096
[1, 333, 4096]
MMDiT Joint Block
hidden 1536 · heads 24×64
AdaLN modulation · joint attention · MLP
dual-attn layers: 0
完整执行 ×24
Output Projection
context split
unpatchify
[1, 16, 128, 128]
ControlNet residual branch
[1, 17, 128, 128]
完整执行 ×23
ACTUAL-CUDA-FULL-TOPOLOGY
BF16 · NVIDIA A100-SXM4-40GB · native 1024² · 1 shape-equivalent denoiser timestep
MMDiT Joint Block 与条件注入
图像与文本双流参与同一 joint attention；ControlNet residual 保持独立语义。
4. MMDiT Joint Block 与条件注入
图像与文本双流参与同一 joint attention；ControlNet residual 保持独立语义。
SD3-CONTROL-INPAINT-FULL
MMDiT joint block
AdaLN / timestep modulation
hidden 1536
Joint Attention
image QKV ↔ context QKV
Gated MLP
separate image/context updates
Residual combine
Control branch
Condition encode / patchify
[1, 17, 128, 128]
ControlNet transformer stack
blocks ×23
Per-layer residual injection
→ matching MMDiT blocks
ACTUAL-CUDA-FULL-TOPOLOGY
BF16 · NVIDIA A100-SXM4-40GB · native 1024² · 1 shape-equivalent denoiser timestep
VAE 边界与运行证据
VAE 仅把运行中真实执行的 Encode/Decode 标为 ACTUAL-CUDA；中间通道阶梯来自固定配置。
5. VAE 边界与运行证据
VAE 仅把运行中真实执行的 Encode/Decode 标为 ACTUAL-CUDA；中间通道阶梯来自固定配置。
SD3-CONTROL-INPAINT-FULL
Source / reference image
VAE Encode 已执行
[1, 3, 1024, 1024]
VAE Encoder
Down blocks ×4
[1, 16, 128, 128]
Denoised latent
actual denoiser output
[1, 16, 128, 128]
VAE Decoder
MidBlock + UpDecoder ×4
spatial upsample ×8
[1, 3, 1024, 1024]
Pixel output
native 1024²
[1, 3, 1024, 1024]
STATIC-CONFIG 通道阶梯：512 → 512 → 256 → 128；latent_channels=16
Trace evidence
实例化参数合计
9.716B
峰值 CUDA allocated
9.01 GiB
行级 shape
3,999 rows / 95,984 hits
模块级 shape
2,707 rows / 2,728 hits
运行边界
native 1024² · batch 1
随机初始化 BF16
结构/shape 学习证据；不代表生成质量
ACTUAL-CUDA-FULL-TOPOLOGY
BF16 · NVIDIA A100-SXM4-40GB · native 1024² · 1 shape-equivalent denoiser timestep