Increasing ADG guidance weight leads to faded images on SD1.5

[chaObserv]

I attempted to test ADG on SD1.5. Following the modification on SD3 ADG pipeline, I ran several tests on SD1.5, but the generated images consistently ended up with unnaturally low saturation.

The examples were generated using the EulerDiscreteScheduler with 25 inference steps and the sandcastle prompt.

[jinc7461]

Thank you for your interest in our work.

Our released implementation is designed specifically for SD3.5. Porting it to SD1.5 involves addressing significant architectural differences, particularly regarding model output types (epsilon-prediction/v-prediction) and noise scheduling, which likely caused the saturation issues.

If you could share the code or pipeline you modified, it would greatly help us help you pinpoint the problem.

[chaObserv]

I found some issues in my modification, such as the wrong value of angle constraint (set to pi instead of pi/3), but the issue is still valid.

The code for StableDiffusionPipeline. Most of them are from SD3.5 ADG pipeline.
This modification is only for the regular SD1.5 that is noise prediction model. The part for init_noise_sigma is to properly handle EDM style sigmas used in EulerDiscreteScheduler.

noise_pred_uncond, noise_pred_text = noise_pred.chunk(2)
weight = self.guidance_scale
n, c, w, h = noise_pred_text.shape
if self.scheduler.step_index is None:
    sigma = self.scheduler.sigmas[0]
else:
    sigma = self.scheduler.sigmas[self.scheduler.step_index]

if self.scheduler.init_noise_sigma <= 1:
    # VP/DDPM alpha_cumprod
    alpha_t = (1 - sigma**2) ** 0.5
    sigma_t = sigma
else:
    # EDM style sigmas
    alpha_t = 1.0
    sigma_t = sigma

weight = weight - 1
weight = weight*(weight>0)+1e-3
latent_hat_text = (latents - sigma_t * noise_pred_text) / alpha_t
latent_hat_uncond = (latents - sigma_t * noise_pred_uncond) / alpha_t
latent_diff = latent_hat_text - latent_hat_uncond

latent_theta = torch.acos(call_cos_tensor(latent_hat_text.view(n, -1).to(float), latent_hat_uncond.view(n, -1).to(float)))
latent_theta_new = torch.clip(weight*latent_theta, -angle_constraint, angle_constraint)
_, perp = compute_perpendicular_component(latent_diff, latent_hat_uncond)
latent_v_new = torch.cos(latent_theta_new)*latent_hat_text

latent_p_new = perp*torch.sin(latent_theta_new)/torch.sin(latent_theta)*(torch.sin(latent_theta)>1e-3)+perp*weight*(torch.sin(latent_theta)<=1e-3)
latent_new = latent_v_new + latent_p_new
noise_pred = (latents - alpha_t*latent_new)/sigma_t
noise_pred = noise_pred.view(n, c, w, h).to(latents.dtype)

norm_dims = tuple(range(1, noise_pred.ndim))
latent_new_norm = torch.linalg.vector_norm(latent_new, dim=norm_dims)

adg_norm_list.append(latent_new_norm[0].item())

A updated example with 50 steps is here:

I also adjusted the constraint (pi/3 vs pi/6) to see the change. It appears to relieve the faded effect. However, the overall looking is still worse than CFG in term of saturation.

I suspect that SD1.5 relies on the CFG to have saturation. Figure 2 in paper showed that increasing ADG weight would reduce the value of norm. For SD1.5, it seems some amplification might be needed to achieve a good visual result.

[jinc7461]

This is a really interesting finding—thanks a lot for the contribution.
Your conclusion looks spot on: SD1.5 relies heavily on CFG to boost saturation (otherwise images look washed out), whereas newer models like SD3.5 maintain normal saturation even without it. Since our method suppresses saturation, it naturally struggles with v1.5.
We will update the README shortly to document this limitation and explicitly credit you for this discovery. Thanks again.