Fix DDIMParallelScheduler batch path ignoring final_alpha_cumprod#14216
Open
Osamaali313 wants to merge 1 commit into
Open
Fix DDIMParallelScheduler batch path ignoring final_alpha_cumprod#14216Osamaali313 wants to merge 1 commit into
Osamaali313 wants to merge 1 commit into
Conversation
`DDIMParallelScheduler`'s parallel path must match its sequential path
(the ParaDiGMS correctness guarantee is parallel == sequential). For the
terminal step (`prev_timestep < 0`), the scalar methods use
`self.final_alpha_cumprod`:
# _get_variance (l.274) and step (l.448)
alpha_prod_t_prev = self.alphas_cumprod[prev_timestep] if prev_timestep >= 0 else self.final_alpha_cumprod
but the batched methods hardcode `1.0`:
# _batch_get_variance and batch_step_no_noise
alpha_prod_t_prev[prev_t < 0] = torch.tensor(1.0)
`self.final_alpha_cumprod` equals `1.0` only when `set_alpha_to_one=True`;
with `set_alpha_to_one=False` (the default in Stable Diffusion 1.x scheduler
configs) it is `alphas_cumprod[0]` (~0.999). So on the final denoising step
the parallel path diverges from the sequential one. Use
`self.final_alpha_cumprod` in both batched methods, matching the scalar
siblings (it is already moved to the correct device before use).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
DDIMParallelSchedulerimplements ParaDiGMS, whose whole premise is that the parallel path produces the same result as the sequential DDIM path. For the terminal step (prev_timestep < 0), the scalar methods useself.final_alpha_cumprod:but the batched methods hardcode
1.0:self.final_alpha_cumprodis set at init to:So it equals
1.0only whenset_alpha_to_one=True. Withset_alpha_to_one=False— the default in Stable Diffusion 1.x scheduler configs — it isalphas_cumprod[0](~0.999). On the final denoising step the batched path therefore diverges from the scalar path, breaking the parallel==sequential guarantee.Fix
Use
self.final_alpha_cumprodin both batched methods, matching the scalar siblings:final_alpha_cumprodis already moved to the correct device beforebatch_step_no_noiseuses it (line 577), and the scalar siblings reference it the same way.Reproduction
Porting the verbatim terminal-step arithmetic of both paths, with
set_alpha_to_one=False(final_alpha_cumprod ≈ 0.999):The current code diverges by ~6.8e-2 on the final step; the fix makes the parallel path bit-exactly equal to the sequential path.