Reverse diffusion in inference

[kuzhang]

Dear Author,
I found your work very interesting. Really nice work.

When I looked at the code, I found that the training part is missing. Will it be released in the future?

Secondly, I found the in line 820 of diffcast.py file, you manually set this
'''
preds = self.model_predictions(x, t, cond=cond, ctx=cond, idx=cond,)
'''
the global motion prior and frag_idx are set to residual. I am curious about why you are doing this and removing the global motion prior and frag_idx information in the reverse diffusion? Looking forward to your reply and many thanks in advance.

[DeminYu98]

Hi,
Thank you for your interest to our work. We regret that we will not release the full training code due to certain limitations.
And thank you so much to point the bug. The code of incorporating conditions into unet can be found at

DiffCast/diffcast.py

Line 868 in d135aa4

pred_noise, x_start, *_ = self.model_predictions(frames_pred, time_cond, cond=cond, ctx=ctx, idx=idx, clip_x_start = True, rederive_pred_noise = True)

and the code in line 820 is not used actually, we apply DDIM sample method in the reverse diffusion.