Issue

Describe the solution you'd like.
Integrate source code from kynkaat/guidance-interval into huggingface/diffusers in callbacks.py.

Additional context.

Although, we have a pr#7761 provide a solution as dynamic-classifier-free-guidance, it only support to disable CFG after a certain number of inference steps. However, in the paper of 'Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models', it is stated that it is best to only do guidance in the middle of denoising process.

Moreover, I want it to be natively support for all diffusion models with tasks including text-to-image, image-to-image, text guided image editing, inpainting, etc.

Reference

@inproceedings{Kynkaanniemi2024,
 author = {Tuomas Kynkäänniemi and
 Miika Aittala and
 Tero Karras and
 Samuli Laine and
 Timo Aila and
 Jaakko Lehtinen},
 title = {Applying Guidance in a Limited Interval Improves Sample and Distribution Quality in Diffusion Models},
 booktitle = {Proc. NeurIPS},
 year = {2024},
}