Imperial College London, University of Bristol, MediaTek Research Published in NeurIPS '23 Diffusion Model Workshop Link: https://arxiv.org/abs/2311.00157
Recently, Zhang et al. have proposed the Diffusion Exponential Integrator Sampler (DEIS) for fast generation of samples from Diffusion Models. It leverages the semi-linear nature of the probability flow ordinary differential equation (ODE) in order to greatly reduce integration error and improve generation quality at low numbers of function evaluations (NFEs). Key to this approach is the score function reparameterisation, which reduces the integration error incurred from using a fixed score function estimate over each integration step. The original authors use the default parameterisation used by models trained for noise prediction -- multiply the score by the standard deviation of the conditional forward noising distribution. We find that although the mean absolute value of this score parameterisation is close to constant for a large portion of the reverse sampling process, it changes rapidly at the end of sampling. As a simple fix, we propose to instead reparameterise the score (at inference) by dividing it by the average absolute value of previous score estimates at that time step collected from offline high NFE generations. We find that our score normalisation (DEIS-SN) consistently improves FID compared to vanilla DEIS, showing an FID improvement from 6.44 to 5.57 at 10 NFEs for our CIFAR-10 experiments.
Coming soon.
Coming soon.
Want to cite this paper ?
@inproceedings{
xia2023score,
title={Score Normalization for a Faster Diffusion Exponential Integrator Sampler},
author={Guoxuan Xia and
Duolikun Danier and
Ayan Das and
Stathi Fotiadis and
Farhang Nabiei and
Ushnish Sengupta and
Alberto Bernacchia},
booktitle={NeurIPS 2023 Workshop on Diffusion Models},
year={2023},
url={https://openreview.net/forum?id=AQvPfN33g9}
}