Abstract
Diffusion Model, a new generative model family, has taken the world by storm after the seminal paper by Ho, Jain, and Abbeel (2020). While diffusion models are often described as a probabilistic Markov Chain, their fundamental principle lies in the decade-old theory of Stochastic Differential Equation (SDE), as found out later by Song et al. (2021). In this article, we will go back and revisit the ‘fundamental ingredients’ behind the SDE formulation, and show how the idea can be ‘shaped’ to get to the modern form of Score-based Diffusion Models. We’ll start from the very definition of ‘score’, how it was used in the context of generative modeling, how we achieve the necessary theoretical guarantees, how the design choices were made and finally arrive at the more ‘principled’ framework of Score-based Diffusion. Throughout the article, we provide several intuitive illustrations for ease of understanding.
Citation
@inproceedings{das2024,
author = {Das, Ayan},
title = {Building {Diffusion} {Model’s} Theory from Ground Up},
booktitle = {ICLR 2024 BlogPost Track},
date = {2024-02-15},
url = {https://iclr-blogposts.github.io/2024/blog/diffusion-theory-from-scratch/},
langid = {en}
}