Building Diffusion Model’s theory from ground up

International Conference on Learning Representation (ICLR) BlogPost Track, 2024
Generative Models

Ayan Das

University of Surrey, UK

MediaTek Research UK


February 15, 2024


Diffusion Model, a new generative model family, has taken the world by storm after the seminal paper by Ho, Jain, and Abbeel (2020). While diffusion models are often described as a probabilistic Markov Chain, their fundamental principle lies in the decade-old theory of Stochastic Differential Equation (SDE), as found out later by Song et al. (2021). In this article, we will go back and revisit the ‘fundamental ingredients’ behind the SDE formulation, and show how the idea can be ‘shaped’ to get to the modern form of Score-based Diffusion Models. We’ll start from the very definition of ‘score’, how it was used in the context of generative modeling, how we achieve the necessary theoretical guarantees, how the design choices were made and finally arrive at the more ‘principled’ framework of Score-based Diffusion. Throughout the article, we provide several intuitive illustrations for ease of understanding.

Ho, Jonathan, Ajay Jain, and Pieter Abbeel. 2020. “Denoising Diffusion Probabilistic Models.” In NeurIPS.
Song, Yang, Diederik P. Kingma, Abhishek Kumar, Stefano Ermon, and Ben Poole. 2021. “Score-Based Generative Modeling Through Stochastic Differential Equations.” In ICLR.
Blog (self-hosted) Blog (official) Code


BibTeX citation:
  author = {Das, Ayan},
  title = {Building {Diffusion} {Model’s} Theory from Ground Up},
  booktitle = {ICLR 2024 BlogPost Track},
  date = {2024-02-15},
  url = {},
  langid = {en}
For attribution, please cite this work as:
Das, Ayan. 2024. “Building Diffusion Model’s Theory from Ground Up.” In ICLR 2024 BlogPost Track.