Lecture Calendar
- The schedule below is tentative, and some topics may change.
- For each lecture we link to suggested readings below the calendar table.
- During the regular lecture times (Tuesdays and Thursdays at 11:00am), Prof. Sudderth will present the posted slides in PCB 1200.
- Video recordings of course meetings will be posted (after some delay) to the CS274B YuJa channel Links to an external site..
|
Number |
Date |
Topics |
Readings |
Materials |
|
1A |
4/02/2024 |
Survey of Applications and Algorithms |
Slides: Overview Download Overview |
|
|
1B |
4/04/2024 |
Directed and Undirected Graphical Models |
Slides: Markov Download Markov (1-41) |
|
|
2A |
4/09/2024 |
Inference Problems, Variable Elimination Algorithms |
Slides: Markov Download Markov (42-74), Elimination Download Elimination (1-8) |
|
|
2B |
4/11/2024 |
Message Passing Algorithms, Belief Propagation |
Slides: Elimination
Download Elimination (15-30), BP
Download BP (1-21) |
|
|
3A |
4/16/2024 |
Loopy Belief Propagation |
Slides: BP Download BP (22-49), Loopy Download Loopy (1-8) |
|
|
3B |
4/18/2024 |
Junction Trees, Max-Product Belief Propagation |
Slides: Loopy Download Loopy (9-56) |
|
|
4A |
4/23/2024 |
Learning Directed Graphical Models, L2 & L1 Regularization |
Parameter learning |
Slides: Directed Learning Download Directed Learning (1-39) |
|
4B |
4/25/2024 |
Learning Undirected Graphical Models, Exponential Families |
Exponential families | Slides: Directed Learning Download Directed Learning (40-55), Undirected Learning Download Undirected Learning (1-20) |
|
5A |
4/30/2024 |
Learning Undirected Graphical Models, Structure Learning |
Exponential families | Slides: Undirected Learning Download Undirected Learning (21-57) |
|
5B |
5/02/2024 |
Information Theory, Learning Markov Trees |
Structure learning | Slides: Structure Download Structure, EM Download EM (1-8) |
|
6A |
5/07/2024 |
Expectation Maximization (EM) Algorithms |
EM algorithms | Slides: EM Download EM (6-50) |
|
6B |
5/09/2024 |
Monte Carlo, MCMC Methods |
Monte Carlo |
Slides: Monte Carlo Download Monte Carlo (1-33) |
|
7A |
5/14/2024 |
MCMC, Metropolis-Hastings, Gibbs Samplers |
MCMC |
Slides: Monte Carlo Download Monte Carlo (28-68) |
|
7B |
5/16/2024 |
MCMC, (Blocked) Gibbs Samplers, Kalman Filters |
MCMC, Gaussian models | Slides: Monte Carlo Download Monte Carlo (69-74), Filtering Download Filtering (1-28) |
|
8A |
5/21/2024 |
Particle Filters, Particle Belief Propagation |
Slides: Filtering Download Filtering (29-45), Particle BP Download Particle BP |
|
|
8B |
5/23/2024 |
Mean Field and Bethe Variational Methods |
Slides: Variational Download Variational (1-29) | |
|
9A |
5/28/2024 |
Bayesian Nonparametrics |
Slides: Variational Download Variational (30-42), BNP Download BNP (1-40) | |
|
9B |
5/30/2024 |
Bayesian Nonparametrics |
Slides: BNP Download BNP (37-92) |
|
|
10A |
6/04/2024 |
Structured Deep Generative Models |
Dynamical VAEs Links to an external site., Structured VAEs Links to an external site. |
|
|
10B |
6/06/2024 |
Final Project Presentations |
|
|
|
- |
6/13/2024 |
Final Project Reports Due |
|
|
Suggested Readings
We provide suggested readings from several sources. You do not need to read all of them. Instead, we suggest you compare various options, and choose the reference whose style you like best. Barber's Bayesian Reasoning and Machine Learning Links to an external site. is freely available online, and is a good place to start.
Acronyms for Primary Resources
- BRML Links to an external site.: Bayesian Reasoning and Machine Learning, David Barber, Cambridge University Press 2012. Free online.
- MLaPP Links to an external site.: Machine Learning: A Probabilistic Perspective, Kevin Murphy, MIT Press 2012. Excerpt online.
- PRML Links to an external site.: Pattern Recognition and Machine Learning, Christopher Bishop, Springer 2007. Free online.
- GEV Links to an external site.: Graphical Models, Exponential Families, and Variational Inference, Martin Wainwright & Michael Jordan, Foundations & Trends in Machine Learning, 2008.
- EBS: Graphical Models for Visual Object Recognition and Tracking, Erik B. Sudderth, PhD Thesis (Chapter 2), MIT 2006.
Graphical Model Tutorials
- A Brief Introduction to Graphical Models & Bayesian Networks Links to an external site., K. Murphy, 1998.
- Graphical Models Links to an external site., M. Jordan, Statistical Science 2004.
Directed & Undirected Graphs: Factorization & Markov Properties
- BRML Links to an external site.: Chapters 2-4, excluding Sec. 3.4.
- MLaPP Links to an external site.: Sec. 10.1-10.2, 10.5, 19.1-19.4.
- PRML Links to an external site.: Sec. 8.1-8.3.
- GEV Links to an external site.: Sec. 2.1-2.4.
- EBS: Sec. 2.2.1-2.2.3.
Inference via Variable Elimination
- BRML Links to an external site.: Sec. 5.1, 5.3.
- MLaPP Links to an external site.: Sec. 20.3.
- PRML Links to an external site.: Sec. 8.4.
Inference via Belief Propagation: Sum-Product & Max-Product
- BRML Links to an external site.: Sec. 5.1-5.2.
- MLaPP Links to an external site.: Sec. 20.2.
- PRML Links to an external site.: Sec. 8.4.
- GEV Links to an external site.: Sec. 2.5.
- EBS: Sec. 2.2.5, 2.3.2.
- Factor Graphs and the Sum-Product Algorithm Links to an external site., F. Kschischang, B. Frey, & H. A. Loeliger, IEEE Trans. Info Theory 2001.
Inference via Junction Tree Propagation
- BRML Links to an external site.: Chapter 6.
- MLaPP Links to an external site.: Sec. 20.4-20.5.
- GEV Links to an external site.: Sec. 2.5.
- A Short Course on Graphical Models Links to an external site., M. Paskin, 2003.
Exponential Family Distributions: Learning & Inference
- BRML Links to an external site.: Chapter 8 excluding Sec. 8.4, 8.8.
- MLaPP Links to an external site.: Sec. 9.2.
- PRML Links to an external site.: Sec. 2.4.
- GEV Links to an external site.: Chapter 3.
- EBS: Sec. 2.1.
Learning (Directed & Undirected) Graphical Model Parameters
- BRML Links to an external site.: Chapter 9 excluding Sec. 9.5.
- MLaPP Links to an external site.: Sec. 10.4, 19.5.
- GEV Links to an external site.: Sec. 6.1.
Learning via the Expectation Maximization (EM) Algorithm
- BRML Links to an external site.: Sec. 11.1-11.3.
- MLaPP Links to an external site.: Sec. 11.4.
- PRML Links to an external site.: Chapter 9.
- GEV Links to an external site.: Sec. 6.2.
- EBS: Sec. 2.3.3.
- A View of the EM Algorithm that Justifies Incremental, Sparse, and Other Variants Links to an external site., R. Neal & G. Hinton, 1998.
Learning (Directed & Undirected) Graphical Model Structure
- BRML Links to an external site.: Sec. 9.5-9.6, 12.1-12.3, 12.5.
- MLaPP Links to an external site.: Chapter 26 excluding Sec. 26.5-26.7.
Inference & Learning for Gaussian Graphical Models
- BRML Links to an external site.: Sec. 8.4, Chapter 24.
- MLaPP Links to an external site.: Sec. 10.2.5, 18.1-18.4, 19.4.4, 20.2.3, 26.7.
- PRML Links to an external site.: Sec. 13.3.
- A Unifying Review of Linear Gaussian Models Links to an external site., S. Roweis & Z. Ghahramani, Neural Computation 1999.
- Bayesian Modeling of Uncertainty in Low-Level Vision Links to an external site., R. Szeliski, IJCV 1990.
Monte Carlo Methods: Rejection & Importance Sampling
- BRML Links to an external site.: Sec. 27.1-27.2.
- MLaPP Links to an external site.: Sec. 23.1-23.4.
- PRML Links to an external site.: Sec. 11.1.
- EBS: Sec. 2.4.
- Introduction to Monte Carlo Methods Links to an external site., Sec. 1-3, D. MacKay, 1999.
Particles & Sequential Monte Carlo
- BRML Links to an external site.: Sec. 27.6.
- MLaPP Links to an external site.: Sec. 23.5.
- An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo Links to an external site., Cappe, Godsill, & Moulines, IEEE 2007.
- Nonparametric Belief Propagation, Sudderth, Ihler, Isard, Freeman, & Willsky, CACM 2010.
Markov Chain Monte Carlo (MCMC): Gibbs & Metropolis-Hastings
- BRML Links to an external site.: Sec. 27.3-27.5.
- MLaPP Links to an external site.: Chapter 24.
- PRML Links to an external site.: Sec. 11.2-11.4.
- Introduction to Monte Carlo Methods Links to an external site., Sec. 4-8, D. MacKay, 1999.
- An Introduction to MCMC for Machine Learning Links to an external site., Andrieu, de Freitas, Doucet, & Jordan, Machine Learning 2003.
Variational Methods: Naive & Structured Mean Field
- BRML Links to an external site.: Sec. 28.3-28.4.
- MLaPP Links to an external site.: Chapter 21.
- PRML Links to an external site.: Sec. 10.1-10.6.
- GEV Links to an external site.: Chapter 5.
- EBS: Sec. 2.3.1.
- Variational Message Passing Links to an external site., J. Winn & C. Bishop, JMLR 2005.
Variational Methods: Bethe Approximations, Loopy & Reweighted BP
- BRML Links to an external site.: Sec. 28.7.
- MLaPP Links to an external site.: Sec. 22.1-22.4.
- GEV Links to an external site.: Sec. 4.1, Chapter 7.
- EBS: Sec. 2.3.2.
- Understanding Belief Propagation and its Generalizations Links to an external site., J. Yedidia, W. Freeman, & Y. Weiss, IJCAI 2001.
Discriminative Learning: Conditional Random Fields & Structural SVMs
- MLaPP Links to an external site.: Sec. 19.6-19.7.
- An Introduction to Conditional Random Fields Links to an external site., C. Sutton & A. McCallum, Foundations & Trends in ML 2011.
- Structured Prediction and Learning in Computer Vision Links to an external site., S. Nowozin & C. Lampert, CVPR 2012.
Bayesian Nonparametrics: Dirichlet Processes
- MLaPP Links to an external site.: Sec. 25.2.
- EBS: Sec. 2.5.
- Modern Bayesian Nonparametrics Links to an external site., P. Orbanz & Y. W. Teh, NIPS 2011.
Dynamical & Structured Variational Autoencoders
- An Introduction to Variational Autoencoders Links to an external site., D. Kingma & M. Welling, Foundations & Trends in ML 2019.
- Graphical models and exponential families in the age of differentiable programming Links to an external site., M. Johnson, UAI 2019.
- Deep generative modeling of sequential data with dynamical variational autoencoders Links to an external site., S. Leglaive, X. Alameda-Pineda, & L. Girin, ICASSP 2021.
- Sequential Latent Variable Models & Filtering Links to an external site., J. Marino, 2020.