Lecture Calendar
- The schedule below is tentative, and some topics may change.
- For each lecture we link to suggested readings below the calendar table.
- During the regular lecture times (Tuesdays and Thursdays at 12:30pm), Prof. Sudderth will present the posted slides via Zoom.
To join lectures, use a UCI zoom account (and login with your UCInetID) to connect to the CS274B Zoom meeting: https://uci.zoom.us/j/95800730270 Links to an external site. - To join the live Zoom meeting, you will need to use the password e-mailed to registered students.
- After each lecture, a recording will be posted to the CS274B YuJa channel Links to an external site.
|
Number |
Date |
Topics |
Readings |
Materials |
|
1A |
3/30/2021 |
Survey of Applications and Algorithms |
||
|
1B |
4/01/2021 |
Directed and Undirected Graphical Models |
||
|
2A |
4/06/2021 |
Inference Problems, Variable Elimination Algorithms |
Markov slides Download Markov slides (36-55), Elimination slides Download Elimination slides (1-19) |
|
|
2B |
4/08/2021 |
Message Passing Algorithms, Belief Propagation |
Elimination slides
Download Elimination slides (20-35), BP slides
Download BP slides (1-26) |
|
|
3A |
4/13/2021 |
Junction Trees, Loopy Belief Propagation |
BP slides Download BP slides (27-32), Loopy slides Download Loopy slides (1-33) |
|
|
3B |
4/15/2021 |
Learning Directed Graphical Models, L2 & L1 Regularization |
Loopy slides Download Loopy slides (34-47), Directed Learning slides Download Directed Learning slides (1-33) |
|
|
4A |
4/20/2021 |
Exponential Families of Distributions |
Exponential families |
Directed Learning slides Download Directed Learning slides (34-56), Undirected Learning slides Download Undirected Learning slides (17-25) |
|
4B |
4/22/2021 |
Learning Undirected Graphical Models, Graph Structure Learning, Information Theory |
Undirected Learning slides Download Undirected Learning slides | |
|
5A |
4/27/2021 |
Learning Markov Trees, Expectation Maximization (EM) Algorithms |
Structure Learning slides Download Structure Learning slides, EM Algorithm slides Download EM Algorithm slides | |
|
5B |
4/29/2021 |
Monte Carlo Methods |
Monte Carlo | Monte Carlo slides Download Monte Carlo slides (1-24) |
|
6A |
5/04/2021 |
MCMC Methods, Metropolis-Hastings |
Monte Carlo slides Download Monte Carlo slides (25-41) | |
|
6B |
5/06/2021 |
MCMC Methods, Gibbs Samplers |
||
|
7A |
5/11/2021 |
Kalman Filters, Particle Filters |
||
|
7B |
5/13/2021 |
Particle Belief Propagation |
Particles | Filtering slides Download Filtering slides (37-51), Particle BP slides Download Particle BP slides |
|
8A |
5/18/2021 |
Mean Field Variational Methods |
||
|
8B |
5/20/2021 |
Bethe Variational Methods, Loopy BP |
Loopy BP | Variational slides Download Variational slides (31-53), DBN slides Download DBN slides |
|
9A |
5/25/2021 |
Bayesian Nonparametrics |
BNP Theory slides Download BNP Theory slides | |
|
9B |
5/27/2021 |
Stochastic and Memoized Variational Inference |
||
|
10A |
6/01/2021 |
Structured Variational Autoencoders |
Tutorial slides Links to an external site., Tutorial video Links to an external site. |
|
|
10B |
6/03/2021 |
Final Project Presentations |
|
|
|
- |
6/10/2021 |
Final Project Reports Due |
|
|
Suggested Readings
We provide suggested readings from several sources. You do not need to read all of them. Instead, we suggest you compare various options, and choose the reference whose style you like best. Barber's Bayesian Reasoning and Machine Learning Links to an external site. is freely available online, and is a good place to start.
Acronyms for Primary Resources
- BRML Links to an external site.: Bayesian Reasoning and Machine Learning, David Barber, Cambridge University Press 2012. Free online.
- MLaPP Links to an external site.: Machine Learning: A Probabilistic Perspective, Kevin Murphy, MIT Press 2012. Excerpt online.
- PRML Links to an external site.: Pattern Recognition and Machine Learning, Christopher Bishop, Springer 2007. Free online.
- GEV Links to an external site.: Graphical Models, Exponential Families, and Variational Inference, Martin Wainwright & Michael Jordan, Foundations & Trends in Machine Learning, 2008.
- EBS: Graphical Models for Visual Object Recognition and Tracking, Erik B. Sudderth, PhD Thesis (Chapter 2), MIT 2006.
Graphical Model Tutorials
- A Brief Introduction to Graphical Models & Bayesian Networks Links to an external site., K. Murphy, 1998.
- Graphical Models Links to an external site., M. Jordan, Statistical Science 2004.
Directed & Undirected Graphs: Factorization & Markov Properties
- BRML Links to an external site.: Chapters 2-4, excluding Sec. 3.4.
- MLaPP Links to an external site.: Sec. 10.1-10.2, 10.5, 19.1-19.4.
- PRML Links to an external site.: Sec. 8.1-8.3.
- GEV Links to an external site.: Sec. 2.1-2.4.
- EBS: Sec. 2.2.1-2.2.3.
Inference via Variable Elimination
- BRML Links to an external site.: Sec. 5.1, 5.3.
- MLaPP Links to an external site.: Sec. 20.3.
- PRML Links to an external site.: Sec. 8.4.
Inference via Belief Propagation: Sum-Product & Max-Product
- BRML Links to an external site.: Sec. 5.1-5.2.
- MLaPP Links to an external site.: Sec. 20.2.
- PRML Links to an external site.: Sec. 8.4.
- GEV Links to an external site.: Sec. 2.5.
- EBS: Sec. 2.2.5, 2.3.2.
- Factor Graphs and the Sum-Product Algorithm Links to an external site., F. Kschischang, B. Frey, & H. A. Loeliger, IEEE Trans. Info Theory 2001.
Inference via Junction Tree Propagation
- BRML Links to an external site.: Chapter 6.
- MLaPP Links to an external site.: Sec. 20.4-20.5.
- GEV Links to an external site.: Sec. 2.5.
- A Short Course on Graphical Models Links to an external site., M. Paskin, 2003.
Exponential Family Distributions: Learning & Inference
- BRML Links to an external site.: Chapter 8 excluding Sec. 8.4, 8.8.
- MLaPP Links to an external site.: Sec. 9.2.
- PRML Links to an external site.: Sec. 2.4.
- GEV Links to an external site.: Chapter 3.
- EBS: Sec. 2.1.
Learning (Directed & Undirected) Graphical Model Parameters
- BRML Links to an external site.: Chapter 9 excluding Sec. 9.5.
- MLaPP Links to an external site.: Sec. 10.4, 19.5.
- GEV Links to an external site.: Sec. 6.1.
Learning via the Expectation Maximization (EM) Algorithm
- BRML Links to an external site.: Sec. 11.1-11.3.
- MLaPP Links to an external site.: Sec. 11.4.
- PRML Links to an external site.: Chapter 9.
- GEV Links to an external site.: Sec. 6.2.
- EBS: Sec. 2.3.3.
- A View of the EM Algorithm that Justifies Incremental, Sparse, and Other Variants Links to an external site., R. Neal & G. Hinton, 1998.
Learning (Directed & Undirected) Graphical Model Structure
- BRML Links to an external site.: Sec. 9.5-9.6, 12.1-12.3, 12.5.
- MLaPP Links to an external site.: Chapter 26 excluding Sec. 26.5-26.7.
Inference & Learning for Gaussian Graphical Models
- BRML Links to an external site.: Sec. 8.4, Chapter 24.
- MLaPP Links to an external site.: Sec. 10.2.5, 18.1-18.4, 19.4.4, 20.2.3, 26.7.
- PRML Links to an external site.: Sec. 13.3.
- A Unifying Review of Linear Gaussian Models Links to an external site., S. Roweis & Z. Ghahramani, Neural Computation 1999.
- Bayesian Modeling of Uncertainty in Low-Level Vision Links to an external site., R. Szeliski, IJCV 1990.
Monte Carlo Methods: Rejection & Importance Sampling
- BRML Links to an external site.: Sec. 27.1-27.2.
- MLaPP Links to an external site.: Sec. 23.1-23.4.
- PRML Links to an external site.: Sec. 11.1.
- EBS: Sec. 2.4.
- Introduction to Monte Carlo Methods Links to an external site., Sec. 1-3, D. MacKay, 1999.
Particles & Sequential Monte Carlo
- BRML Links to an external site.: Sec. 27.6.
- MLaPP Links to an external site.: Sec. 23.5.
- An Overview of Existing Methods and Recent Advances in Sequential Monte Carlo Links to an external site., Cappe, Godsill, & Moulines, IEEE 2007.
- Nonparametric Belief Propagation, Sudderth, Ihler, Isard, Freeman, & Willsky, CACM 2010.
Markov Chain Monte Carlo (MCMC): Gibbs & Metropolis-Hastings
- BRML Links to an external site.: Sec. 27.3-27.5.
- MLaPP Links to an external site.: Chapter 24.
- PRML Links to an external site.: Sec. 11.2-11.4.
- Introduction to Monte Carlo Methods Links to an external site., Sec. 4-8, D. MacKay, 1999.
- An Introduction to MCMC for Machine Learning Links to an external site., Andrieu, de Freitas, Doucet, & Jordan, Machine Learning 2003.
Variational Methods: Naive & Structured Mean Field
- BRML Links to an external site.: Sec. 28.3-28.4.
- MLaPP Links to an external site.: Chapter 21.
- PRML Links to an external site.: Sec. 10.1-10.6.
- GEV Links to an external site.: Chapter 5.
- EBS: Sec. 2.3.1.
- Variational Message Passing Links to an external site., J. Winn & C. Bishop, JMLR 2005.
Variational Methods: Bethe Approximations, Loopy & Reweighted BP
- BRML Links to an external site.: Sec. 28.7.
- MLaPP Links to an external site.: Sec. 22.1-22.4.
- GEV Links to an external site.: Sec. 4.1, Chapter 7.
- EBS: Sec. 2.3.2.
- Understanding Belief Propagation and its Generalizations Links to an external site., J. Yedidia, W. Freeman, & Y. Weiss, IJCAI 2001.
Discriminative Learning: Conditional Random Fields & Structural SVMs
- MLaPP Links to an external site.: Sec. 19.6-19.7.
- An Introduction to Conditional Random Fields Links to an external site., C. Sutton & A. McCallum, Foundations & Trends in ML 2011.
- Structured Prediction and Learning in Computer Vision Links to an external site., S. Nowozin & C. Lampert, CVPR 2012.
Bayesian Nonparametrics: Dirichlet Processes
- MLaPP Links to an external site.: Sec. 25.2.
- EBS: Sec. 2.5.
- Modern Bayesian Nonparametrics Links to an external site., P. Orbanz & Y. W. Teh, NIPS 2011.