Introduction to Probabilistic Graphical Models and Deep Generative Models

Pierre Latouche - Pierre-Alexandre Mattei
UCA+Ecole Polytechnique, INRIA

Master recherche specialité "Mathématiques Appliquées",
Parcours M2 Mathématiques, Vision et Apprentissage (ENS Paris-Saclay), 1er semestre, 2022/2023

This year, the lectures will be taught in English. Homeworks and projects may be submitted in English or in French. Most lectures will be given at ENS Paris Saclay on Wednedays (9-12am). Some lectures will be given via zoom. Aude Sportisse will be the new TA. Documents (slides, proofs, some video recordings) will be uploaded regularly here.


Internship offer: Deep generative models for the joint analysis of networks and continuous data
Internship offer: Are ensembles better at uncertainty quantification?
Internship offer: Quantifying the uncertainty of any algorithm handling missing values with a conformal procedure
Exam of last year

Homework 1

These exercises are due on or before December 16th 2022 and should be submitted on the drop box available online. They can be done in groups of two students. The write-up can be in English or in French. Please submit your answers as a (unique) zip file that you will name MVA DM1<your name>.zip if you worked alone or MVA DM1 <name1> <name2>.zip with both of your names if you worked as a group of two. Indicate your name(s) as well in the documents. Note that the zip file should weight no more than 16Mb. Only such zip files with such names will be evaluated. Your solutions as well as your codes should be present in the zip file. Homework Data
=> upload your solutions online (drop box)

Homework 2

These exercises are due on or before January 20th 2023 and should be submitted on the drop box available online. They can be done in groups of two students. The write-up can be in English or in French. Please submit your answers as a (unique) zip file that you will name MVA DM2<your name>.zip if you worked alone or MVA DM2 <name1> <name2>.zip with both of your names if you worked as a group of two. Indicate your name(s) as well in the documents. Note that the zip file should weight no more than 16Mb. Only such zip files with such names will be evaluated. Your solutions as well as your codes should be present in the zip file. Again, we recommend you to write your code and report thanks to a notebook or Markdown file (in R or Python for the first exercise, and in Python for the second one). Homework 2
=> upload your solutions online (drop box)

Dates of classes

Below is the tentative schedule.

Date Lecturer Topics
October 5th

Amphi 0I10 (bâtiment Ouest, RdC)
Pierre Latouche Introduction
Maximum likelihood
Linear regression
Logistic regression
October 12th

zoom
Pierre Latouche K-means
EM
Gaussian mixtures
October 19th

zoom
Pierre-Alexandre Mattei Directed graphical models
Undirected graphical models
November 9th

Amphi 0I10 (bâtiment Ouest, RdC)
Pierre Latouche Bayesian linear regression
Gaussian processes
EM revisited
Model selection
November 16th

zoom
Pierre Latouche Approximate inference I: variational techniques
Bayesian GMM + VEM, Expectation propagation
November 23th

zoom
Pierre-Alexandre Mattei Sum-product algorithm
HMM
December 7th

zoom
Pierre-Alexandre Mattei Approximate inference II: Monte Carlo, MCMC
December 14th

Amphi 1G58 (bâtiment Sud, au dessus de l'accueil)
Pierre-Alexandre Mattei Approximate inference III: amortized variational inference
Deep latent variable models, Variational auto-encoders
January 4th

Amphi 0I10 (bâtiment Ouest, RdC)
Pierre-Alexandre Mattei Deep generative models beyond VAEs
GAN, autoregressive models, normalizing flows, ...
January 11th

Amphi 0I10 (bâtiment Ouest, RdC)

Final Exam !

Description

This course provides a unifying introduction to probabilistic modelling through the framework of graphical models, together with their associated learning and inference algorithms. It is one of the few historical courses at the core of the MVA program. Recent developments in deep latent variable models (deep neural networks, variational autoencoders, generative adversarial networks, ...) are now part of the program of this course.

References

The course is based on the book Pattern Recognition and Machine Learning by C. Bishop . It is also based on the book Machine Learning a Probabilistic Perspective by Kevin P. Murphy.


Last updated: December 14th, 2022.