Introduction to (deep) Probabilistic Graphical Models

Pierre Latouche - Pierre-Alexandre Mattei
Université de Paris+Ecole Polytechnique, INRIA

Master recherche specialité "Mathématiques Appliquées",
Parcours M2 Mathématiques, Vision et Apprentissage (ENS Paris-Saclay), 1er semestre, 2021/2022

This year, the lectures will be taught in English. Homeworks and projects may be submitted in English or in French. Most lectures will be given at ENS Paris Saclay (Grand-Amphi 1G58) on Wednedays (9-12am). Otmane Sakhi will be the TA. Documents (slides, proofs, some video recordings) are uploaded regularly here.


Homework 1

These exercises are due on or before November 19th 2021 and should be submitted on the drop box available online. They can be done in groups of two students. The write-up can be in English or in French. Please submit your answers as a (unique) zip file that you will name MVA DM1<your name>.zip if you worked alone or MVA DM1 <name1> <name2>.zip with both of your names if you worked as a group of two. Indicate your name(s) as well in the documents. Note that the zip file should weight no more than 16Mb. Only such zip files with such names will be evaluated. Your solutions as well as your codes should be present in the zip file. Homework Data

Homework 2

These exercises are due on or before January 14th 2022 and should be submitted on the drop box available online. They can be done in groups of two students. The write-up can be in English or in French. Please submit your answers as a (unique) zip file that you will name MVA DM2<your name>.zip if you worked alone or MVA DM2 <name1> <name2>.zip with both of your names if you worked as a group of two. Indicate your name(s) as well in the documents. Note that the zip file should weight no more than 16Mb. Only such zip files with such names will be evaluated. Your solutions as well as your codes should be present in the zip file. Again, we recommend you to write your code and report thanks to a notebook or Markdown file (in R or Python for the first exercise, and in Python for the second one). Homework 2
=> upload your solutions online (drop box)

Exam

The PGM exam is on Wednesday January 12th (9am -> 12am) in Le Grand Amphi 1G58 of ENS Paris Saclay. Documents are not allowed.

Internship: advanced discrete optimization meets machine learning

In this internship, we aim at addressing the problem of Bayesian variable selection for high-dimensional linear and nonlinear regression. More details here.

Dates of classes

Below is the tentative schedule.

Date Lecturer Topics
October 6th

Le Grand-Amphi 1G58
Pierre Latouche Introduction
Maximum likelihood
Linear regression
Logistic regression
October 13th

zoom click here
Pierre Latouche K-means
EM
Gaussian mixtures
October 20th

Le Grand-Amphi 1G58
Pierre Latouche Bayesian linear regression
Gaussian processes
EM revisited
Model selection
November 10th

Le Grand-Amphi 1G58
Pierre Latouche Approximate inference I: variational techniques
Bayesian GMM + VEM, Expectation propagation
November 24th

Le Grand-Amphi 1G58
Pierre-Alexandre Mattei Directed graphical models
Undirected graphical models
November 29th

zoom click here
Pierre-Alexandre Mattei Sum-product algorithm
HMM
December 1st

zoom click here Warnings: this lecture will be given between 2 and 5pm !
Pierre-Alexandre Mattei Approximate inference II: Monte Carlo, MCMC
December 8th

Le Grand-Amphi 1G58
Pierre-Alexandre Mattei Approximate inference III: amortized variational inference
Deep latent variable models, Variational auto-encoders
December 15th

Le Grand-Amphi 1G58
Pierre-Alexandre Mattei Deep generative models beyond VAEs
GAN, autoregressive models, normalizing flows, ...
January 12th

Le Grand-Amphi 1G58

Final Exam !

Description

This course provides a unifying introduction to probabilistic modelling through the framework of graphical models, together with their associated learning and inference algorithms.

References

The course will be based on the book Pattern Recognition and Machine Learning by C. Bishop Book


Last updated: December 10th, 2021.