Introduction to Probabilistic Graphical Models and Deep Generative Models

Pierre Latouche - Pierre-Alexandre Mattei
UCA+Ecole Polytechnique, INRIA

Master recherche specialité "Mathématiques Appliquées",
Parcours M2 Mathématiques, Vision et Apprentissage (ENS Paris-Saclay), 1er semestre, 2022/2023

This year, the lectures will be taught in English. Homeworks and projects may be submitted in English or in French. Most lectures will be given at ENS Paris Saclay on Wednedays (9-12am). Some lectures will be given via zoom. Aude Sportisse will be the new TA. Documents (slides, proofs, some video recordings) will be uploaded regularly here.

Internship offer: Deep generative models for the joint analysis of networks and continuous data
Internship offer: Are ensembles better at uncertainty quantification?
Internship offer: Quantifying the uncertainty of any algorithm handling missing values with a conformal procedure

Exam of last year

Homework 1

These exercises are due on or before December 16th 2022 and should be submitted on the drop box available online. They can be done in groups of two students. The write-up can be in English or in French. Please submit your answers as a (unique) zip file that you will name MVA DM1<your name>.zip if you worked alone or MVA DM1 <name1> <name2>.zip with both of your names if you worked as a group of two. Indicate your name(s) as well in the documents. Note that the zip file should weight no more than 16Mb. Only such zip files with such names will be evaluated. Your solutions as well as your codes should be present in the zip file. Homework Data
=> upload your solutions online (drop box)

Homework 2

These exercises are due on or before January 20th 2023 and should be submitted on the drop box available online. They can be done in groups of two students. The write-up can be in English or in French. Please submit your answers as a (unique) zip file that you will name MVA DM2<your name>.zip if you worked alone or MVA DM2 <name1> <name2>.zip with both of your names if you worked as a group of two. Indicate your name(s) as well in the documents. Note that the zip file should weight no more than 16Mb. Only such zip files with such names will be evaluated. Your solutions as well as your codes should be present in the zip file. Again, we recommend you to write your code and report thanks to a notebook or Markdown file (in R or Python for the first exercise, and in Python for the second one). Homework 2
=> upload your solutions online (drop box)

Dates of classes

Below is the tentative schedule.

Date	Lecturer	Topics
October 5th Amphi 0I10 (bâtiment Ouest, RdC)	Pierre Latouche	Introduction Maximum likelihood Linear regression Logistic regression
October 12th zoom	Pierre Latouche	K-means EM Gaussian mixtures
October 19th zoom	Pierre-Alexandre Mattei	Directed graphical models Undirected graphical models
November 9th Amphi 0I10 (bâtiment Ouest, RdC)	Pierre Latouche	Bayesian linear regression Gaussian processes EM revisited Model selection
November 16th zoom	Pierre Latouche	Approximate inference I: variational techniques Bayesian GMM + VEM, Expectation propagation
November 23th zoom	Pierre-Alexandre Mattei	Sum-product algorithm HMM
December 7th zoom	Pierre-Alexandre Mattei	Approximate inference II: Monte Carlo, MCMC
December 14th Amphi 1G58 (bâtiment Sud, au dessus de l'accueil)	Pierre-Alexandre Mattei	Approximate inference III: amortized variational inference Deep latent variable models, Variational auto-encoders
January 4th Amphi 0I10 (bâtiment Ouest, RdC)	Pierre-Alexandre Mattei	Deep generative models beyond VAEs GAN, autoregressive models, normalizing flows, ...
January 11th Amphi 0I10 (bâtiment Ouest, RdC)		Final Exam !

Description

This course provides a unifying introduction to probabilistic modelling through the framework of graphical models, together with their associated learning and inference algorithms. It is one of the few historical courses at the core of the MVA program. Recent developments in deep latent variable models (deep neural networks, variational autoencoders, generative adversarial networks, ...) are now part of the program of this course.

References

The course is based on the book Pattern Recognition and Machine Learning by C. Bishop . It is also based on the book Machine Learning a Probabilistic Perspective by Kevin P. Murphy.

Last updated: December 14th, 2022.