CSS.313.1 : Representation Learning (Autumn 2020)

Lectures

• (September 21, 2020) Lecture 1 : PCA, Power Method

• (November 4, 2020) Lecture 2 : Probabilistic PCA, Non-linear PCA, CCA, Probabilistic CCA

• (November 9, 2020) Lecture 3 : Non-linear CCA - I

• (November 11, 2020) Lecture 4 : Non-linear CCA - II, Kernal-CCA

• (November 16, 2020) Lecture 5 : MDS, Isomap, SNE, t-SNE

• (November 18, 2020) Lecture 6 : Classical Non-parametric Inference-I

• (November 25, 2020) Lecture 7 : Classical Non-parametric Inference-II

• (December 2, 2020) Lecture 8 : Variational Auto-encoders - I

• (December 7, 2020) Lecture 9 : Variational Auto-encoders - II

• (December 14, 2020) Lecture 10 : Generative Adversarial Networks, Optimal Transport Theory

• (December 16, 2020) Lecture 11 : Training Generative Adversarial Networks (by Arnab Mondal, IIT Delhi)

• (December 21, 2020) Lecture 12 : Deep Bandits - I (by Prof. Sandeep Juneja, TIFR)

• (December 23, 2020) Lecture 13 : Deep Bandits - II (by Prof. Sandeep Juneja, TIFR)

• (December 28, 2020) Lecture 14 : More on Adversarial Learning - I (by Prof. Prathosh AP, IIT Delhi)

• (December 30, 2020) Lecture 15 : More on Adversarial Learning - II (by Prof. Prathosh AP, IIT Delhi)

• (January 4, 2021) Lecture 16 : Manifold Learning - I (by Prof. Hariharan Narayanan, TIFR)

• (January 6, 2021) Lecture 16 : Manifold Learning - II (by Prof. Hariharan Narayanan, TIFR)

• (January 11, 2021) Lecture 17 : Active Learning - I (by Sumit Sekhar, Adobe Research)

• (January 13, 2021) Lecture 18 : Active Learning - II (by Sumit Sekhar, Adobe Research)

• (January 20, 25, 27, 2021) Lecture 19, 20, 21 - Student Presentations

Assignments/Final Project

• (Uploaded on November 30, 2020, Due by December 21, 11:30 am) Assignment 1

• (Uploaded on December 21, 2020, Due by January 18, 11:30 am) Assignment 2

Reference Books

• There is no standard textbook, but we will use several books, recent publications, and course materials from similar offerings in other universities.

• [R1] Deep Learning, Ian Goodfellow, Yoshua Bengio, and Aaron Courville

• [R2] Understanding Machine Learning : From Theory to Algorithms, Shai Shalev-Shwartz and Shai Ben-David

• [R3] UIUC's Course on Representation Learning : Algorithms and Models

• [R4] Princeton's Course on Theoretical Machine Learning

• [R5] Stanford's Course on Machine Learning

• [R6] Stanford's Course on Deep Learning

• [R7] An Introduction to Variational Autoencoders

General Information

• Instructor : Himanshu Asnani

• Venue : Online

• Class Timings : Mondays, 11:30 am - 1:00 pm and Wednesdays, 11:30 am - 1:00 pm

• Grading : Class Participation (10%), Homeworks (60%), Final Project (30%)

• The course CANNOT be taken for the qualifiers.

• Assignments : We will be using Jupyter Notebook for the programming aspect of the course. Programming language will be Python and the installation guide can be located here.

• Pre-requisities : Preferable (but not absolutely compulsory) if the Machine Learning course has been taken.

• Tentative Schedule :

  • Module I [Dimensionality Reduction] : PCA, CCA, kernel-PCA/CCA, Probabilistic PCA/CCA, Non-linear PCA/CCA, Non-linear Methods (Isomap, SNE, t-SNE)

  • Module II [Generative Models] : Generative vs Discriminative Models, Classical Non-parametric Inference, Variational Autoencoders (VAEs), Generative Adversarial Networks (GANs)

  • Module III [Miscellaneous Topics and Applications] : Landscape of Information Estimation