10701: INTRO TO MACHINE LEARNING

10-701, Fall 2018

WEH 7500, Mon & Wed 3:00PM - 4:20PM

Instructors Pradeep Ravikumar (pradeepr at cs dot cmu dot edu)
Ziv Bar-Joseph (zivbj at andrew dot cmu dot edu)

Teaching Assistants Daniel Bird (dpbird at andrew dot cmu dot edu)
Shubhranshu Shekhar (shubhras at andrew dot cmu dot edu)
Zirui Wang (ziruiw at andrew dot cmu dot edu)
Adithya Raghuraman (araghura at andrew dot cmu dot edu)
Jing Mao (jingmao at andrew dot cmu dot edu)
Umang Bhatt (usb at andrew dot cmu dot edu)
Sarah Mallepalle (smallepa at andrew dot cmu dot edu)
Yang Gao (yanggao at andrew dot cmu dot edu)
Chieh Lin (chiehl1 at andrew dot cmu dot edu)

Office Hours Pradeep Ravikumar: Tuesday: 2:00pm-3:00pm in GHC 8111
Ziv Bar-Joseph: Monday after class
Daniel Bird: Tuesday 9:00am-10:00am in GHC 8110
Shubhranshu Shekhar: Thursday 10:30am-11:30am in HBH 3037
Zirui Wang: Thursday 3:00pm-4:00pm at GHC 8th Floor Study Area
Adithya Raghuraman: Wednesday 10:00am-11:00am at GHC 8th Floor Dining Area
Jing Mao: Monday 10:00am-11:00am at GHC 8th Floor Dining Area
Umang Bhatt: Monday 11:00pm-12:00pm at GHC 8th Floor
Sarah Mallepalle: Friday 11:00am-12:00pm at GHC 5th Floor
Yang Gao: Tuesday 4:00pm-5:00pm in REH 244
Chieh Lin: Tuesday 2:00pm-3:00pm in GHC 8127

Grading 40% Homeworks, 15% Exam 1, 15% Exam 2, 25% Project, 5% Class Participation

Textbooks Lectures are intended to be self-contained. For supplementary readings, with each lecture, we will have pointers to either online reference materials, or chapters from the following books:

CB: Pattern Recognition and Machine Learning, Christopher Bishop.
KM: Machine Learning: A probabilistic perspective, Kevin Murphy.
HTF: The Elements of Statistical Learning: Data Mining, Inference and Prediction, Trevor Hastie, Robert Tibshirani, Jerome Friedman.
TM: Machine Learning, Tom Mitchell.

Course Details Syllabus. Piazza Discussion Board. Homeworks. Project. Livestream

Tentative Schedule

Date Inst. Topic Readings Notes

Module: Foundations

Aug 27 ZB Intro, Three Axes of ML: Data, Algorithms, Tasks, MLE (slides) KM Chap. 1

Aug 29 ZB Bayesian Estimation, MAP, Decision Theory, Model-free, Risk Minimization (slides, notes) KM Chap 2, 5, 6,
TM Chap 6

Sept 3 No Class, Labor Day

Sept 5 ZB Non-parametric Models: K nearest neighbors, Kernel regression (slides, notes) TM Chap. 8
HTF Chap. 6, 13
KM Chap. 14 HW1 out

Sept 10 No Class, Jewish New Year

Module: Prediction, Parametric Methods

Sept 12 ZB Regression: Linear Regression (slides, notes (1), (2)) CB Chap. 3

Sept 17 PR Regularized, Polynomial, Logistic Regression (slides, notes) CB Chap. 4

Sept 19 PR Decision Trees (slides) TM Chap. 3
HTF Chap. 9 HW 1 due/
HW2 out

Sept 24 PR Naive Bayes, Generative vs Discriminative (slides) CB Chap. 4

Sept 26 PR Neural Networks and Deep Learning (slides) CB Chap. 5
KM Chap. 28

Oct 1 PR Neural Networks and Deep Learning, I, II (slides, notes) CB Chap. 5
KM Chap. 28

Oct 3 ZB Support Vector Machines 1 (slides, notes) CB Chap. 6, 7 HW 2 due/
HW3 out

Oct 8 ZB Support Vector Machines 2 (slides, notes) CB Chap. 6, 7

Oct 10 ZB Boosting, Surrogate Losses, Ensemble Methods (slides, notes) HTF Chap 10

Module: Unsupervised Learning

Oct 15 ZB Clustering, Kmeans (slides, notes) HTF Chap. 14.1-14.3

Oct 17 ZB Clustering: Mixture of Gaussians, Expectation Maximization (slides, notes) CB Chap 9 HW 3 due

Oct 22 Midterm 17:00 - 19:00 Location: Rashid Auditorium

Module: Theoretical considerations

Oct 24 PR Generalization, Model Selection (slides) HTF Chap. 7 Project topic selection

Oct 29 PR Learning Theory: Statistical Guarantees for Empirical Risk Minimization (slides)

Module: Representation Learning

Oct 31 Guest Representation Learning: Feature Transformation, Random Features, PCA (slides) HTF Chap. 14.5 HW4 out

Nov 5 PR Representation Learning: PCA Continued, ICA (slides) HTF Chap. 14.7

Nov 7 PR Graphical Models: Representation (slides) KM Chap. 10, 19, 20 Project Proposal Submission

Module: Graphical and Sequence Models

Nov 12 PR Graphical Models: Inference (slides) KM Chap. 10, 19, 20

Nov 14 PR Graphical Models: Learning (slides) KM Chap. 10, 19, 20 HW 4 due

Nov 19 ZB Sequence Models: HMMs (slides) KM Chap. 17

Nov 21 No Class, Thanksgiving

Nov 26 BOTH Industry lecture

Nov 28 ZB Sequence Models: State Space Models, other time series models (slides1, slides2) KM Chap. 18

Module: Actions

Dec 3 ZB Reinforcement Learning (slides1) TM Chap 13

Dec 5 Exam 2

Dec 10 Final Project Presentations

Dec 13 Final Projects Due

Homeworks

Projects

Climate Data

Forensic Investigation

Music Generation

Single Cell Analysis

Social Circle Analysis

Speech Recognition

Identifying Duplicate Questions