It is hard to imagine anything more fascinating than automated systems that improve their own performance. The study of learning from data is commercially and scientifically important. This course is designed to give a graduate-level student a thorough grounding in the methodologies, technologies, mathematics and algorithms currently needed by people who do research in learning and data mining or who may need to apply learning or data mining techniques to a target problem.

The topics of the course draw from classical statistics, from machine learning, from data mining, from Bayesian statististics and from statistical algorithmics.

Students entering the class with a pre-existing working knowledge of probability, statistics and algorithms will be at an advantage, but the class has been designed so that anyone with a strong numerate background can catch up and fully participate.

Class lectures: Tuesdays & Thursdays 10:30am-11:50am, Wean Hall 7500 starting on Thursday September 4th, 2003

Review sessions: Thursdays 5:00pm- 6:15pm, Newell Simon Hall 1305 starting on Thursday September 11st, 2003 (details)

Instructors:

• Tom Mitchell, Wean Hall 5309, x8-2611, Office hours: by appointment through Sharon Woodside, sharon.woodside@cs.cmu.edu
• Andrew Moore, NSH 3117, x8-7599, Office hours: by appointment through Jean Harpley, jean@cs.cmu.edu

Teaching Assistants:

• Ning Hu, Wean Hall 3711, x8-1557, Office hours: Fridays 3:00pm-5:00pm
• Jiayong Zhang, NSH 4108, x8-8461, Office hours: Tuesdays 2:00pm-4:00pm
• Rong Zhang, Wean Hall 5302, Office hours: Wednesdays 2:00pm-4:00pm

Textbook:

• Primary textbook: Machine Learning, Tom Mitchell.
• Recommended (optional) textbook: Pattern Classification (2nd Edition), Duda, Hart and Stork.
• Recommended (optional) textbook: Neural Networks for Pattern Recognition, Chris Bishop.

Course Website (this page):

• http://www.cs.cmu.edu/afs/cs.cmu.edu/project/theo-20/www/mlc/index.html

Grading:

• Final grades will be based on midterm (25%), homework (5 assignments and 2 miniprojects, 35%), and final exam (40%)

Policy on late homework:

• Homework is worth full credit at the beginning of class on the due date.
• It is worth half credit for the next 48 hours.
• It is worth zero credit after that.
• You must turn in at least 6 of the 7 homework (5 assignments and 2 miniprojects), even if for zero credit, in order to pass the course.
• Free exemption: We will ignore your lowest homework grade for the semester.

Policy on collaboration:

• You may wish to discuss the homework with other students. If you like, you may form groups of two students and turn in one homework solution with up to two names on it. (Of course collaboration on exams is cheating and grounds for immediate failure and worse!)

Homework assignments

• HW1: Decision Trees. Out Sep 18th, due 10:30am Thursday October 2nd (duration 2 weeks).
• Instructions (PDF / PostScript), data for the first experiment, data for the second experiment (training set, test set).
• Or you can get the zipped package with all the files included.
• Solutions in PDF or PostScript.
• For questions and clarifications, contact Rong Zhang.
• HW2: Neural Nets and Cross-validation. Out Oct 2nd, due 10:30am Oct 14th (duration 12 days).
• Instructions in PDF or PostScript. (Updated Oct 3rd)
• Or you can get the LaTeX source.
• Solutions in PDF or PostScript.
• For questions and clarifications, contact Jiayong Zhang.
• HW3: Miniproject on Text Classification or Face Recognition. Out Oct 14th, due 10:30am Oct 28th (duration 2 weeks).
• Instructions in PDF or PostScript or LaTex.
• TEXT data and NaiveBayes code.
• FACE data and BackProp code.
• For questions and clarifications, contact Jiayong Zhang.
• HW4: VC-dimension, SVM, and/or KNN. Out Oct 28th, due 10:30am Nov 4th (duration 1 week).
• Instructions in PDF, PostScript or MSWord.
• Data files 1d.clean and 1d.noise.
• Matlab script svm.m.
• Solutions in PDF.
• For questions and clarifications, contact Rong Zhang.
• HW5: Bayes Nets. Out Nov 4th, due Nov 11th (duration 1 week).
• Instructions in PDF, PostScript or LaTex Source.
• Example C code example.c.
• Solutions in PDF or PostScript.
• For questions and clarifications, contact Jiayong Zhang.
• HW6: Miniproject (may be on any ML topic and data set). Out Nov 11th, due 10:30am Nov 25th (duration 2 weeks).
  • Instructions in PDF, PostScript or LaTex Source.
  • For questions and clarifications, contact Ning Hu.
• HW7: GMM, K-means, HMM, and MDP. Out Nov 25th, due 10:30am Dec 4th (duration 9 days).
  • Instructions in PDF, PostScript or LaTex Source.
  • Solutions in PDF or PostScript.
  • For questions and clarifications, contact Ning Hu.

Lecture schedule (and online slides if available)

Review sessions