Fall 2014
Foundations of Machine Learning
Course#: CSCI-GA.2566-001
Instructor: Mehryar Mohri
Graders/TAs:
Vitaly Kuznetsov,
Andres Munoz Medina,
Scott Yang.
Mailing
List
Course Description
This course introduces the fundamental concepts and methods of machine
learning, including the description and analysis of several modern
algorithms, their theoretical basis, and the illustration of their
applications. Many of the algorithms described have been successfully
used in text and speech processing, bioinformatics, and other areas in
real-world products and services. The main topics covered are:
- Probability tools, concentration inequalities
- PAC model
- Rademacher complexity, growth function, VC-dimension
- Perceptron, Winnow
- Support vector machines (SVMs)
- Kernel methods
- Decision trees
- Boosting
- Density estimation, maximum entropy models
- Logistic regression
- Regression problems and algorithms
- Ranking problems and algorithms
- Halving algorithm, weighted majority algorithm, mistake bounds
- Learning automata and transducers
- Reinforcement learning, Markov decision processes (MDPs)
It is strongly recommended to those who can to also attend
the
Machine Learning Seminar. Those interested in further
pursuing the study of machine learning could also attend
the Advanced
Machine Learning class.
Location and Time
Warren Weaver Hall Room 109,
251 Mercer Street.
Mondays 5:10 PM - 7:00 PM.
Prerequisite
Familiarity with basics in linear algebra, probability, and analysis
of algorithms.
Projects and Assignments
There will be 3 to 4 assignments and a project. The final grade is
essentially the average of the assignment and project grades. The
standard high level of integrity
is expected from all students, as with all CS courses.
Lectures
- Lecture 00: Introduction to convex optimization.
- Lecture 01: Introduction to machine learning, basic definitions, probability tools.
- Lecture
02: PAC model, guarantees for learning with finite hypothesis sets.
- Lecture
03: Rademacher complexity, growth function, VC-dimension, learning
bounds for infinite hypothesis sets.
- Lecture
04: Support vector machines (SVMs), margin bounds.
- Lecture 05: Kernel methods.
- Lecture 06: Boosting.
- Lecture 07: Density estimation, Maxent models, multinomial logistic regression.
- Lecture 08: On-line learning.
- Lecture 09: Ranking.
- Lecture 10: Multi-class classification.
- Lecture 11: Regression.
- Lecture 12: Reinforcement learning.
- Lecture 13: Learning languages.
Textbook
The following is the required textbook for the class. It covers all
the material presented (and a lot more):
Technical Papers
An extensive list of recommended papers for further reading is
provided in the lecture slides.
Homework
Previous years