Syllabus >

Grading and
other important
information >

Homework >

DS-GA 1014: Optimization and Computational Linear Algebra for Data Science
(Fall 2018, NYU CDS)

Afonso S. Bandeira
bandeira [at] cims [dot] nyu [dot] edu

Lectures: Tue 11am-12.40pm at CDS(60 5th Av.) room 150

Afonso's Office Hours: Tue 1.45pm-2.45pm at
CDS(60 5th Av.)603  (or by appointment)
Afonso has sporadic extra office hours on Wednesdays at WWH1123, announced weekly

Section Leader: Brett Bernstein
(Section material).
Sections: W 11am-11.50am at SILV 405
Brett's Office hours: W 1.30pm-2.30pm
at CDS 650 (or by appointment)
Graders (email address; add [at] nyu [dot] edu): Efe Onaran (eo766), Ruofan Wang (rw2268), Luca Venturi (lv800).

Piazza page for this course here.

Syllabus: This course will cover the basics of optimization and computational linear algebra used in Data Science. Contents: Vector spaces and linear transformations: rank, dimension, etc. Linear systems: conditioning, least squares. Singular-value decomposition/principal-component analysis, Rayleigh quotients. Applications: Spectral clustering, dimension reduction, Page Rank. Local optima and global optima. Constrained optimization. Optimality conditions and matrix calculus. Gradient descent and Stochastic Gradient Descent. Newton's method and Quasi-Newton methods (BFGS and L-BFGS). Linear Optimization, Duality, and Convex optimization. Conjugate Gradient. Some Applications: Lasso, compressed sensing. Problems on Graphs.
Important: This course description is preliminary, and so it is subject to change.
I am here to help: If you have any question, comment, feedback, want to brainstorm about any research idea, etc, just email me and we'll schedule a time to meet.
Feedback: Also, if you have any comment or feedback on the class (it's going too fast, too slow, you want me to cover more of something, or less of something else, etc) please let me know (in person or through email) or submit a comment to this google form. Having direct feedback from you is the best way for me to try give lectures that you like! (keep in mind that I don't know who sent me the comment or feedback and there is no way for me to answer, for questions use email instead).

Books: All three books are optional
  • Strang: Introduction to Linear Algebra (there are very good lecture videos based on this book)
  • Nocedal & Wright: Numerical Optimization (should be available online via NYU here)
  • Boyd & Vandenberghe: Convex Optimization (available online here)
Grading and other important information:
  • Grading: 40% Homework, 20% Midterm (Nov 6th in class), 40% Final. Exams are open book/notes.

Homework (typically weekly, due on Tuesday before class at the CDS front desk):

Extended Syllabus: