Optimization Methods for Data Science I

rsaab(-at-)math.ucsd.edu

Office: AP&M 5157

Office Hours:

Wednesday 5:00 pm - 6:00 pm (or by appointment)

j2pham(-at-)ucsd.edu

Office: APM 6414

Office Hours: Friday 4:00-6:00 pm

Ziyan Zhu

ziz276(-at-)ucsd.edu

Office: APM 1121

Office Hours: Thursday

Zi Yang

ziy109(-at-)ucsd.edu

Office: APM 1210

Office Hours: Tuesday 2:00-4:00 pm and Wednesday 1:00-3:00 pm

**Sept. 25**Welcome to Math 173A!**Sept. 27****My lecture notes:**Lecture notes 1. (May contain errors or typos, use at your own risk)**Oct. 5**The**Gradescope code**for this course is 9NRYRR**Oct. 8****My lecture notes:**Lecture notes 2. (May contain errors or typos, use at your own risk)**Oct. 13****By popular demand, you may use Python or R, in addition to Matlab, for the programming part of your HW.****Oct. 15****My lecture notes:**Lecture notes 3. (May contain errors or typos, use at your own risk)**Oct. 19****HW 2**solutions. (May contain errors or typos, use at your own risk)**Nov. 2****My lecture notes:**Lecture notes 4. (May contain errors or typos, use at your own risk)**Nov. 11****My lecture notes:**Lecture notes 4 (updated). (May contain errors or typos, use at your own risk)**Nov. 11****My lecture notes:**Lecture notes 5. (May contain errors or typos, use at your own risk)**Nov. 13****HW3 Solutions:**Homework 3 solutions. (May contain errors or typos, use at your own risk)**Nov. 14****HW2 Solutions:**Homework 2 solutions. (May contain errors or typos, use at your own risk)**Nov. 14****HW1 Solutions:**Homework 1 solutions. (May contain errors or typos, use at your own risk)**Nov. 14****HW4 Solutions:**Homework 4 solutions. (May contain errors or typos, use at your own risk)**Nov. 14****Midterm 1 Solutions:**MT 1 solutions. (May contain errors or typos, use at your own risk)**Nov. 14****Extra notes on steepest descent:**here . (May contain errors or typos, use at your own risk)**Nov. 14****Midterm 2 Solutions:**MT 2 solutions. (May contain errors or typos, use at your own risk)**Nov. 27****My lecture notes:**Lecture notes 6. (May contain errors or typos, use at your own risk)**Dec. 4 -- *updated* December 8.****My lecture notes:**Lecture notes 7. (May contain errors or typos, use at your own risk)**Dec. 7****HW5 Solutions:**Homework 5 solutions. (May contain errors or typos, use at your own risk)**Dec. 10****HW6 Solutions:**Homework 6 solutions. (May contain errors or typos, use at your own risk)**Dec. 10****Final Exam *Seating Chart*:**Here .

- Extra Problems 1(A). (May contain errors or typos, use at your own risk)
- Extra Problems 1(B). (May contain errors or typos, use at your own risk)
- Extra Problems 2(A). (May contain errors or typos, use at your own risk)
- Extra Problems 3(A). (May contain errors or typos, use at your own risk)
- Extra Problems 4. (May contain errors or typos, use at your own risk)
- Extra Problems 5-6(B). (May contain errors or typos, use at your own risk)
- Extra Problems 7(B). (May contain errors or typos, use at your own risk)

**Catalog Description.**

Introduction to convexity: convex sets, convex functions; geometry of hyperplanes; support functions for convex sets; hyperplanes and support vector machines. Linear and quadratic programming: optimality conditions; duality; primal and dual forms of linear support vector machines; active-set methods; interior methods. Prerequisites: MATH 20C or MATH 31BH and MATH 20F or 31AH. Students who have not completed listed prerequisites may enroll with consent of instructor.**Additional Resources.**

While there is no required textbook for the course, (parts of) the following books/resources may be useful:- For a good resource on SVM's and their primal/dual formulation, you may also refer to these lecture notes from A. Ng.
- For Constrained Optimization, Lagrange multipliers, and duality you may also refer to these lecture notes from Y. Singer.
- For Newton's Method, you may also refer to these lecture notes from from R. Freund.
- Boyd and Vandenberghe, Convex Optimization. Link
- Charles L. Byrne, A First Course in Optimization. Link
- Chong and Zak, Introduction to Optimization, Wiley, 2013
- Pedregal, Introduction to Optimization, Springer, 2006

**Matlab:**- Your best friend is the "help" command in MATLAB.
- An introduction to MATLAB is available here.
- You can get Matlab from here.