Overview
This course introduces the foundational theory and algorithms of machine learning. The goal of this course is to endow the student with a) a solid understanding of the foundational concepts of machine learning, and b) the ability to derive and analyze machine learning algorithms. Topics to be covered include empirical risk minimization, PAC learning, Agnostic PAC learning, perceptron, linear regression, nearest neighbors, decision trees, boosting, structural risk minimization, surrogate loss functions, stochastic gradient descent, support vector machines, kernel methods, multi-class classification, and clustering etc. Slides and homework assignments will be released on CCLE and this website. Homework solutions will only be released on CCLE.
Prerequisites
Two years of college mathematics, including calculus, linear algebra, probability and statistics, and the ability to write computer programs.
Textbook:
Shalev-Shwartz, Shai, and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge University Press, 2014.
Logistics
- Time: Monday and Wednesday 2:00PM - 3:50PM
- Location: Online
- Instructor: Quanquan Gu (Email: qgu at cs dot ucla dot edu)
-
Teaching Assistant: Yewen Wang (Email: wyw10804 at gmail dot com)
- Office hours:
- The instructor’s office hour is Thursday 9:30am-10:30am, online.
- The TA’s office hour is Tuesday 9:00am-11:00am, online
- Course Website: https://uclaml.github.io/CS260-Spring2020/
- Course Forum: piazza.com/ucla/spring2020/cs260/home (If you haven’t already, sign up here.)
Grading Policy
Grades will be computed based on the following factors:
- Homework 35%
- Quiz 5%
- Midterm 30%
- Final Project 30%
Schedule
| # | Date | Topics | Reading | Homework |
|---|---|---|---|---|
| 1 | 3/30 | Introduction (slides) | Chapter 1, 2.1 | HW1 Out |
| 2 | 4/1 | Empirical Risk Minimization, PAC Learning (slides)(slides_annotated) | Chapter 2 | |
| 4/3 | TA Session Week 1 (slides) | |||
| 3 | 4/6 | Agnostic PAC Learning (slides)(slides_annotated) | Chapter 3 | |
| 4 | 4/8 | Uniform Convergence (slides)(slides_annotated) | Chapter 4 | |
| 4/10 | TA Session Week 2 (slides) | |||
| 5 | 4/13 | Bias-Complexity Tradeoff (slides)(slides_annotated) | Chapter 5, 11 | |
| 6 | 4/15 | VC dimension (slides) (slides_annotated) | Chapter 6 | HW1 Due, HW2 Out |
| 4/17 | TA Session Week 3 (slides) | |||
| 7 | 4/20 | VC dimension Cont. (slides) (slides annotated) | Chapter 6, 28 | |
| 8 | 4/22 | Nonuniform Learnability (slides) (slides annotated) | Chapter 7 | |
| 4/24 | TA Session Week 4 (slides) | |||
| 9 | 4/27 | Perceptron/Linear regression (slides)(slides_annotated) | Chapter 9 | |
| 10 | 4/29 | Nearest Neighbors (slides) (slides annotated) | Chapter 19 | HW2 Due, HW3 Out |
| 5/1 | TA Session Week 5 (slides) | |||
| 11 | 5/4 | Decision Trees (slides) (slides annotated) | Chapter 18 | Project Proposal Due |
| 12 | 5/6 | Boosting (slides)(slides annotated) | Chapter 10 | |
| 5/8 | TA Session Week6 (slides on CCLE-Week6) | |||
| 5/11 | Midterm Exam | |||
| 13 | 5/13 | Convex Learning and SGD (slides)(slides annotated) | Chapter 12, 14 | |
| 5/15 | TA Session Week7 (slides on CCLE-Week7) | |||
| 14 | 5/18 | Regularization Stability (slides) (slides annotated) | Chapter 13 | HW3 Due, HW4 Out |
| 15 | 5/20 | Support Vector Machines (slides) (slides annotated) | Chapter 15 | |
| 5/22 | TA Session Week8 (slides) | |||
| 5/25 | Memorial Day Holiday | HW4 Due, HW5 Out | ||
| 16 | 5/27 | Kernel Methods (slides) (slides annotated) | Chapter 16 | |
| 5/29 | TA Session Week9 (slides) | |||
| 17 | 6/1 | Multi-class Classification (slides) (slides annotated) | Chapter 17 | |
| 18 | 6/3 | Clustering (slides) (slides annotated) | Chapter 22 | |
| 6/5 | TA Session Week10 (slides) | HW5 Due | ||
| 6/10 | Final Project Presentation | |||
| 6/11 | Project Report Due |
Academic Integrity Policy
Students are encouraged to read the UCLA Student Conduct Code for Academic Integrity.
Homework
There will be about 6 homework assignments during the semester as we cover the corresponding material. Homework consists of both mathematical derivation, algorithm analysis and programming. Homework is required to be written in Latex. Latex homework template can be found here. The lowest homework score will be dropped for you.
Unless otherwise indicated, you may talk to other students about the homework problems but each student must hand in their own answers and write their own code in the programming part. You also must indicate on each homework with whom you collaborated and cite any other sources you use including Internet sites. Students cannot use old solution sets for this class or solution manual to the textbook under any circumstances.
Homework assignments will be submitted through Gradescope. You should have received an invite to Gradescope after you get enrolled in this class. Login via the invite, and submit the homework assignments on time.
Please submit your homework on time. Homework is worth full credit before the due date. It is worth zero credit after the due date.
Exam
There will be one midterm. The exam is a take-home, 24-hours, open-book exam. If you need a makeup exam, please email us by May 1st.
Quiz
There will be 6 in-class pop-up quiz for the purpose of reviewing the newly learned concepts. The quizzes are closed book and closed notes. No electronic aids or cheat sheets are allowed. We will drop the lowest quiz score for you.
Project
Students are required to do a project in this class. The goal of the course project is to provide the students an opportunity to explore research directions in optimization or machine learning. Therefore, the project should be related to the course content. An expected project consists of • A novel and sound solution to an interesting problem • Comprehensive literature review and discussion • Thorough theoretical/experimental evaluation and comparisons with existing approaches
The best outcome of the project is a manuscript that is publishable in major machine learning conferences (COLT, ICML, NeurIPS, ICLR, AISTATS, UAI etc.) or journals (Journal of Machine Learning Research, Machine Learning).
Instruction can be found here, and template for proposal and final report can be found here.
Please refer to syllabus for more details.