View on GitHub

CS260 [Fall2021] Machine Learning Algorithms


This course introduces the foundational theory and algorithms of machine learning. The goal of this course is to endow the student with a) a solid understanding of the foundational concepts of machine learning, and b) the ability to derive and analyze machine learning algorithms. Topics to be covered include empirical risk minimization, PAC learning, Agnostic PAC learning, perceptron, linear regression, nearest neighbors, decision trees, boosting, structural risk minimization, surrogate loss functions, stochastic gradient descent, support vector machines, kernel methods, multi-class classification, and clustering etc. Slides and homework assignments will be released on CCLE and this website. Homework solutions will only be released on CCLE.


Two years of college mathematics, including calculus, linear algebra, probability and statistics, and the ability to write computer programs.


Shalev-Shwartz, Shai, and Shai Ben-David. Understanding machine learning: From theory to algorithms. Cambridge University Press, 2014.

Programming Language



Grading Policy

Grades will be computed based on the following factors:


# Date Topics Reading Homework
1 9/27 Introduction (slides)(slides_annotated) Chapter 1, 2.1  
2 9/29 Empirical Risk Minimization, PAC Learning (slides)(slides_annotated) Chapter 2 HW1 Out
  10/1 TA Session Week 1 (1A slides)(1B slides)(1C slides)    
3 10/4 Agnostic PAC Learning (slides)(slides_annotated) Chapter 3  
4 10/6 Uniform Convergence (slides)(slides_annotated) Chapter 4  
  10/8 TA Session Week 2 (1A slides)(1B slides)(1C slides)    
5 10/11 Bias-Complexity Tradeoff (slides)(slides_annotated) Chapter 5, 11 HW1 Due, HW2 Out
6 10/13 VC dimension (slides) (slides_annotated) Chapter 6  
  10/15 TA Session Week 3 (1A slides)(1B slides)(1C slides)    
7 10/18 VC dimension Cont. (slides) (slides annotated) Chapter 6, 28  
8 10/20 Nonuniform Learnability (slides) (slides annotated) Chapter 7  
  10/22 TA Session Week 4 (1A slides)(1B slides)(1C slides)    
9 10/25 Perceptron/Linear regression (slides)(slides_annotated) Chapter 9 HW2 Due, HW3 Out
10 10/27 Nearest Neighbors (slides) (slides annotated Chapter 19  
  10/29 TA Session Week 5 (1A slides)(1B slides)(1C slides)    
11 11/1 Decision Trees (slides) (slides annotated) Chapter 18 Project Proposal Due
12 11/3 Boosting (slides)(slides annotated) Chapter 10  
  11/5 TA Session Week 6 (slides on CCLE-Week6) (1A slides)(1B slides)    
  11/8 Midterm Exam    
13 11/10 Convex Learning and SGD (slides)(slides annotated) Chapter 12, 14 HW3 Due, HW4 Out
  11/12 TA Session Week7 (1A slides)(1B slides)(1C slides)    
14 11/15 Convex Learning and SGD (slides) (slides annotated) Chapter 12, 14  
15 11/17 Regularization Stability (slides) (slides annotated) Chapter 13  
  11/19 TA Session Week 8 (1A slides)(1B slides)(1C slides)    
16 11/22 Support Vector Machines (slides) (slides annotated) Chapter 15 HW4 Due, HW5 Out
17 11/24 Kernel Methods (slides) (slides annotated) Chapter 16  
  11/26 Thanksgiving holidays    
18 11/29 Multi-class Classification (slides) (slides annotated) Chapter 17  
19 12/1 Clustering (slides) (slides annotated) Chapter 22  
  12/3 TA Session Week10 (1A slides)(1B slides)(1C slides)    
  12/6     HW5 Due
  12/8 Final Project Presentation    
  12/12     Project Report/Slides Due

Academic Integrity Policy

Students are encouraged to read the UCLA Student Conduct Code for Academic Integrity.


There will be about 5 homework assignments during the semester as we cover the corresponding material. Homework consists of both mathematical derivation, algorithm analysis and programming. Homework is required to be written in Latex. Latex homework template can be found here. The lowest homework score will be dropped for you.

Unless otherwise indicated, you may talk to other students about the homework problems but each student must hand in their own answers and write their own code in the programming part. You also must indicate on each homework with whom you collaborated and cite any other sources you use including Internet sites. Students cannot use old solution sets for this class or solution manual to the textbook under any circumstances.

Homework assignments will be submitted through Gradescope. You should have received an invite to Gradescope after you get enrolled in this class. Login via the invite, and submit the homework assignments on time.

Please submit your homework on time. Homework is worth full credit before the due date. It is worth zero credit after the due date.


There will be one midterm. The exam is a take-home, 24-hours, open-book exam. If you need a makeup exam, please email us by Nov 1st.


There will be 6 in-class pop-up quiz for the purpose of reviewing the newly learned concepts. The quizzes are closed book and closed notes. No electronic aids or cheat sheets are allowed. We will drop the lowest quiz score for you.


Students are required to do a project in this class. The goal of the course project is to provide the students an opportunity to explore research directions in optimization or machine learning. Therefore, the project should be related to the course content. An expected project consists of • A novel and sound solution to an interesting problem • Comprehensive literature review and discussion • Thorough theoretical/experimental evaluation and comparisons with existing approaches

The best outcome of the project is a manuscript that is publishable in major machine learning conferences (COLT, ICML, NeurIPS, ICLR, AISTATS, UAI etc.) or journals (Journal of Machine Learning Research, Machine Learning).

Instruction can be found here, and template for proposal and final report can be found here.

Please refer to syllabus for more details.