CS 461: Machine Learning Principles

Instructor: Karl Stratos (karl.stratos@rutgers.edu)

Course description. This course is a rigorous introduction to machine learning aimed at advanced undergraduate students in computer science, mathematics, and statistics.

What to expect. Machine learning is a fusion of ideas from multiple disciplines, such as computer science, optimization, and statistics, with the goal of characterizing and automating the process of generalization. Because it builds on so many technical skills, all the way from formal proofs to hacking, many students are overwhelmed and even upset by the amount of materials they are expected to follow when they are first exposed to the subject (I call this phenomenon "ML hazing"). If you find yourself feeling this way, know that you are not alone. Also know that you will not master this vast subject through this course alone; to do that, you will need years of experience. That said, the course is designed to provide a solid starting point by focusing on fundamental topics.

Goals.
  1. Understanding the goals, capabilities, and principles of machine learning
  2. Acquiring mathematical tools to formalize machine learning problems
  3. Acquiring implementation skills to build practical machine learning systems
Audience and prerequisites. No previous exposure to machine learning is assumed. However, the course will be most beneficial for students with some programming experience and familiarity with basic concepts in probability and statistics, calculus, and linear algebra. Examples of such concepts include More specifically, prerequisites are as follows:
  • Required: M250 (linear algebra), 112 (data structures), 206 (discrete II)
  • Recommended: M251 (multivariable calculus)
  • Alternatives to 206: M477 (probability), S379 (basic probability theory), or instructor's permission
There will be an entrance quiz on the first day of the class to help you evaluate whether you have the necessary background.

Grading.
  1. Assignments: 50%
  2. Entrance quiz: 5%
  3. Quizzes (excluding the entrance quiz): 15%
  4. Final: 30%
The assignment report must be written in LaTeX using a provided assignment report template. If you have never used LaTeX before, you can pick it up quickly (tutorial, style guide).

Academic integrity policy.
  1. Assignments: collaboration is allowed and encouraged, as long as you (1) write your own solution entirely on your own, and (2) specify names of student(s) you collaborated with in your writeup. If you find a solution online, clearly acknowledge the source and still write your own solution entirely on your own. Copying solutions from others or from the internet is strictly prohibited.
  2. Quizzes and final: cheating is strictly prohibited.
If the student is caught in cheating/plagiarism, the incident will be reported to the office of student conduct and he/she will get zero point for the assignment/quiz/exam, which will result in a low final grade or a fail.

Online textbooks (for optional reading).
  1. Pattern Recognition and Machine Learning (Bishop, 2006)
  2. Machine Learning: A Probabilistic Perspective (Murphy, 2012)
  3. Foundations of Machine Learning (Mohri, Rostamizadeh, and Talwalkarby, 2018)
Tentative plan.
Week Topics
1 General introduction, review of prerequisites
2 Regression, nearest neighbors, risk, least squares, empirical risk minimization
3 Linear regression, maximum likelihood estimation, gradient descent
4 Generlized linear regression, overfitting, regularization
5 Error decomposition, bias-variance tradeoff
6 Classification, logistic regression
7 Stochastic gradient descent, perceptron, classification losses
8 Large margin learning, representer theorem
9 Support vector machines (SVMs), kernel machines, duality
10 Decision trees for regression and classification
11 Ensemble methods, boosting, gradient boosting
12 Generative classifiers, mixture models
13 Latent-variable models, expectation maximization
14 Deep learning, backpropagation
15 Other topics in machine learning, review