|Time and Location||M 9:30-11:30am, Room 4419|
Prof. Liang Huang (huang @ cs.qc), Instructor|
Kai Zhao (z.kaayy @ gmail), TA
|LH: M 11:30-12pm, CS Lab |
KZ: M 3-4pm & F 4-5pm, CS Lab
Additional office hours available before exams.
|Textbooks||Unlike other fields of CS, there isn't a standard textbook in machine learning because, (1) the field is developing so fast that classical textbooks become outdated constantly, and (2) most recent textbooks were written from a statistical or engineering perspective (rather than a CS one), which makes it unnecessarily complicated.
In light of this I recommend (for references only; this course is self-contained):
How can we make computers to act without being explicitly programmed and to improve with experience?
In the past decade, machine learning has given us self-driving cars, practical speech recognition, effective web search, accurate spam filters, and a vastly improved understanding of the human genome. Machine learning is so pervasive today that you probably use it dozens of times a day without knowing it.
This course will survey the most important algorithms and techniques in the field of machine learning. The treatment of math will be rigorous, but unlike most other machine learning courses which focus on the statistical or engineering perspective with tons of equations, my course will focus on the geometric intuitions and the algorithmic perspective. I will try my best to visualize every concept.
|1||Jan 28||Intro to ML|
Intro to Python and numpy (Kai)
|2||Feb 4||slides for lectures 2-4: Online Learning: Perceptron and Beyond.
Linear separation; geometric computation of margin.
|3||Feb 11||Perceptron Convergence Proof; geometry vs. algebra|
Voted and Averaged Perceptron
MIRA (geometric derivation)
|President's Day. Class moved to Wednesday.
||4||W Feb 20||
passive-aggressive; p-aggressive MIRA|
demo of perceptron vs. aggressive MIRA
multiclass decision boundaries (geometry)
|HW1 out: (avg) perceptron, (aggressive) MIRA, feature preprocessing.
|5||Feb 25||engineering tips: shuffling,
variable learning rate.|
feature preprocessing: rescaling/centering, binning, categorical to binary.
slides for lectures 5-6: Kernels and Kernelized Online Learning.
Non-linear features and feature combination; XOR
|6||Mar 4||kernelized perceptron in primal and dual (Kai)|
polynomial kernel on examples: XOR, circle.
|7||Mar 11||Gaussian kernel (distorts distance); geometric interpretation on the original and feature spaces; connection to instance-based learning (nearest neighbor)
Kernel properties: Mercer, gram matrix, positive semidefinite (eigenvalue); example of a non-kernel
slides for lectures 7-8: SVM and convex optimization.
SVM optimization in primal (without bias).
||8||Mar 18||SVM optimization in dual.|
Hildreth algorithm (Kai).
|HW2 out: kernelized perceptron and SVM.|
HW2 due on Friday 4/5.
|11||Apr 8||slides for Lectures 9-10: structured prediction.
From Multiclass to Structured Classification.
|12||Apr 15||Dynamic Programming. Viterbi vs. Dijkstra.|
Fast Averaging Trick (Daume, 2006).
Proof of Convergence.
Perceptron with Inexact Search: Early Update, Violation-Fixing, LaSO, etc.
(Collins and Roark:2004; Daume and Marcu, 2005; Huang et al 2012)
|13||Apr 22||Unsupervised Learning|
|project proposal due 4/24.
mixture of Gaussians
|HW3 out: structured perceptron w/ exact and inexact search.|
|Final Project Midway Presentations||17||May 20|