Class Homepage
Class Information:
Lectures: MWF 9:00 -- 9:50am. @ ALP 3610.
Final: Wed, Jun 12, 8:00am -- 10:00am. @ ALP 3610. [NO FINAL]
The lectures will be focusing on half classical optimizations and half on the emerging optimization methods. The classical optimization part will be mostly on duality in linear and nonlinear problems, augmented Lagrangian method, the interior method will be introduced, but not required. Applications will be discussed. Coding for this part will not require implementation, packages are allowed.
For midterm, it will be on the duality part mostly.
The next things are the stochastic gradient descent, we will start from the some gradient descent methods and spend some time on the basic things of probability.
The last part of the lectures will be the regression/classification, introduce some methods including the neural networks. For all projects, the GPU computation will not be required.
Class Resources:
- Syllabus page.
- Google Colab: https://colab.research.google.com/ Links to an external site.
- Kaggle competition: https://www.kaggle.com/competitions Links to an external site.
- l1 minimization weblog: http://www.pyrunner.com/weblog/2016/05/26/compressed-sensing-python/ Links to an external site.
- python packages for optimization (updating): https://colab.research.google.com/drive/1bFcr1J0a6Euwp09vY_IID-LRxCKCdm_I Links to an external site.
- SGD demo-1 (updating): https://colab.research.google.com/drive/1ieBQS_lgL4W-MazfrwcmAgeeQGfCBXNd Links to an external site.
- Perceptron example: https://colab.research.google.com/drive/15vee5Ze3GT7W2NnLcLDw0Nyk4c0STmNs Links to an external site.
- Stanford cs231: http://cs231n.github.io/ Links to an external site.
- Solutions to Stanford cs231: https://github.com/MahanFathi/CS231/ Links to an external site.
- An example for SVM and FC: https://cs.stanford.edu/people/karpathy/cs231nfiles/minimal_net.html Links to an external site.
Course Schedule (Tentative, updated April 19th):
[Lecture 1] Duality in Linear Programming I [Link
Download Link][Ch17]
[Lecture 2] Duality in Linear Programming II [Link
Download Link][Ch17]
[Lecture 3] Understanding Duality I [Link
Download Link][Ch17]
[Lecture 4] Understanding Duality II [Link
Download Link][Ch17][HW 1][ref
Download ref]
[Lecture 5] Nonlinear problem, equality constraint I [Link
Download Link][Ch20] [ref
Download ref]
[Lecture 6] Nonlinear problem, equality constraint II [Link
Download Link][Ch20][Prj1]
[Lecture 7] Nonlinear problem, KKT condition [Link
Download Link][Ch21][HW 2]
[Lecture 8] Methods, nonlinear constrained problem I [Link
Download Link][Ch23] [ref
Download ref]
[Lecture 9] Methods, nonlinear constrained problem II [Link
Download Link][Ch23] [ref
Download ref][ref
Download ref]
[Lecture 10] Methods, nonlinear constrained problem III [Link
Download Link][Ch23] [ref
Download ref]
[Lecture 11] L1 optimization (LASSO, sparsity) [Link
Download Link][Code] [refs]
[Lecture 12] Review I [Link
Download Link]
[Lecture 13] Review II [Link
Download Link]
[Lecture 14] Midterm [Link
Download Link]
[Lecture 15] Stochastic gradient methods I [Link
Download Link][Code
Links to an external site.][Prj2][ref
Download ref][ref
Download ref][ref
Download ref][ref
Download ref]
[Lecture 16] Stochastic gradient methods II [Link
Download Link][----][Fprj][ref
Links to an external site.][ref
Links to an external site.][ref
Links to an external site.]
[Lecture 17] Stochastic gradient methods III [Link][Code] [ref][ref]
[Lecture 18] Stochastic gradient methods IV [Link][Code] [ref][ref]
[Lecture 19] Perceptron Model [Link][Code
Links to an external site.] [ref
Links to an external site.][ref
Links to an external site.]
[Lecture 20] Multilayer Perceptron Model and BP [Link][Code] [ref
Links to an external site.][ref
Links to an external site.][ref
Links to an external site.][ref
Links to an external site.]
[Lecture 21] Linear SVM [Link][Code] [ref
Links to an external site.][ref
Links to an external site.][ref
Links to an external site.][ref
Links to an external site.]
[Lecture 22] Nonlinear SVM [Link][Code]
[Lecture 23] Decision Tree [Link][Code] [ref
Links to an external site.][ref
Links to an external site.][ref
Links to an external site.]
[Lecture 24] Convolution Neural Network [Link
Download Link][Code] [ref
Links to an external site.]