MATH 110B LEC A: OPTIMIZATION II (44715)

Class Homepage

Class Information:

Lectures: MWF 9:00 -- 9:50am. @ ALP 3610.

Final: Wed, Jun 12, 8:00am -- 10:00am. @ ALP 3610. [NO FINAL]

 

The lectures will be focusing on half classical optimizations and half on the emerging optimization methods. The classical optimization part will be mostly on duality in linear and nonlinear problems, augmented Lagrangian method, the interior method will be introduced, but not required. Applications will be discussed. Coding for this part will not require implementation, packages are allowed.

For midterm, it will be on the duality part mostly.

The next things are the stochastic gradient descent, we will start from the some gradient descent methods and spend some time on the basic things of probability.

The last part of the lectures will be the regression/classification, introduce some methods including the neural networks. For all projects, the GPU computation will not be required.

 

Class Resources:

Course Schedule (Tentative, updated April 19th):

[Lecture 1] Duality in Linear Programming I             [
    

Download Link][Ch17]
[Lecture 2] Duality in Linear Programming II            [
    

Download Link][Ch17]
[Lecture 3] Understanding Duality I                     [
    

Download Link][Ch17]
[Lecture 4] Understanding Duality II                    [
    

Download Link][Ch17][HW 1][
    

Download ref] 
[Lecture 5] Nonlinear problem, equality constraint I    [
    

Download Link][Ch20]      [
    

Download ref]
[Lecture 6] Nonlinear problem, equality constraint II   [
    

Download Link][Ch20][Prj1]
[Lecture 7] Nonlinear problem, KKT condition            [
    

Download Link][Ch21][HW 2]
[Lecture 8] Methods, nonlinear constrained problem I    [
    

Download Link][Ch23]      [
    

Download ref]
[Lecture 9] Methods, nonlinear constrained problem II   [
    

Download Link][Ch23]      [
    

Download ref][
    

Download ref]
[Lecture 10] Methods, nonlinear constrained problem III [
    

Download Link][Ch23]      [
    

Download ref]
[Lecture 11] L1 optimization (LASSO, sparsity)          [
    

Download Link][Code]      [refs]
[Lecture 12] Review I                                   [
    

Download Link]
[Lecture 13] Review II                                  [
    

Download Link]
[Lecture 14] Midterm                                    [
    

Download Link]
[Lecture 15] Stochastic gradient methods I              [
    

Download Link][Code
    

Links to an external site.][Prj2][
    

Download ref][
    

Download ref][
    

Download ref][
    

Download ref] 
[Lecture 16] Stochastic gradient methods II             [
    

Download Link][----][Fprj][ref
    

Links to an external site.][ref
    

Links to an external site.][ref
    

Links to an external site.] 
[Lecture 17] Stochastic gradient methods III            [Link][Code]      [ref][ref] 
[Lecture 18] Stochastic gradient methods IV             [Link][Code]      [ref][ref] 
[Lecture 19] Perceptron Model                           [Link][Code
    

Links to an external site.]      [ref
    

Links to an external site.][ref
    

Links to an external site.] 
[Lecture 20] Multilayer Perceptron Model and BP         [Link][Code]      [ref
    

Links to an external site.][ref
    

Links to an external site.][ref
    

Links to an external site.][ref
    

Links to an external site.]  
[Lecture 21] Linear SVM                                 [Link][Code]      [ref
    

Links to an external site.][ref
    

Links to an external site.][ref
    

Links to an external site.][ref
    

Links to an external site.]      
[Lecture 22] Nonlinear SVM                              [Link][Code] 
[Lecture 23] Decision Tree                              [Link][Code]      [ref
    

Links to an external site.][ref
    

Links to an external site.][ref
    

Links to an external site.]
[Lecture 24] Convolution Neural Network                 [
    

Download Link][Code]      [ref
    

Links to an external site.]