Class Homepage
Course Information
Lectures: MWF 10:00 -- 10:50am. @ ALP 2600.
Final: Mon, March 18, 10:30am -- 12:30pm. @ ALP 2600. [UPDATED March 9th, 2019]
Class Resources: [Last Updated March 19th, 2019]
- Math 110A Syllabus 2019W. A complete guidance of this class. Please read first. The downloadable version is at here Download here.
-
Link
Download Link to an electronic copy of the textbook: Numerical Optimization, J. Nocedal and S. Wright. Caution: This book has many typos, we will use the 2nd Edition instead of the 1st Edition. The erratas of 2nd Edition are listed in class files [2
Download 2, 3
Download 3] just in case if you are confused with something in book.
- Linear Algebra reviews: A few 3A course websites [2016f,2017w,2015f]
- Multivariable Calculus reviews (esp. Taylor expansion) : Find your calculus book. : )
- Python tutorials: Here are a few tutorials online from search engines.
- [Beginner] http://python.berkeley.edu/learn/ Links to an external site.
- [Beginner] https://learnpythonthehardway.org/ Links to an external site.
- [DataSci] https://jakevdp.github.io/PythonDataScienceHandbook/ Links to an external site.
- [DataSci] https://realpython.com/tutorials/data-science/ Links to an external site.
- Other optimization references:
- scipy.optimize tutorial: https://docs.scipy.org/doc/scipy-1.1.0/reference/tutorial/optimize.html Links to an external site.
- The Basics of Practical Optimization Download The Basics of Practical Optimization
-
Undergraduate Nonlinear Continuous Optimization Download Undergraduate Nonlinear Continuous Optimization
- Install Python3 [Anaconda Links to an external site.] on your own machine.
- Online Python REPL: https://repl.it/site/languages/python3 Links to an external site.
- Get used to Python: Project 0
Course Schedule:
-
Introduction to optimization
- [Lecture 1] Continuous and Discrete optimization, constrained and unconstrained optimization, global and local optimization. [Link Download Link][pp. 5-8]
- [Lecture 2] Optimization algorithms and convexity. [Link Download Link][pp.8-9, 12-17,19-21]
- [Lecture 3] Rates of convergence. [Link Download Link][pp. 619]
-
Line search method
- [Lecture 4] Step length and Wolfe Condition. [Link Download Link][pp. 30-35]
- [Lecture 5] Wolfe Condition, Cont'd. [Link Download Link][pp. 30-35]
- [Lecture 6] Convergence and Backtracking. [Link Download Link][pp. 37-41]
- [Lecture 7] Newton's method and convergence rate. [Link Download Link][pp. 44-46]
-
Trust region method
- [Lecture 8] Trust region method and Cauchy point related algorithms. [Link Download Link][pp. 67-69, 71-77]
- [Lecture 9] Global convergence and local convergence. [Link Download Link][pp. 77-79, 82-83, 93-95]
- [Lecture 10] Local convergence. [Link from Lecture 9][pp. 93-95]
- [Lecture 11] Review [Link Download Link, Link Download Link]
- [Lecture 12] Midterm #1 (closed book) [Link Download Link]
-
Conjugate gradient and Quasi-Newton methods
- [Lecture 13] Linear conjugate gradient method. [Link Download Link][pp. 102-111]
- [Lecture 14] Linear conjugate gradient method, Cont'd. [Link Download Link][Link Links to an external site.][pp. 111-113]
- [Lecture 15] BFGS and SR1 methods. [Link Download Link][pp. 136-146]
- [Lecture 16] Superlinear convergence. [Link Download Link][pp. 46-47, 149, 153-160]
-
Linear programming: Simplex method
- [Lecture 17] Basic properties of LP. [Link Download Link][pp. Reference book Download Reference book 305-330 ]
- [Lecture 18] Basic properties of LP, Cont'd. [Link Download Link][pp. same as LEC17, 330-334]
- [Lecture 19] Simplex method by hand and algorithm. [Link Download Link]
- [Lecture 20] Simplex method by hand and algorithm, Cont'd. [Link Download Link]
- [Lecture 21] Review[Link Download Link, Link Download Link]
- [Lecture 22] Midterm #2 (closed book) [Link Download Link]
-
Linear and non-linear least square methods
- [Lecture 23] Linear least square problems. [Link Download Link, pp 245-254]
- [Lecture 24] Nonlinear least square problems. [Link Download Link, pp 245-254]
-
Global search algorithms
- [Lecture 25] Genetic Algorithm. [Link Download Link][reference code Links to an external site.]
- [Lecture 26] Differential Evolution. [Link Download Link][reference code Links to an external site.]
- [Lecture 27] Review [Link Download Link]
- [Lecture 28] Review [Link Download Link]
- Final exam (closed book) [Link Download Link]
Assignments:
Projects:
- Project 0 [NO SUBMISSION]
- Project 1
- Project 2
- Project 3
- Project 4
- Project 5
- Project 6 Download Project 6