IKT719 - Advanced Optimization

Overview

This course targets competences in formulating, analyzing, and solving non-linear optimization problems. As opposed to the introductory course IKT720, here problems are not necessarily convex. The first part deals with formulation and theoretical analysis of optimization problems. The second part is concerned with algorithms.

Prerequisites

Linear algebra, analysis, convex optimization, MATLAB programming.

Syllabus

1. Smooth optimization problems and algorithms
2. Methods for optimization of non-smooth problems
3. Distributed and decentralized algorithms
4. Online optimization methods
5. Methods for global and non-convex optimization

Module 1

This is the module I teach, where the student learns to formulate optimization problems, analyze their properties, derive the optimality conditions to possibly obtain a closed form solution, apply an iterative method when necessary or convenient, an study its convergence.

Course Material

Since I could not find any reference collecting all the material that I deemed most relevant, I wrote these lecture notes, which have gradually grown over the years. These notes are intended to serve as i) material for self-study, ii) lecturing material, and iii) reference. To achieve this end, the contents are clearly structured and explicitly organized to the paragraph level.

Teaching Approach

This module is fast-paced and requires a certain degree of mathematical sophistication of the student. The contents expand prior knowledge that the student is assummed to have providing a more rigorous treatment and deeper insight relative to the previous course IKT720.

Since this topic may be difficult to digest, the student is expected to carefully study the lecture notes before each session. A collection of quiz questions throughout the notes prompt the student to reflect on the concepts presented there, review additional material when necessary, and stimulate persistence in the student's long-term memory. The answers to the quiz questions can be submitted before the corresponding lecture to obtain extra grade points.

The lectures then build the big picture and query students to both help them understand the core concepts and to detect those parts that were not understood. To some extent, this follows a flipped-classroom approach.

Main Bibliography

1. Daniel Romero, Advanced Optimization Lecture Notes.
2. Dimitri P. Bertsekas. Nonlinear Programming. Third edition, Athena Scientific, 2016.
3. Yurii Nesterov. Lectures on convex optimization. Springer, 2018.
4. Stephen P. Boyd, and Lieven Vandenberghe. Convex optimization. Cambridge university press, 2004.