【SI152笔记】part6：约束非线性优化问题

SI152: Numerical Optimization

Lecture 14: Quadratic Programming

Active-set Method

If an optimal active-set A∗ (i.e., a set of inequalities satisfied as equalities at a solution) is known in advance, then a solution x∗ can be found as a solution.

Suppose we have an iterate \(x^k\) and a guess \(A^k\) of an optimal active set. Compute \(d^k\) as the solution to the subproblem

\[\begin{aligned} &\min_{d}~ (x^k + d) + \dfrac{1}{2} (x^k +d)^T H (x^k +d) \\ \text{s.t. }~ & A_i (x^k +d) + b_i =0 , i\in \mathcal{E}\cup\mathcal{A}^k \end{aligned} \]

If x^k + d^k is feasible, then set \(x_{k+1} \gets x_k + d_k\) and let \(A_{k+1} \gets A_k\)
Else, set \(x_{k+1} \gets x_k + α_k d_k\), where \(α_k\) is the largest value such that \(x_{k+1}\) satisfies all constraints. Let \(A_{k+1}\) be the set of constraints active at \(x_{k+1}\).

Interior Point Method

Lecture 15: Penalty Methods

\[\min_x f(x) \text{ s.t. } c(x) = 0 \]

Quadratic Penalization

the unconstrained quadratic penalty subproblem：

\[\min_x \phi(x, \nu) = f(x) + \frac{\nu}{2} \lVert c(x) \rVert^2_2 \]

where \(ν ≥ 0\) is a penalty parameter. \(ν\to\infty\) when in iteration.

Exact Penalty Function

A penalty function \(φ(x; ν)\) is exact if there exists \(ν∗\) such that for all \(ν > ν∗\), a local solution of the constrained problem is a local minimizer of \(φ(x; ν)\).

\[φ(x; ν) := f(x) + ν \lVert c(x) \rVert _1 \]

Augmented Lagrangians

Alternating Direction Method of Multipliers

Lecture 16: Barrier Methods

\[\begin{aligned} &\min_{x}~ f(x) \\ \text{s.t. }~ & c_{\mathcal{E}} = 0 \\ & c_{\mathcal{I}} + s = 0, s\geq 0 \end{aligned} \]

The challenge the problem is ALL with the inequalities/bounds.

Thus, create a subproblem that “maintains” the bounds in an easier way:

\[\begin{aligned} &\min_{x}~ f(x) + \mu \sum_{i\in \mathcal{I}} \ln s_i \\ \text{s.t. }~ & c_{\mathcal{E}} = 0 \\ & c_{\mathcal{I}} + s = 0, s\geq 0 \end{aligned} \]

Solve for a sequence of barrier parameters such that \(µ\to 0\).

Lecture 17: Sequential Quadratic Programming (SQP)