# Optimization

Designing systems that optimize a set of metrics subject to constraints.

## The optimization process

minimize f(x) subject to x ∈ X

Minimize f(x) can be replaced by maximize -f(x)

``````                                           +------+
+---+Change<-----+
|   |Design|     |no
|   +------+     |
+---------+   +-----v-----+       +--+--+
Design         +--->  Initial+--->Evaluate   +------->Good?|
Specifications     |  Design |   |Performance|       |     |
+---------+   +-----------+       +--+--+
|
|yes
v
Final
Design
``````

Optimize with respect to data, as intuition can be misleading.

## Translating real world problems

There are some books describing the process to transform real world optimization problems to optimization problems

• Optimization: Algorithms and Applications (R.K. Arora)
• Optimization Concepts and Applications in Engineering (2nd edition, A. Keane, P. Nair)
• Computational Approaches for Aerospace Design (P.Y. Papalambros, D.J. Wilde)
• Principals of Optimal Design (Cambridge University Press, 2017)

## Constraints

Constraints can be numerical (for example x ⋝ 4) but should always include the boundary (in the example 4). Excluding it (x > 4), the solution can move infinitely close to 4 without ever reaching it, which means no solution can be found.

## Critical Points

A function f(x) may have a global minimum but may have multiple local minima. A zero derivative is a necessary condition for a local minimum but not a sufficient condition. The second derivative has to be >0, so the point is at the bottom of the bowl.