Designing systems that optimize a set of metrics subject to constraints.

The optimization process

minimize f(x) subject to x ∈ X

Minimize f(x) can be replaced by maximize -f(x)

                                       |   |Design|     |no
                                       |   +------+     |
                   +---------+   +-----v-----+       +--+--+
Design         +--->  Initial+--->Evaluate   +------->Good?|
Specifications     |  Design |   |Performance|       |     |
                   +---------+   +-----------+       +--+--+

Optimize with respect to data, as intuition can be misleading.

Translating real world problems

There are some books describing the process to transform real world optimization problems to optimization problems

  • Optimization: Algorithms and Applications (R.K. Arora)
  • Optimization Concepts and Applications in Engineering (2nd edition, A. Keane, P. Nair)
  • Computational Approaches for Aerospace Design (P.Y. Papalambros, D.J. Wilde)
  • Principals of Optimal Design (Cambridge University Press, 2017)


Constraints can be numerical (for example x ⋝ 4) but should always include the boundary (in the example 4). Excluding it (x > 4), the solution can move infinitely close to 4 without ever reaching it, which means no solution can be found.

Critical Points

A function f(x) may have a global minimum but may have multiple local minima. A zero derivative is a necessary condition for a local minimum but not a sufficient condition. The second derivative has to be >0, so the point is at the bottom of the bowl.