Optimization direct methods- Simplex optimization and derivative methods

As we saw in the previous article [P228] "Direct optimization methods – Single-variable and Simplex methods", these methods are based on an optimization strategy well-suited to experimental phenomena; they consist in proceeding by successive iterations, starting from an initial experiment and converging towards an optimum zone. In the case of systems involving several factors, this is the Simplex method, a direct optimization method that does not require the development of a mathematical model. The principle of the method is to move away from the worst test, assuming that the direction taken will be the right one. Given the effectiveness of the initial method, some authors have recommended a number of modifications that take into account the response obtained with each new trial.

These methods concern :

• either the expansion or contraction of the simplex in the "right" direction: these are the methods of Nelder and Mead ("Modified Simplex") and Routh or Van der Wiel ("Super Modified Simplex");

• either optimizing the "right" direction, while also approaching the best trial (Weighted Centroid method)

• or the simultaneous elimination of several trials (Multi-Move method), considering that the group of trials is divided into two populations: the "good" group and the "bad" group, which will be eliminated in the next iteration.

This article describes

• the principles of evolution for each method. Application examples are given, and calculations are detailed for each application;

• an analysis and comparison of the various methods;

• a guide to choosing an optimization method.