Parameter Scaling

A second problem common to most optimisation problems is vast changes in sensitivity of the objective function to perturbations in different model parameters. This is mainly a practical problem caused by the limited precision of floating point representations of the parameters and objective function on computer systems.

The initialisation values of the parameters can also be of greatly differing sizes, which are not necessarily related to the typical expected variation in the parameters e.g. A coordinate for the fit of a direct beam mark may have a starting value of around 2000.0, but typically change by a several tenths of a pixel, whereas it's height might be of a similar initial value but change by a few hundred units, and the initial standard deviations of the Gaussian profiles modelling the peak may be of the order of a pixel, also with a typical variation of a few tenths of a pixel.

Whilst the values of the model parameter value changes are not known precisely, it is possible to estimate the magnitude of such changes with sufficient accuracy that a scaling transformation may be applied to the parameter values prior to the optimisation algorithm so that they all have approximately similar values. The inverse transformation is applied before calculation of the objective function. This allows the sensitivity of the objective function to be roughly equal to perturbations in the values of any of the parameters.

If ${\bf v}_i$ is the vector of parameter values at iteration i, so ${\bf v}_1$ is the vector of initialisation values, ${\bf C}$ is the diagonal matrix of typical sizes by which each parameter is expected to change, ${\bf p}_i$ is the vector of scaled parameter values which the minimisation algorithm sees at each iteration, and ${\bf s}$ is a translation vector between the scaled parameter values and un-scaled values, then all the initial scaled parameter values are set to an initial constant vector ${\bf t}$, and the translations are all set to ${\bf v}_1$. The scaled parameter values ${\bf p}$ are related to the un-scaled values by the following equation:

$${\bf p}_i = {\bf C}^{-1}({\bf v}_i - {\bf s}) + {\bf t}$$ (1)

The un-scaled parameter values are calculated from the scaled values by the following equation:

$${\bf v}_i = {\bf C} ({\bf p}_i - {\bf t}) + {\bf s}$$ (2)

These scaling relationships allow all the initial minimisation parameters to start with the same value, and to be equally sensitive to perturbations in their values. This form has been adopted to avoid the minimisation parameters having values close to zero.

For general function fitting default values are provided automatically for the typical change sizes c and these are usually quite adequate. These values may nevertheless be individually set by the user.

All scaled parameters are set initially to 100.0 and the scaling means that the typical variation of the scaled values is $\pm 1.0$ unit. This parameter transformation together with care taken in calculating the objective function allows all operations to be performed using single precision (32-bit) arithmetic with no evidence of numerical instability.