MotoFit Benchmarks

I recently noticed that the reflectivity software package Motofit ( by Andrew Nelson (ANSTO, Australia) was benchmarked against other common reflectivity software.

Andrew provides model data against which Motofit, Parratt32, drydoc, and reflfit were compared. Below is a table of how well the various fitting routines available in yanera works with that same data, and same inital conditions.

On the left is a list of the constrained and unconstrained fitting methods. By "constrained", I mean whether one can set maximum and minimum limits for a parameter. In all cases, a parameter can be held fixed. We record the number of iterations, the execution time (in minutes:seconds), and the final chi-squared of fit for each method.

Layer models

Shown in the left table all methods work well to fit the data using the multi-layer model type that Parratt32 and Motofit uses. Except for the steepest descent method, all reach a solution in about a minute. Even 1000 iterations of simulated annealing reaches a reasonable chi-squared quickly. Most remarkable are the Levenberg-Marquardt algorithms which converge and write to output in less than 2 seconds.

Layer models

Yanera, like drydoc software, approximates interface roughness with small slabs when fitting using analytical expressions of the scattering density. I set up the model to fit using the equation (a+b*z) for the very thick layer, with a=10x10^-6 and b=0.001x10^-6 as inital conditions. We know that b should be 0 in the final fit.

In the cases tested, fitting is much slower. Levenberg-Marquardt algorithms work well to achieve resonable chi-squared in about 8 minutes of compute time. It should be noted that they converge very rapidly at first, but slow down after the first few iterations. This is a consequence of the slow approach of b to 0.

The Neadler-Mead simplex method iterates much faster (since is doesn't take derivatives), but converges much more slowly. The final result by simulated annealing is comparable to the layer model, but takes much longer. So far, only one method has failed show much convergence.

Layer model

Function model

Iterations Time Chi-Sq
Iterations Time Chi-Sq

500 01:12.93 1.353

500 01:15.70 1.141

500 01:10.88 1.255

500 00:57.38 51.269

1500 00:07.03 1.145
1500 02:49.94 72.110
12 00:01.20 1.106

16 00:01.64 1.106
13 07:03.45 1.550

1000/50 01:38.38 2.375
1000/50 18:47.10 2.654
20 00:01.24 1.297

22 00:01.47 1.297

500 00:20.36 1.596

18 01:07.04 1.106

> 20:00.00
8 00:00.82 1.106
13 07:43.75 1.242

Starting Chi-Sq 5378.800

Starting Chi-Sq 1978.9

Generated on Thu May 29 10:56:33 2008 by  doxygen 1.5.5