Surrogate-assisted optimisation has proven success in the continuous domain, but only recently begun to be explored for other representations, in particular permutations. The use of Gaussian kernel-based models has been proposed, but only tested on small problems.
This case study considers much larger instances, in the experimental setting of a real-world ordering problem. We also investigate whether creating models using different distance metrics generates a diverse ensemble. Results demonstrate the following effects of use to other researchers: (i) Numerical instability in matrix inversion is a factor across all metrics, regardless of algorithm used. The likelihoood increases significantly once the models are parameterised using evolved solutions as well as the initial random pop- ulation; (ii) This phase transition is also observed in different indicators of model quality. For example, predictive accuracy typically decreases once models start to include data from evolved samples. We explain this transition in terms of the distribution of samples and Gaussian kernel basis of the models; (iii) Measures of how well models predict rank-orderings are less affected; (iv) Benchmark compar- isons show that using surrogate models decreases the number of evaluations required to find good solutions, without affecting quality.