A challenging problem in both engineering and computer science is that of minimising a function for which we have no mathematical formulation available, that is expensive to evaluate, and that contains continuous and integer variables, for example in automatic algorithm configuration. Surrogate-based algorithms are very suitable for this type of problem, but most existing techniques are designed with only continuous or only discrete variables in mind. Mixed-Variable ReLU-based Surrogate Modelling (MVRSM) is a surrogate-based algorithm that uses a linear combination of rectified linear units, defined in such a way that (local) optima satisfy the integer constraints. Unlike other methods, it also has a constant run-time per iteration. This method outperforms the state of the art on several synthetic benchmarks with up to 238 continuous and integer variables, and achieves competitive performance on two real-life benchmarks: XG-Boost hyperparameter tuning and Electrostatic Precipitator optimisation.

, , ,
doi.org/10.1145/3449726.3463136
Genetic and Evolutionary Computation Conference
Centrum Wiskunde & Informatica, Amsterdam (CWI), The Netherlands

Bliek, L., Guijt, A., Verwer, S., & de Weerdt, M. (2021). Black-box mixed-variable optimisation using a surrogate model that satisfies integer constraints. In GECCO 2021 Proceedings (pp. 1851–1859). doi:10.1145/3449726.3463136